Power grid data intelligent processing method and system based on big data

Abstract

The invention relates to the technical field of power grid data intelligent processing, and discloses a power grid data intelligent processing method and system based on big data. Comprising the steps that load data, hardware performance parameters, operation state parameters and network delay values of edge nodes, data processing requirements of distributed energy access points, power grid operation log records, communication protocol data, bandwidth utilization rate data and cooperative feedback data are collected; carrying out grouping fusion based on the load, the performance and the delay, and determining a cooperation priority; judging load surge according to the priority and the operation state to form a cooperation degree parameter; generating a resource allocation scheme in combination with the priority and the bandwidth utilization rate; calculating a response speed index according to the distribution scheme and the log record; and performing prediction analysis on the response speed index and the communication protocol data to obtain a stable operation guarantee mechanism, performing iterative updating in combination with feedback data, and outputting a final cooperative adjustment result. According to the method, quick response and efficient cooperation of the power grid under the condition of sudden failure or load surge can be realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent power grid data processing technology, and in particular to a method and system for intelligent power grid data processing based on big data. Background Technology

[0002] Currently, the power grid, as the core infrastructure for energy supply in modern society, is crucial for ensuring people's livelihoods through its stability and efficiency. With the integration of new energy sources, surging electricity demand, and the widespread application of distributed energy, power grid data processing faces higher requirements for real-time performance and intelligence. However, existing methods often fall short in dynamic and complex scenarios, failing to meet the efficiency and reliability requirements of power grid operation.

[0003] In existing technologies, power grid data processing solutions often rely on fixed rules or preset models, resulting in a simplistic approach to assessing node computing power and network latency, and a lack of dynamic resource allocation. For example, in the event of regional faults or sudden load surges, traditional methods often fail to quickly assess the computing power and network status of each node, leading to unreasonable resource allocation and slow response times. Furthermore, existing solutions generally lack the ability to dynamically adjust the degree of cooperation among multiple nodes, making it difficult for the system to quickly formulate efficient collaborative strategies in the event of emergencies.

[0004] Existing technologies cannot assess node capabilities in real time and dynamically adjust collaboration, making it difficult to quickly formulate efficient solutions in case of emergencies. Summary of the Invention

[0005] This invention provides a method and system for intelligent processing of power grid data based on big data, so as to achieve rapid response and efficient coordination of the power grid in the event of sudden failure or load surge.

[0006] Firstly, in order to solve the above-mentioned technical problems, the present invention provides a method for intelligent processing of power grid data based on big data, comprising:

[0007] Collect load data, hardware performance parameters, operating status parameters, network latency values, data processing requirements of distributed energy access points, dynamic change records of power grid operation logs, communication protocol data between devices, bandwidth utilization data of communication networks, and device collaboration feedback data from the previous collaboration process of edge nodes.

[0008] Based on the load data, the hardware performance parameters, and the network latency value, grouping and fusion analysis is performed to obtain a priority sequence for inter-device collaboration;

[0009] Based on the priority sequence, the operating status parameters, and the network latency value, the degree of load surge is determined, and a cooperation level parameter is formed.

[0010] The data processing requirements and the collaboration level parameters are regrouped, and a resource allocation scheme is generated based on the priority sequence and the bandwidth utilization data.

[0011] Based on the resource allocation scheme and the dynamic change record, the real-time response speed is judged to obtain the response speed index;

[0012] By performing stable operation prediction on the response speed index and the communication protocol data, a stable operation guarantee mechanism is obtained;

[0013] Based on the stable operation guarantee mechanism and the equipment collaboration feedback data, and combined with the priority sequence, the final equipment collaboration adjustment result is obtained through iterative updates.

[0014] Secondly, the present invention provides a power grid data intelligent processing system based on big data, comprising:

[0015] The data acquisition module is used to collect load data, hardware performance parameters, operating status parameters, network latency values, data processing requirements of distributed energy access points, dynamic change records of power grid operation logs, communication protocol data between devices, bandwidth utilization data of communication networks, and device collaboration feedback data from the previous collaboration process of edge nodes.

[0016] The priority sequence module is used to perform grouping and fusion analysis based on the load data, the hardware performance parameters and the network latency value to obtain a priority sequence for inter-device cooperation.

[0017] The collaboration level module is used to determine the degree of load surge based on the priority sequence, the running status parameters, and the network latency value, and to form a collaboration level parameter.

[0018] The resource allocation module is used to regroup the data processing requirements and the cooperation level parameters, and generate a resource allocation scheme based on the priority sequence and the bandwidth utilization data.

[0019] The response speed module is used to determine the real-time response speed based on the resource allocation scheme and the dynamic change record, and obtain the response speed index.

[0020] The operation assurance module is used to perform stable operation prediction on the response speed index and the communication protocol data to obtain a stable operation assurance mechanism;

[0021] The equipment adjustment module is used to iteratively update the equipment collaboration adjustment result based on the stable operation guarantee mechanism and the equipment collaboration feedback data, combined with the priority sequence.

[0022] Thirdly, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the big data-based intelligent power grid data processing method described in any one of the above.

[0023] Fourthly, the present invention also provides a computer-readable storage medium comprising a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the above-described intelligent power grid data processing method based on big data.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] (1) This invention can more accurately reflect the comprehensive capabilities of nodes by grouping and fusing multi-dimensional information such as load data, hardware performance and network latency of edge nodes. Compared with the method that relies on fixed rules, this comprehensive analysis method enables the system to identify suitable nodes more quickly when facing sudden loads or failures, thereby improving the rationality of resource scheduling and the efficiency of task allocation.

[0026] (2) This invention introduces dynamic judgment of operating status parameters and feedback data during the collaboration process, and performs iterative optimization in combination with priority sequence, so that the degree of collaboration can be continuously adjusted with changes in the operating environment. As a result, the system can avoid performance degradation caused by rigid resource allocation in high-load scenarios, and enhance the continuity and stability of task processing.

[0027] (3) This invention forms a stable operation guarantee mechanism by predictively analyzing the response speed index and communication protocol data, and updates and optimizes parameters when an anomaly is detected, enabling the system to improve its adaptability to abnormal situations while ensuring real-time performance. Therefore, this method can achieve relatively stable equipment cooperation in complex power grid operating environments. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the intelligent power grid data processing method based on big data provided in the first embodiment of the present invention;

[0029] Figure 2 This is a schematic diagram of the structure of a power grid data intelligent processing system based on big data, provided in the second embodiment of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Reference Figure 1 The first embodiment of the present invention provides a method for intelligent processing of power grid data based on big data, including the following steps:

[0032] S11 collects load data, hardware performance parameters, operating status parameters, network latency values, data processing requirements of distributed energy access points, dynamic change records of power grid operation logs, communication protocol data between devices, bandwidth utilization data of communication networks, and device collaboration feedback data from the previous collaboration process of edge nodes.

[0033] S12, based on the load data, the hardware performance parameters and the network latency value, perform grouping and fusion analysis to obtain a priority sequence for inter-device collaboration;

[0034] S13, Based on the priority sequence, the operating status parameters, and the network latency value, determine the degree of load surge and form a cooperation level parameter;

[0035] S14, the data processing requirements and the cooperation level parameters are regrouped, and a resource allocation scheme is generated based on the priority sequence and the bandwidth utilization data;

[0036] S15, Based on the resource allocation scheme and the dynamic change record, perform a real-time response speed judgment to obtain a response speed index;

[0037] S16, perform stable operation prediction on the response speed index and the communication protocol data to obtain a stable operation guarantee mechanism;

[0038] S17. Based on the stable operation guarantee mechanism and the equipment collaboration feedback data, and combined with the priority sequence, the final equipment collaboration adjustment result is obtained through iterative updates.

[0039] In step S11, it is necessary to collect the load data, hardware performance parameters, operating status parameters, network latency values, data processing requirements of distributed energy access points, dynamic change records of power grid operation logs, communication protocol data between devices, bandwidth utilization data of communication networks, and device collaboration feedback data from the previous collaboration process, including:

[0040] In this embodiment, the load data refers to the real-time processing pressure generated by the edge node during task execution. During data collection, the length of the task queue, the task scheduling rate, and the number of completed tasks can be obtained by calling the process management interface in the node's operating system. Taking Linux as an example, the number of CPU tasks can be read from the ` / proc / stat` file, and then combined with the `sar` command to count the process scheduling frequency, thereby calculating the task load level. This data can intuitively reflect the task busyness of the node at different time periods.

[0041] In this embodiment, the hardware performance parameters are used to characterize the physical processing capabilities of the node. These parameters include processor clock speed, number of cores, memory capacity, and storage bandwidth. Data can be collected directly using system command-line tools; for example, under Linux, `lscpu` can be used to obtain CPU clock speed and number of cores, and `free -m` can be used to view memory capacity; under Windows, similar information can be obtained using the WMIC (Windows Management Instrumentation Command-line) command. The collected results are compared with the device's factory configuration file to ensure data accuracy.

[0042] In this embodiment, the operating status parameters include network data traffic rate and processor utilization, which reflect the real-time operating status of the node. Data traffic rate can be measured by deploying a traffic collector based on the NetFlow or sFlow protocol, counting the number of bytes and packets per unit time, and calculating the result in Mbps. Processor utilization can be collected in real time using the `top` or `vmstat` command, recording the CPU utilization percentage. This parameter effectively characterizes the resource usage status of the node.

[0043] In this embodiment, the network latency value represents the round-trip time (RTT) between a node and a target node or server. It is obtained by sending ICMP probe packets and calculating the RTT (Round Trip Time), for example, by executing a ping command to send several packets and taking the average; or by testing the connection latency through a TCP three-way handshake. The collected results are in milliseconds, reflecting the real-time communication quality of the current network link.

[0044] In this embodiment, the data processing requirements of the distributed energy access points are used to characterize the amount and type of data that different access points need to process per unit time. By installing data acquisition modules at the access points, the rate at which they upload power measurement data, equipment log data, and status monitoring data to the control center is statistically analyzed. For example, at a photovoltaic access point, the number of voltage, current, and power sampling points uploaded per second can be counted and converted to Mbps to clarify the real-time processing requirements of that access point.

[0045] In this embodiment, the dynamic change records of the power grid operation log are used to track time-series events related to node status during power grid operation. The logs are obtained by accessing the log server of the power grid management system, calling the Syslog protocol interface, and parsing the event types and timestamps in the logs, such as node power-on, power-off, load increase, and fault alarms. The raw text logs are then converted into structured records using a log parsing tool (such as Logstash), forming a dynamic change record set.

[0046] In this embodiment, the communication protocol data between devices is used to reflect the message interaction between nodes. During acquisition, communication packet capture tools (such as Wireshark) or log parsing modules can be used to parse messages of protocols such as Modbus, DNP3, and IEC61850, extracting message types, command content, and transmission timestamps. The parsed data can form a communication sequence, reflecting the interaction patterns between devices and potential latency.

[0047] In this embodiment, the bandwidth utilization data of the communication network is used to characterize the proportion of link bandwidth resources occupied. Specifically, the data is collected by deploying traffic monitoring tools, such as iftop, nload, or an SNMP-based network management system, on the network interface of the edge node. The uplink and downlink traffic values ​​of the interface per unit time are counted and divided by the maximum physical bandwidth of the link to calculate the bandwidth utilization percentage. This data can intuitively reflect the link's busy level.

[0048] In this embodiment, the device collaboration feedback data from the previous collaboration process is used to evaluate the effectiveness of the previous round of collaboration strategy. This data can be automatically generated by the task scheduling module after task execution, and includes indicators such as task completion rate, average task response latency, and communication packet loss rate. For example, the system records the proportion of tasks successfully completed by each node within a specified period to the total number of tasks, calculates the average response latency, and calculates the packet loss rate through message verification. This feedback data is collected through node log uploads or the task scheduler's status reporting mechanism, providing a basis for optimizing subsequent collaboration strategies.

[0049] In step S12, based on the load data, the hardware performance parameters, and the network latency value, a grouping and fusion analysis needs to be performed to obtain a priority sequence for inter-device collaboration, including:

[0050] The load data and the hardware performance parameters are grouped to obtain computing power indicators;

[0051] The computing power index and the network latency value are input into a preset fusion analysis model to generate fusion results;

[0052] Based on the fusion result, a sorting operation is performed to determine the collaboration priority of each edge node, thereby obtaining a priority sequence for inter-device collaboration.

[0053] In this embodiment, load data and hardware performance parameters are grouped to obtain computing power indicators. Specifically, the selected hardware parameters include processor clock speed, number of cores, memory capacity, and storage bandwidth, while the load data includes processor utilization, memory utilization, task queue length, and task completion rate. The acquisition period is set to 1 second. This setting is based on the conventional requirements of edge computing monitoring. It is well known in the art that the acquisition period is usually controlled between milliseconds and seconds to ensure that operational fluctuations are captured without causing acquisition redundancy. In this scenario, to balance data integrity and system overhead, the acquisition period is fixed at 1 second. Data statistics are averaged using a 60-second window. This setting is determined based on experimental statistical results. When analyzing historical operation logs, it was found that a 60-second window can balance short-term fluctuations and long-term trends. If the window is shortened to 30 seconds, the fluctuations are too large, and if it is extended to 120 seconds, the sensitivity is insufficient. Therefore, 60 seconds was ultimately selected as the statistical window.

[0054] For the above data, the K-means clustering method was used to group the nodes, with the number of clusters ranging from 2 to 5. This range is not a fixed empirical value, but rather the final number of groups was determined by dynamically calculating the silhouette coefficient using the algorithm. For example, when the number of candidate clusters is 2, 3, 4, and 5, the corresponding silhouette coefficients are calculated respectively. If the difference is less than 3%, the group with fewer clusters is selected to ensure grouping stability. If the difference is significant, the cluster with the highest silhouette coefficient is selected as the final result.

[0055] In this embodiment, a silhouette coefficient is used to determine the optimal number of groups. Specifically, the candidate range for the number of clusters is first set between 2 and 5. The clustering algorithm is run sequentially, and the clustering results are calculated for each number of clusters. The silhouette coefficient is obtained by comparing the average distance between nodes within the same cluster and the average distance between that node and nodes in adjacent clusters. Its value ranges from -1 to 1; the closer the value is to 1, the more compact the samples within the cluster and the more separated the clusters are. The system calculates the average silhouette coefficient for each cluster number and selects the cluster number with the largest value as the final number of groups.

[0056] For example, the silhouette coefficient is 0.55 when the number of clusters is 2, 0.62 when the number of clusters is 3, 0.64 when the number of clusters is 4, and 0.63 when the number of clusters is 5. Therefore, 4 is ultimately chosen as the number of groups. To avoid over-segmentation of samples when the scores of multiple clusters are close, this embodiment sets a relative difference threshold through experimental statistics. That is, when the difference in the silhouette coefficient of the candidate clusters is less than 3%, the case with fewer clusters is preferred. This threshold is shown through multiple clustering experiments on historical running data. When the difference in the number of clusters is within 3%, further increasing the number of clusters does not significantly improve the classification effect. On the contrary, it will lead to increased computational complexity and too few samples in some clusters. Therefore, 3% is taken as a reasonable balance value. Through this rule, the grouping results can fully reflect the differences in node performance while maintaining the stability of the model and the efficiency of computation.

[0057] In this embodiment, when the difference in average silhouette coefficient under different cluster numbers is less than 3%, for example, when the number of clusters is 3, the silhouette coefficient is 0.61 and when the number of clusters is 4, the silhouette coefficient is 0.62. The difference between the two is only 0.01, which is much less than the 3% difference threshold. At this time, the system will prioritize the case with fewer clusters, that is, select 3 as the final number of clusters, in order to reduce the instability caused by over-division.

[0058] During clustering, each feature is assigned a fixed weight, specifically based on the following criteria: processor clock speed and number of cores each account for 25%, determined by common knowledge in the field that computing power primarily depends on the processor's clock frequency and the number of parallel cores. These two factors contribute the most to performance in edge computing scenarios, hence their high weight. Memory capacity accounts for 10%, a proportion based on consensus that memory capacity has a marginal effect on node computing performance, and its contribution is typically lower than that of processor cores. Simply increasing memory has limited impact on overall performance, and its contribution is usually... Around 10%; processor utilization and task completion rate each account for 15%, which is the result of dynamic calculation. The system compares multiple candidate weight combinations (such as 10%, 15%, 20%) and calculates the silhouette coefficient after clustering. Finally, it finds that 15% of both results in the best clustering effect. Other parameters such as memory utilization, task queue length, and storage bandwidth are set to 10%, 5%, and 5%, respectively. The basis for this division is that although these features will affect node performance, their contribution to the overall experimental statistics is lower than that of clock speed, number of cores, and processor utilization, so they are given lower weights.

[0059] This hierarchical weighting method highlights the importance of hardware performance and key load parameters while preventing a single feature from having an excessive impact on the results. After grouping, each node is assigned to a specific cluster, and its weighted distance to the cluster center is calculated. The smaller the distance, the closer its overall performance is to the high-performance center. For example, a node with a weighted distance of 0.12 is classified as belonging to the high-performance cluster; another node with a weighted distance of 0.38 is classified as belonging to the medium-performance cluster. The system normalizes these grouping and distance results to the range of 0 to 1, forming a numerical computational capability index for subsequent fusion analysis with network latency.

[0060] In this embodiment, computational capability metrics and network latency values ​​are input into a preset fusion analysis model to generate a fusion result. The fusion analysis model is constructed using a multilayer perceptron neural network, and the input data only includes the computational capability metrics and network latency values ​​of the nodes. The input data needs to be standardized before entering the model. The computational capability metrics are normalized between 0 and 1, and the latency values ​​are recorded in milliseconds. These values ​​are adjusted to an acceptable range for the model through logarithmic transformation and standard deviation normalization. The model contains two hidden layers: the first layer has 64 nodes, and the second layer has 32 nodes. Both layers use a modified linear function as the activation function. The output layer is a normalized score ranging from 0 to 1; a larger value indicates that the node is more suitable for high-priority tasks. During the training phase, the model uses historical log data from the past 30 days as samples and is trained using the Adam optimization algorithm. The learning rate is set to 0.001, the batch size is 256, the maximum number of iterations is 50, and training is terminated early if there is no improvement after 5 consecutive iterations.

[0061] In real-world environments where sufficient historical data is lacking or model accuracy is insufficient, a weighted scoring method can be used as an alternative. This method, during its design, reverses the latency value, for example by using the reciprocal of the normalized latency value (1 / latency) or by subtracting the node's latency value from the maximum latency value, ensuring that a smaller latency value results in a higher score, maintaining logical consistency with the computing power metric. In the weighting process, the computing power metric is weighted at 70%, and the network latency value at 30%. This weighting is based on statistical analysis of historical operational samples: in a set of test data covering 30 days, the correlation coefficient between the computing power metric and task completion rate was approximately 0.72, while the correlation coefficient between network latency value and task completion rate was approximately 0.31. The former contributes significantly more, hence its higher weight. This weighting method can directly generate a fused score without relying on a complex model, used to determine the priority of nodes in collaboration.

[0062] In this embodiment, a ranking operation is performed based on the fusion results to determine the collaboration priority of each edge node. Specifically, all nodes are sorted in descending order according to the numerical value of the fusion results to obtain a priority sequence. To avoid frequent ranking fluctuations, a ranking stability threshold of three is set, meaning that the ranking change of a node between two adjacent sampling periods does not exceed three positions. If it does, only a maximum increase or decrease of two positions per second is allowed. Nodes with a fusion result difference of less than 2% are considered to be in a tie. In the case of a tie, the node with lower latency is selected first, followed by the node with stronger computing power. If they are still in the same position, the node with lower bandwidth utilization is selected. In the final output priority sequence, the identifier of each node, the fusion result, the computing power index, the latency value, and the final ranking are recorded as inputs for subsequent collaboration level calculations and resource allocation.

[0063] In step S13, it is necessary to determine the degree of load surge based on the priority sequence, the running status parameters, and the network latency value, and form a cooperation level parameter, including:

[0064] When the data flow rate in the running status parameters exceeds the preset rate threshold, the data flow rate and the processor utilization rate in the running status parameters are weighted and calculated in combination with the priority sequence to obtain the load surge value.

[0065] The load surge value and the network latency value are fused and analyzed to form a cooperation level parameter.

[0066] In this embodiment, the preset rate threshold is determined comprehensively based on the power grid operation scenario and historical operation data. The system first collects data traffic records for each edge node over the past thirty days, calculates the average and peak traffic values ​​for that node under normal operating conditions, and then selects a reasonable range between the two as a reference. To ensure the threshold's adaptability, the system sets the threshold at 80% to 90% of the peak traffic. For example, in a typical distribution area, the peak traffic of a single node is approximately 750 megabits per second, and the average is approximately 400 megabits per second. Therefore, the threshold can be set to 600 megabits per second, which is higher than the normal average load and can reflect potential overload trends in advance. For different types of nodes, such as backbone nodes, access nodes, or distributed energy access points, the threshold can be set within the range of 500 to 900 megabits per second to adapt to the needs of different business scenarios.

[0067] In this embodiment, when the data traffic rate of a node exceeds a preset threshold, a weighted calculation is performed on the data traffic rate and processor utilization rate using a priority sequence to calculate the load surge level. To avoid the dominance effect between data of different dimensions, the original data first needs to be normalized. Specifically, the data traffic rate is converted to Gbps and normalized to the maximum statistical rate of 10Gbps using a minimum-maximum normalization method, i.e., the rate value is divided by 10 to convert it to the range of 0 to 1; the processor utilization rate is naturally between 0% and 100%, so it can be directly normalized to the range of 0 to 1 by dividing by 100. This ensures that the traffic rate and processor utilization rate are comparable within the same numerical range, avoiding the situation where traffic characteristics completely mask CPU characteristics.

[0068] After normalization, the system performs a weighted calculation according to set weights, with data traffic rate weighted at 60% and processor utilization rate weighted at 40%. This weighting is based on experimental statistics of historical operational data: in a 30-day log sample, the correlation coefficient between node task failure rate and data traffic rate was 0.68, while the correlation coefficient with processor utilization rate was 0.45. The former has a greater impact on load surges and therefore receives a higher weight in the weighting. For example, a node's original data traffic rate is 8.2Gbps, which is normalized to 0.82; its processor utilization rate is 70%, which is normalized to 0.70. The weighted result is 0.82 × 0.6 + 0.70 × 0.4 = 0.768. This result serves as a load surge level value; the closer the value is to 1, the higher the load risk state of the node in the current period, thus requiring appropriate adjustments in subsequent resource allocation.

[0069] In this embodiment, the load surge value and network latency value are then fused and analyzed to generate a cooperation level parameter. Network latency values ​​are recorded in milliseconds. Since the latency differences between different nodes are significant, directly using these values ​​would lead to calculation results biased towards extreme values; therefore, standardization is required first. Specifically, the system calculates the minimum and maximum latency of all nodes within a fixed time window (e.g., five minutes), and maps the latency value of a given node to a range of zero to one using an interval scaling method. For example, if the minimum latency observed in the current window is five milliseconds and the maximum is fifty milliseconds, then when the latency of a given node is twelve milliseconds, the standardized value is approximately 0.16.

[0070] After standardization, the latency value is weighted and fused with the load surge value obtained in the previous step. To balance the impact of computational load and communication latency, the weights are set to 50% each, meaning the load surge value and the latency standardization value contribute equally to the fusion process. For example, if a node's load surge value is 0.78 and its latency standardization value is 0.52, the fused result is 0.65. This result is the collaboration level parameter, used to comprehensively reflect the node's operational pressure. A higher value indicates that the node is under greater pressure in the current time period, and the system will reduce the task allocation ratio for that node in subsequent resource allocation stages; a lower value indicates that the node still has remaining processing capacity and can undertake more tasks, thereby achieving more reasonable collaboration and scheduling.

[0071] In step S14, the data processing requirements and the cooperation level parameters need to be regrouped, and a resource allocation scheme needs to be generated based on the priority sequence and the bandwidth utilization data, including:

[0072] Clustering operations are performed on the data processing requirements and the collaboration level parameters to obtain the number of groups;

[0073] When the number of groups exceeds a preset group number threshold, a weighted calculation is performed on the groups by combining the priority sequence and the bandwidth utilization data to obtain the bandwidth allocation parameters.

[0074] Based on the bandwidth allocation parameters, differentiated bandwidth allocation is performed to determine the resource allocation scheme.

[0075] In this embodiment, clustering operations are first performed on the data processing requirements and collaboration level parameters. The data processing requirements reflect the data transmission rate of the distributed energy access point per unit time. For example, in a photovoltaic access scenario, the amount of voltage, current, and power sampled data transmitted per second can be counted, ranging from tens of kbps to one or two Mbps. The collaboration level parameter reflects the stress level of a node under current load and latency conditions, with a value ranging from zero to one; a larger value indicates greater stress. To ensure comparability of data with different dimensions in the clustering operation, standardization is required first. The standardization method is interval scaling, i.e., within a five-minute window, the minimum and maximum values ​​of the data processing requirements are counted, and the value of a node is mapped to the range of zero to one by subtracting the minimum value and then dividing by the interval difference. The collaboration level parameter itself is already within the range of zero to one, so no further processing is required.

[0076] After standardization, the two types of data are input into the K-means clustering algorithm for further grouping. The number of candidate groups is set to two to five, and clustering is run one by one. In each case, the average silhouette coefficient is calculated to measure the grouping effect. When the distribution differences between different clusters are significant, as indicated by an average silhouette coefficient greater than 0.6 and a difference greater than 0.2 between the center points of different clusters on the standardized coordinates, the system will select three or four groups, as this effectively distinguishes high-demand, medium-demand, and low-demand nodes. When the differences are small, specifically, as indicated by an average silhouette coefficient less than 0.55 and a difference less than 0.15 between the cluster centers, it indicates that the processing needs and collaboration pressure of the nodes are relatively similar. In this case, the system will select two groups to avoid unnecessary over-division. This method ensures that the grouping reflects the differences between nodes without causing overly fine grouping due to insufficient differences, thus ensuring the stability and rationality of the grouping results.

[0077] In this embodiment, when the number of regroups exceeds a preset threshold, a weighted calculation of the grouping needs to be performed by combining the priority sequence and bandwidth utilization data. The preset grouping threshold can be set to two based on the business scenario; that is, when the grouping result is three or more groups, it indicates a significant difference in data processing needs and collaboration pressure among the nodes, requiring further refinement of the allocation strategy. The priority value ranges from zero to one, with higher values ​​indicating more important nodes. For example, nodes above 0.8 are critical nodes, those between 0.5 and 0.8 are intermediate nodes, and those below 0.5 are auxiliary nodes. The bandwidth utilization rate ranges from zero to one hundred percent, with lower values ​​indicating less idle links. Therefore, reverse normalization is required before calculation, mapping utilization rates below 30% to high values, 30% to 70% to medium values, and above 70% to low values. Subsequently, a weighted calculation is performed based on a 60% weighting for the priority value and a 40% weighting for the reversed bandwidth utilization rate value to obtain the bandwidth allocation parameters. For example, if a node has a priority value of 0.8 and a bandwidth utilization rate of 70%, after reverse processing it becomes 0.45, and the weighted result is 0.66. This value is used as a bandwidth allocation parameter, indicating that the node should receive relatively more bandwidth resources in subsequent resource allocations.

[0078] In this embodiment, differentiated bandwidth allocation is performed based on bandwidth allocation parameters to determine the resource allocation scheme. Specifically, the system first normalizes the bandwidth allocation parameters of each group so that the sum of all allocation parameters is 1, and then allocates the total bandwidth to each group proportionally. This ensures the operability of the allocation result. Taking a power communication link with a total link bandwidth of 200Mbps as an example, if the average allocation parameter of the high-load, high-priority group is 0.8, the medium-load group is 0.15, and the low-load group is 0.05, then after normalization, the proportions are 80%, 15%, and 5%, respectively. Based on this, the system divides the total bandwidth into 160Mbps, 30Mbps, and 10Mbps, and allocates them to the corresponding groups. Within each group, the system further allocates the actual bandwidth according to the proportion of each node's allocation parameters. For example, within the high-priority group, if the allocation parameters of the three nodes are 0.5, 0.3, and 0.2, then the corresponding allocations are 80Mbps, 48Mbps, and 32Mbps. This top-down allocation method ensures a reasonable distribution of overall bandwidth across different groups and allows for fine-grained allocation within groups, thereby ensuring that critical nodes receive sufficient bandwidth while nodes with lighter loads or lower priorities relinquish resources appropriately.

[0079] Regarding bandwidth settings, the data selection in this embodiment is based on the actual situation of power communication: the common bandwidth requirements of power IoT terminals are typically between tens of kbps and several Mbps. Even in high-definition video surveillance scenarios, single-channel video transmission generally does not exceed 10–20 Mbps. Therefore, setting the total link bandwidth to the order of several hundred Mbps is more in line with the typical configuration of power wireless private networks or fiber optic networks, avoiding the unrealistic situation where a single node occupies hundreds of Mbps or even Gbps. Through this setting, the example data can cover most power terminal application scenarios, including telemetry, control commands, protection signals, and video surveillance, thereby ensuring the feasibility of the method.

[0080] In step S15, based on the resource allocation scheme and the dynamic change record, a real-time response speed judgment needs to be performed to obtain a response speed index, including:

[0081] The bandwidth allocated to each node is determined according to the resource allocation scheme.

[0082] Based on the task issuance timestamp and task completion timestamp recorded in the dynamic change record, and combined with the bandwidth value, the effective response latency of each node is calculated;

[0083] The effective response delay is statistically analyzed over time to obtain the average response delay of each node;

[0084] The average response delay is compared with a preset threshold to generate a response speed index.

[0085] In this embodiment, the bandwidth allocated to each node needs to be determined first according to the resource allocation scheme. The resource allocation scheme is generated by the system in step S14 and includes the total bandwidth ratio of each group and the allocation parameters of each node within the group. The specific operation process is as follows: the system first normalizes the bandwidth allocation parameters of each group so that the sum of the parameters of all groups is one, and then multiplies the ratio by the total link bandwidth to obtain the total bandwidth of the group; then, it performs a second normalization according to the bandwidth allocation parameters of each node within the group to calculate the allocation ratio of the node within the group, and finally multiplies the ratio by the total bandwidth within the group to obtain the actual allocated bandwidth of the node.

[0086] For example, if the total link bandwidth is 200 Mbps, and the bandwidth allocation ratio for the high-load group is 80%, the medium-load group is 15%, and the low-load group is 5%, then the total bandwidths for the three groups are 160 Mbps, 30 Mbps, and 10 Mbps, respectively. Within the high-load group, if there are three nodes with allocation parameters of 0.5, 0.3, and 0.2, the 160 Mbps bandwidth of this group will be further divided according to a 5:3:2 ratio, resulting in 80 Mbps, 48 ​​Mbps, and 32 Mbps. In this way, the bandwidth value for each node is calculated through a two-layer allocation, taking into account both the importance between groups and the differences between nodes within a group.

[0087] In this embodiment, the original response latency of the task is then calculated based on the task issuance timestamp and task completion timestamp saved in the dynamic change record. The original latency consists of multiple parts, including transmission latency, queuing latency, and processing latency, among which transmission latency is closely related to the allocated bandwidth. For small data volume, high real-time tasks (such as control commands and remote signaling changes), the response latency should be directly calculated by subtracting the task issuance timestamp from the task completion timestamp, or it is only affected by network latency (RTT) and node processing capabilities, and is basically unrelated to the allocated bandwidth. The direct impact of insufficient bandwidth is packet loss, rather than a linear increase in latency. For large data volume tasks (such as file transfer and video streaming), a correction based on queuing theory or throughput models can be introduced. The task requirement refers to the total amount of data to be transmitted to complete the task, rather than a fixed bandwidth rate. The transmission latency formula can be approximated as the data volume divided by the effective throughput, where the effective throughput is affected by the allocated bandwidth, packet loss rate, and protocol efficiency. In actual deployment, to simplify the calculation, the impact of the allocated bandwidth can be mainly considered, that is, the transmission latency is approximated as the data volume divided by the allocated bandwidth.

[0088] For example, the original latency for a node to complete a task under a bandwidth of 50 Mbps is 100 milliseconds. When the system increases the bandwidth of this node to 80 Mbps according to the resource allocation scheme, the original latency is 50% too high, and the corrected effective latency is 25 milliseconds. Conversely, when the allocated bandwidth decreases from 80 Mbps to 50 Mbps, the original latency is about 50% too low, and the system calculates an effective latency of 100 milliseconds. In this way, the latency calculation results can be ensured to remain comparable under different bandwidth conditions, avoiding interference from bandwidth changes on performance evaluation.

[0089] In this embodiment, the effective response latency of each node is then statistically analyzed over time to form a stable performance indicator. Specifically, the system collects effective response latency data for all tasks within a one-minute statistical window and averages it to obtain the node's average response latency. To avoid interference from individual abnormal tasks, the latency data is preprocessed during the statistical process, removing extreme values ​​exceeding three standard deviations of the mean. The statistical results reflect the overall service level of the node within that time period. For example, if a node completes one thousand tasks in one minute, with effective response latencies ranging from twenty to fifty milliseconds, after removing five outliers exceeding one hundred milliseconds, the system calculates an average response latency of thirty-five milliseconds. If the statistical results for multiple consecutive windows remain between thirty and forty milliseconds, it indicates that the node's response performance is stable; if the average response latency exceeds fifty milliseconds, it indicates that the node has insufficient bandwidth or excessive load, requiring subsequent resource scheduling adjustments.

[0090] Finally, the average response latency is compared with a preset real-time requirement threshold to generate a response speed metric. The threshold is set according to the business scenario; for example, in a real-time power grid control scenario, the threshold can be set to forty milliseconds. When the average response latency of a node is less than the threshold, the response speed metric generated by the system is considered acceptable; when the average response latency exceeds the threshold, it is marked as unacceptable, triggering subsequent bandwidth adjustments or priority reordering. For example, when the average response latency of a node is thirty-five milliseconds, the response speed metric is "acceptable," while when the latency is fifty-five milliseconds, the response speed metric is "unacceptable."

[0091] In this embodiment, the average response latency is finally compared with a preset real-time requirement threshold to generate a response speed index. The threshold is set based on the power grid operation scenario and historical data. The specific implementation is as follows: First, the system statistically analyzes the effective response latency of each node during the past thirty days of operation, constructing a latency distribution curve. Based on this, the percentile of this distribution is selected as the threshold, for example, the 95th percentile latency value is taken as the real-time requirement threshold. This covers most normal tasks while also enabling timely identification of abnormal situations. Second, different threshold ranges are set according to different business scenarios. For example, in real-time power grid control scenarios, the latency tolerance for control commands is low, and the threshold can be set between 30 and 50 milliseconds; in power grid operation monitoring or data acquisition scenarios, the real-time requirement is relatively low, and the threshold can be relaxed to between 100 and 200 milliseconds. Finally, during the deployment phase, the system dynamically corrects the threshold based on actual test results. The specific approach is as follows: First, run the test for one week with 40 milliseconds as the initial threshold. If the proportion of tasks exceeding the threshold is consistently below 5%, it indicates that the threshold is too strict and can be appropriately increased to 50 milliseconds. Conversely, if the proportion of tasks exceeding the threshold exceeds 15%, it indicates that the threshold is too lenient and needs to be lowered to 35 milliseconds.

[0092] In step S16, it is necessary to perform stable operation prediction on the response speed index and the communication protocol data to obtain a stable operation guarantee mechanism, including:

[0093] The response speed metric and the communication protocol data are used to construct a prediction input vector;

[0094] The predicted input vector is input into a preset multilayer perceptron model for feature extraction and nonlinear mapping, and the stability prediction result is output.

[0095] Based on the stability prediction results, a stable operation guarantee mechanism is generated to determine the stability status of inter-device communication and task processing.

[0096] In this embodiment, the response speed index and communication protocol data are first used to construct a prediction input vector. The response speed index, obtained in step S15, includes the node's average response latency, latency standard deviation, and pass / fail flags. The average response latency is in milliseconds and is mapped to a range of zero to one through interval scaling. For example, within a set statistical window, the minimum latency is 20 milliseconds and the maximum latency is 100 milliseconds; therefore, the average latency of a node of 35 milliseconds is converted to 0.1875. The latency standard deviation is also normalized to reflect the degree of fluctuation. The pass / fail flag is represented by binary encoding, with one for pass and zero for fail. Communication protocol data is obtained through log parsing and includes multiple dimensions. For example, the protocol type is represented by one-hot encoding, such as the Modbus protocol corresponding to the vector [1,0,0], and the IEC61850 protocol corresponding to [0,1,0]. The message length is calculated in bytes as an average of one minute and scaled to zero to one using minimum-maximum normalization. The message transmission interval is the average interval time of consecutive messages and normalized. The packet loss rate is directly converted to a value between zero and one as a percentage; for example, a 5% packet loss rate corresponds to 0.05. The number of retransmissions is counted within one minute and normalized according to a preset maximum value; for example, if the upper limit is set to one hundred times, then twenty retransmissions are converted to 0.2. Finally, the system concatenates the above standardized values ​​into a uniformly formatted input vector in a fixed order. For example, a node's response speed indicator provides three features, and the communication protocol data provides five features; combining the two forms a vector of length eight. If there are more protocol types and log features, the length of the input vector can be extended to twenty to thirty dimensions. This input vector serves as the input layer data for the multilayer perceptron model, and is used for subsequent feature extraction and stability prediction.

[0097] In this embodiment, the predicted input vector is then input into a preset multilayer perceptron model for feature extraction and nonlinear mapping. The model consists of an input layer, two hidden layers, and an output layer. The number of nodes in the input layer is the same as the dimension of the predicted input vector. The number of nodes in the hidden layers are 64 and 32, respectively, both using a modified linear function as the activation function. The output layer is a single node, and the Sigmoid function is used to compress the result to between 0 and 1.

[0098] The data required for model training comes from historical power grid operation logs, covering at least 30 days of actual operation samples. Each sample includes the node's response speed indicators and communication protocol characteristics within a certain time window as input vectors, while its corresponding stability state is determined by objective thresholds. Specifically, when the average response latency is less than 40ms and the packet loss rate is less than 2%, it is marked as "normal"; when the response latency is between 40 and 60ms or the packet loss rate is between 2% and 5%, it is marked as "risky"; when the response latency exceeds 60ms or the packet loss rate exceeds 5%, it is marked as "unstable". This threshold-based automatic labeling method avoids interference from subjective human factors and makes the labels repeatable.

[0099] To address the imbalance issue in power grid operation data, where "normal" samples far outnumber "unstable" samples, this embodiment introduces a data resampling and class weighting strategy during model training: oversampling methods (such as SMOTE) are used to expand the data volume for minority class samples, while "risk" and "unstable" samples are assigned higher weights in the loss function; for example, the weight of "normal" samples is set to 1, the weight of "risk" samples to 2, and the weight of "unstable" samples to 3. This approach effectively alleviates the bias caused by class imbalance, enabling the model to more accurately identify risk states and potential instability.

[0100] During training, historical data was first divided into training and validation sets at an 8:2 ratio to ensure the model could learn most features while also validating its generalization ability on independent data. The Adam optimization algorithm was used for training, with an initial learning rate of 0.001, a batch size of 256, and a maximum of 50 iterations. Cross-entropy was used as the loss function during training. After each iteration, the system calculated the accuracy on the validation set; if there was no improvement after five consecutive iterations, training was terminated early to prevent overfitting. After training, the model achieved a prediction accuracy of over 85% and could effectively distinguish between normal nodes and potentially risky nodes.

[0101] The model outputs a stability prediction result between zero and one. A value closer to one indicates a more stable operating state for the node, while a value closer to zero indicates a risk of instability. For example, when a node's output is 0.85, the system determines it to be in a stable state; a result of 0.6 is marked as a risky state, suggesting the need to increase bandwidth or adjust priority; and a result of 0.35 is determined to be an unstable state, requiring immediate triggering of an early warning or switching to a backup node. Ultimately, the system generates a stability assurance mechanism based on this prediction result, providing a reference for the safe operation of the power grid under dynamic loads and complex communication environments.

[0102] In this embodiment, a stable operation guarantee mechanism for determining the stability status of inter-device communication and task processing needs to be generated based on the stability prediction results. To this end, the system pre-sets a stability threshold range, which is determined by statistically analyzing historical operating data and combining it with business requirements. Specifically, during the power grid's trial operation phase, node operating data for thirty consecutive days is collected, and the distribution of all prediction results is calculated. The system selects the 95th percentile as the upper limit of the stable state and the 5th percentile as the lower limit of the unstable state. Based on this, adjustments are made according to the power grid business scenario. For example, in a real-time control scenario, the system uses 0.7 as the lower limit of the normal state, 0.4 to 0.7 as the risk state range, and anything below 0.4 is defined as an unstable state. This setting ensures that most nodes operate within the normal range while effectively distinguishing potential risks.

[0103] During operation, when the predicted value of a node is 0.85, the system classifies it as being in a stable state, and the node can maintain its original resource allocation strategy. When the predicted value is 0.6, it falls into the risk state range, and the system automatically increases the redundant bandwidth allocation ratio for that node, for example, by adding 20% ​​of the spare resources to the original allocation, or by moving it up one level in the priority ranking. When the predicted value is 0.35, which is below the instability threshold, the system immediately triggers the stability protection mechanism, including issuing a fault warning signal, forcibly reallocating bandwidth, and transferring critical tasks to backup nodes. In this way, it is possible to predict and dynamically protect against communication anomalies and task delay risks in advance, ensuring that the power grid can still maintain safe and stable operation in the event of an emergency.

[0104] In step S17, based on the stable operation guarantee mechanism and the equipment collaboration feedback data, and combined with the priority sequence, iterative updates are needed to obtain the final equipment collaboration adjustment result, including:

[0105] Based on the aforementioned stable operation guarantee mechanism, the guarantee mechanism parameters are adjusted to generate an updated guarantee mechanism;

[0106] The update guarantee mechanism is optimized by combining the device collaboration feedback data to form a collaboration adjustment plan;

[0107] The collaborative adjustment schemes are sorted according to the priority sequence, and resource allocation is optimized through load balancing calculations to output the final device collaborative adjustment results.

[0108] In this embodiment, the parameters of the stability guarantee mechanism need to be adjusted first based on the stability guarantee mechanism. The system dynamically corrects key thresholds related to communication and task execution based on the stability prediction results output in step S16. Specifically, if the prediction result of a node falls into the risk range three times consecutively (i.e., between 0.4 and 0.7), the system considers it to be in a potentially unstable state and requires parameter optimization in advance. For example, the communication packet loss rate threshold is initially set to 5% to determine whether communication is normal; if the node prediction result remains in a risky state, it indicates that the existing threshold is too lenient and masks potential problems. In this case, the system will lower the packet loss rate threshold to 3% to more rigorously screen abnormal links. Similarly, the tolerance for task response latency is initially set to fifty milliseconds. If the node prediction result shows that its latency performance is unstable, the system will tighten the tolerance threshold to forty milliseconds to ensure that underperforming nodes can be identified and eliminated more quickly in subsequent collaborations.

[0109] In addition, the guarantee mechanism parameters also include the number of retransmissions and the proportion of redundant bandwidth. When the prediction result indicates that a node is approaching an unstable state, the system will reduce the allowed number of retransmissions from three to two to reduce the latency caused by excessive retries; at the same time, it will increase the redundant bandwidth allocation ratio of that node from 10% to 20% so as to maintain a certain data throughput capacity even when the communication quality deteriorates.

[0110] In this embodiment, the update guarantee mechanism then needs to be optimized based on device collaboration feedback data. Device collaboration feedback data is collected by the system after the previous task allocation and execution, and includes three core dimensions: task completion rate, resource utilization rate, and actual node latency performance. The task completion rate is obtained by statistically analyzing the ratio of the number of tasks allocated to the node to the actual number completed. For example, if 950 out of 1,000 tasks are completed, the task completion rate is 95%. Resource utilization rate is calculated by monitoring processor and memory usage. For example, if CPU usage is 80% and memory usage is 70%, the system can calculate an overall resource utilization rate of 75%. The actual node latency performance is calculated by statistically analyzing the time difference between task issuance and completion, and compared with the corrected latency from the previous step to verify the node's latency level in actual operation.

[0111] During optimization, the system performs threshold checks on the feedback data. When the task completion rate is below 90% for two consecutive statistical windows while the resource utilization is above 80%, it indicates that the node is overloaded. The system will reduce the node's collaboration weight in the update guarantee mechanism, for example, lowering it from 0.7 to 0.5, to reduce its allocation in the next round of tasks. Conversely, when the node's task completion rate exceeds 95% and the resource utilization is below 60%, it indicates that the node has redundant capacity. The system will increase its collaboration weight, for example, from 0.5 to 0.65, to fully utilize its remaining computing resources. Regarding actual latency performance, if a node's average response latency exceeds the 40-millisecond threshold, even if the task completion rate is acceptable, it will be identified as a potential bottleneck. The system will prioritize allocating lower-priority tasks to this node in the collaboration adjustment scheme.

[0112] In this embodiment, the collaboration adjustment schemes are finally sorted according to priority sequence, and resource allocation is further optimized through load balancing calculations. Specifically, the system first comprehensively sorts the collaboration weight and priority value of each node. The priority value ranges from zero to one, with higher values ​​indicating greater importance in task scheduling. For example, nodes with a priority value of 0.9 or higher are classified as critical nodes, 0.7 to 0.9 as important nodes, 0.5 to 0.7 as minor nodes, and those below 0.5 as auxiliary nodes. After sorting, the system dynamically adjusts bandwidth and the number of tasks based on the node's level and real-time resource consumption when allocating resources.

[0113] During the allocation process, the system first allocates basic bandwidth to critical nodes and provides an additional 20% to 30% of guaranteed bandwidth to ensure stable operation under high load. For example, when the total link bandwidth is 200Mbps, a node with a priority value of 0.95 would normally be allocated 40Mbps, but the system will add a 30% guarantee, resulting in an allocation of 52Mbps. In contrast, nodes with priority values ​​below 0.5 maintain only the minimum guaranteed bandwidth, such as 2Mbps, for handling low real-time tasks. For nodes with intermediate priority, the system dynamically adjusts the allocation based on their priority value percentage and current resource utilization. For example, a node with a priority of 0.75 can receive a 15% bandwidth increase when CPU utilization is below 50%, but will not receive any additional bandwidth when CPU utilization exceeds 80% to avoid overload.

[0114] To enhance the robustness of the method, this embodiment also introduces an anomaly handling mechanism. During the data acquisition phase, if the operating parameters of a node cannot be obtained, the system will use the historical average of one or more recent statistical windows (e.g., the last five 1-minute windows) as a temporary substitute. If three consecutive acquisition failures occur, the node will be marked as "data missing" and excluded from high-priority task allocation. During model prediction, if the output result is abnormal (e.g., consecutive prediction values ​​below 0.2 or significant fluctuations exceeding ±50%), the system will revert to the traditional calculation method based on weighted scoring to ensure result availability. In the event of extreme network latency or node failure, the system will trigger a fault-tolerance mechanism, immediately removing the node's allocation quota and reassigning its task to the node with the lowest latency in the same or adjacent clusters to ensure uninterrupted overall task execution. Through these anomaly handling and rollback strategies, the final generated device collaboration adjustment result ensures both the priority response capability of critical task nodes and avoids system failures caused by individual node anomalies, achieving a balance between real-time performance and stability in the power grid.

[0115] In summary, this invention provides a method and system for intelligent processing of power grid data based on big data, so as to achieve rapid response and efficient coordination of the power grid in the event of sudden failures or load surges.

[0116] Reference Figure 2 The second embodiment of the present invention provides a power grid data intelligent processing system based on big data, comprising:

[0117] The data acquisition module is used to collect load data, hardware performance parameters, operating status parameters, network latency values, data processing requirements of distributed energy access points, dynamic change records of power grid operation logs, communication protocol data between devices, bandwidth utilization data of communication networks, and device collaboration feedback data from the previous collaboration process of edge nodes.

[0118] The priority sequence module is used to perform grouping and fusion analysis based on the load data, the hardware performance parameters and the network latency value to obtain a priority sequence for inter-device cooperation.

[0119] The collaboration level module is used to determine the degree of load surge based on the priority sequence, the running status parameters, and the network latency value, and to form a collaboration level parameter.

[0120] The resource allocation module is used to regroup the data processing requirements and the cooperation level parameters, and generate a resource allocation scheme based on the priority sequence and the bandwidth utilization data.

[0121] The response speed module is used to determine the real-time response speed based on the resource allocation scheme and the dynamic change record, and obtain the response speed index.

[0122] The operation assurance module is used to perform stable operation prediction on the response speed index and the communication protocol data to obtain a stable operation assurance mechanism;

[0123] The equipment adjustment module is used to iteratively update the equipment collaboration adjustment result based on the stable operation guarantee mechanism and the equipment collaboration feedback data, combined with the priority sequence.

[0124] It should be noted that the big data-based intelligent power grid data processing system provided in this embodiment of the invention is used to execute all the process steps of the big data-based intelligent power grid data processing method in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0125] This invention also provides an electronic device. The electronic device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a resource allocation program. When the processor executes the computer program, it implements the steps described in the above embodiments of the intelligent power grid data processing method based on big data, for example... Figure 1 The step S11 shown. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the above-described device embodiments, such as the resource allocation module.

[0126] For example, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the electronic device.

[0127] The electronic device may be a desktop computer, laptop, handheld computer, or smart tablet, etc. The electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. It may include more or fewer components than described above, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0128] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the electronic device, connecting all parts of the electronic device via various interfaces and lines.

[0129] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0130] Wherein, if the modules / units integrated in the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0131] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0132] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method for intelligent processing of power grid data based on big data, characterized in that, include: Collect load data, hardware performance parameters, operating status parameters, network latency values, data processing requirements of distributed energy access points, dynamic change records of power grid operation logs, communication protocol data between devices, bandwidth utilization data of communication networks, and device collaboration feedback data from the previous collaboration process of edge nodes. Based on the load data, the hardware performance parameters, and the network latency value, grouping and fusion analysis is performed to obtain a priority sequence for inter-device collaboration; Based on the priority sequence, the operating status parameters, and the network latency value, the degree of load surge is determined, and a cooperation level parameter is formed. The data processing requirements and the collaboration level parameters are regrouped, and a resource allocation scheme is generated based on the priority sequence and the bandwidth utilization data. Based on the resource allocation scheme and the dynamic change record, the real-time response speed is judged to obtain the response speed index; By performing stable operation prediction on the response speed index and the communication protocol data, a stable operation guarantee mechanism is obtained; Based on the stable operation guarantee mechanism and the equipment collaboration feedback data, and combined with the priority sequence, the final equipment collaboration adjustment result is obtained through iterative updates.

2. The intelligent power grid data processing method based on big data according to claim 1, characterized in that, Based on the load data, the hardware performance parameters, and the network latency value, the step of performing grouping and fusion analysis to obtain a priority sequence for inter-device collaboration includes: The load data and the hardware performance parameters are grouped to obtain computing power indicators; The computing power index and the network latency value are input into a preset fusion analysis model to generate fusion results; Based on the fusion result, a sorting operation is performed to determine the collaboration priority of each edge node, thereby obtaining a priority sequence for inter-device collaboration.

3. The intelligent power grid data processing method based on big data according to claim 1, characterized in that, The step of determining the degree of load surge based on the priority sequence, the operating status parameters, and the network latency value to form a cooperation level parameter includes: When the data flow rate in the running status parameters exceeds the preset rate threshold, the data flow rate and the processor utilization rate in the running status parameters are weighted and calculated in combination with the priority sequence to obtain the load surge value. The load surge value and the network latency value are fused and analyzed to form a cooperation level parameter.

4. The intelligent power grid data processing method based on big data according to claim 1, characterized in that, The step of regrouping the data processing requirements and the collaboration level parameters, and generating a resource allocation scheme based on the priority sequence and the bandwidth utilization data, includes: Clustering operations are performed on the data processing requirements and the collaboration level parameters to obtain the number of groups; When the number of groups exceeds a preset group number threshold, a weighted calculation is performed on the groups by combining the priority sequence and the bandwidth utilization data to obtain the bandwidth allocation parameters. Based on the bandwidth allocation parameters, differentiated bandwidth allocation is performed to determine the resource allocation scheme.

5. The intelligent power grid data processing method based on big data according to claim 1, characterized in that, The step of determining the real-time response speed based on the resource allocation scheme and the dynamic change record to obtain a response speed index includes: The bandwidth allocated to each node is determined according to the resource allocation scheme. Based on the task issuance timestamp and task completion timestamp recorded in the dynamic change record, and combined with the bandwidth value, the effective response latency of each node is calculated; The effective response delay is statistically analyzed over time to obtain the average response delay of each node; The average response delay is compared with a preset threshold to generate a response speed index.

6. The intelligent power grid data processing method based on big data according to claim 1, characterized in that, The stable operation guarantee mechanism is obtained by performing stable operation prediction on the response speed index and the communication protocol data, including: The response speed metric and the communication protocol data are used to construct a prediction input vector; The predicted input vector is input into a preset multilayer perceptron model for feature extraction and nonlinear mapping, and the stability prediction result is output. Based on the stability prediction results, a stable operation guarantee mechanism is generated to determine the stability status of inter-device communication and task processing.

7. The intelligent power grid data processing method based on big data according to claim 1, characterized in that, The process of iteratively updating the equipment collaboration based on the stable operation guarantee mechanism and the equipment collaboration feedback data, combined with the priority sequence, to obtain the final equipment collaboration adjustment result includes: Based on the aforementioned stable operation guarantee mechanism, the guarantee mechanism parameters are adjusted to generate an updated guarantee mechanism; The update guarantee mechanism is optimized by combining the device collaboration feedback data to form a collaboration adjustment plan; The collaborative adjustment schemes are sorted according to the priority sequence, and resource allocation is optimized through load balancing calculations to output the final device collaborative adjustment results.

8. A power grid data intelligent processing system based on big data, characterized in that, include: The data acquisition module is used to collect load data, hardware performance parameters, operating status parameters, network latency values, data processing requirements of distributed energy access points, dynamic change records of power grid operation logs, communication protocol data between devices, bandwidth utilization data of communication networks, and device collaboration feedback data from the previous collaboration process of edge nodes. The priority sequence module is used to perform grouping and fusion analysis based on the load data, the hardware performance parameters and the network latency value to obtain a priority sequence for inter-device cooperation. The collaboration level module is used to determine the degree of load surge based on the priority sequence, the running status parameters, and the network latency value, and to form a collaboration level parameter. The resource allocation module is used to regroup the data processing requirements and the cooperation level parameters, and generate a resource allocation scheme based on the priority sequence and the bandwidth utilization data. The response speed module is used to determine the real-time response speed based on the resource allocation scheme and the dynamic change record, and obtain the response speed index. The operation assurance module is used to perform stable operation prediction on the response speed index and the communication protocol data to obtain a stable operation assurance mechanism; The equipment adjustment module is used to iteratively update the equipment collaboration adjustment result based on the stable operation guarantee mechanism and the equipment collaboration feedback data, combined with the priority sequence.

Images