Distributed new energy data scheduling and processing method based on cloud-edge-end collaboration

Abstract

This application discloses a distributed new energy data scheduling and processing method based on cloud-edge-end collaboration, which relates to the field of distributed new energy data analysis technology. Specifically, the application discloses the following steps: the cloud computing center outputs a task scheduling strategy adapted to the task processing capabilities of each edge node to the terminal device based on edge node information, terminal device information, and the edge node task scheduling model; the terminal device uploads the distributed new energy operation data to the corresponding edge node according to the task scheduling strategy and encryption strategy; the edge node performs abnormal data identification and correction on the distributed new energy data based on the abnormal data detection model and the abnormal data repair model. The beneficial effects of this application are: reducing transmission time, improving data processing efficiency, and enhancing the timeliness and accuracy of data output.

Description

Technical Field

[0001] The present application relates to the field of distributed new energy data analysis technology, and in particular to a distributed new energy data scheduling and processing method based on cloud-edge-end collaboration. Background Art

[0002] Among the many forms of renewable energy generation, distributed renewable energy, as a comprehensive energy utilization system close to the user, can directly meet users' diverse energy needs. It features cascaded energy efficiency, minimal losses, good system economics, and flexible operation, making it considered an effective solution to energy bottlenecks. Therefore, distributed renewable energy is becoming a key component of smart distribution networks, providing strong support for the sustainable use of energy and the intelligent development of power grids.

[0003] Accurately collecting massive amounts of operational data from distributed renewable energy is a prerequisite for analyzing its regulation characteristics and optimizing its control. As the construction of the Power Internet of Things (PoT) progresses, the demands for comprehensive state awareness, efficient information processing, and convenient and flexible applications will intensify. This will inevitably lead to an exponential growth in the number of connected devices and data volumes, necessitating changes in data acquisition methods and improvements in data processing timelines.

[0004] While cloud computing can leverage its immense computing power to process and analyze data collected from the device layer and uploaded to the cloud platform, traditional data collection methods typically transmit all data to power plant data centers or the cloud for analysis. Consequently, operational data collection and analysis face challenges such as long communication distances, high data transmission latency, and data loss. The surge in the number of devices has also led to a surge in the amount of data generated by these devices, and the growth rate of network bandwidth has far outstripped the data growth rate, making network bandwidth a bottleneck. Furthermore, the increasingly complex network environment exacerbates network latency, leading to delayed processing of distributed renewable energy data and an inability to effectively remove or repair abnormal data before analysis based on this data. This leads to deviations in subsequent calculations, impacting the timeliness and accuracy of overall grid dispatch.

[0005] The patent, "Computing Resource Scheduling Method and System Based on Cloud Computing," published with publication number CN119396586A and date of publication on February 7, 2025, specifically discloses preprocessing the real-time power data collected by each node in the power grid system to improve data transmission capabilities. Furthermore, resource demand forecasting is performed for each cloud computing node, and the resource demand forecast results are used to optimize the cloud computing resource allocation of each cloud computing node with the optimization goals of minimizing computing resource waste and minimizing system response time. While this solution achieves dynamic allocation of computing resources, it does not address the issues of load imbalance caused by inconsistent performance of edge server nodes during computing task offloading decisions and task processing. Summary of the Invention

[0006] This application addresses the problems in the existing technology that the transmission and processing of distributed new energy data are poor in timeliness and the data is prone to abnormal data that affects the accuracy and timeliness of subsequent power grid scheduling based on distributed new energy data. It provides a distributed new energy data scheduling and processing method based on cloud-edge collaboration. Through the cooperation of cloud computing center, edge nodes and terminal devices, the distributed new energy data collected by the terminal devices are allocated to the corresponding edge nodes for abnormal data identification and repair, reducing the computing pressure of the cloud computing center. At the same time, data processing is performed at the edge nodes closer to the terminal devices, reducing transmission time, improving data processing efficiency, and improving the timeliness and accuracy of data output.

[0007] To achieve the above-mentioned technical objectives, the present application provides a technical solution, which is a distributed new energy data scheduling and processing method based on cloud-edge-end collaboration, including the following steps: the cloud computing center outputs a task scheduling strategy adapted to the task processing capabilities of each edge node to the terminal device based on edge node information, terminal device information and edge node task scheduling model; the terminal device uploads the distributed new energy operation data to the corresponding edge node according to the task scheduling strategy and encryption strategy; the edge node performs abnormal data identification and correction on the distributed new energy data based on the abnormal data detection model and the abnormal data repair model.

[0008] Furthermore, the cloud computing center outputs a task scheduling strategy adapted to the task processing capabilities of each edge node to the terminal device based on the edge node information, terminal device information and the edge node task scheduling model, including: constructing a multi-objective optimization function based on the shortest processing time, minimum energy consumption and lowest load balancing value of task processing; constructing an edge node task scheduling model with the multi-objective optimization objective function and constraint conditions; the cloud computing center obtains the edge node information and terminal device information, performs optimization solution according to the edge node task scheduling model, and outputs the task scheduling strategy to the terminal device.

[0009] Furthermore, constructing the edge node task scheduling model based on the multi-objective optimization objective function and constraint conditions also includes: constructing constraint conditions based on scheduling decision, node uniqueness, node capacity and task completion time.

[0010] Furthermore, the edge node performs abnormal data identification and correction on the distributed new energy data based on the abnormal data detection model and the abnormal data repair model, including: constructing an abnormal data detection model based on prior association and sequence association; constructing an abnormal data repair model based on the relative or absolute position information of the time series data; the edge node performs abnormal data identification on the distributed new energy data based on the abnormal data detection model, uses the identification result as the input of the abnormal data repair model, and outputs the correction value of the abnormal data.

[0011] Furthermore, the construction of the abnormal data repair model based on the relative or absolute position information of the time series data includes: constructing a repair embedding layer based on a high-dimensional embedding map; constructing a position encoding layer based on the relative or absolute position information corresponding to the position encoding; constructing a Transformer decoder layer based on the masked attention mechanism; constructing a feedforward layer based on the repair loss function based on the reconstruction error; and constructing the abnormal data repair model with the repair embedding layer, the position encoding layer, the Transformer decoder layer and the feedforward layer.

[0012] Furthermore, it also includes: constructing an abnormal data detection model based on the Anomaly attention mechanism, training the abnormal data detection model according to historical distributed new energy operation data and encryption parameters; deploying the linear layer of the abnormal data detection model on the terminal device as the embedding layer of the terminal device; when the terminal device outputs the encrypted distributed new energy operation data to the corresponding edge node, the corresponding edge node performs abnormal data identification and encryption parameter update according to the abnormal data detection model, and feeds back the updated encryption parameters to the terminal device, and the terminal device performs the next encryption according to the updated encryption parameters.

[0013] Furthermore, the constructing of the Transformer decoder layer based on the masked attention mechanism also includes: constructing a mask probability based on an abnormal score attention threshold, and constructing a Transformer decoder layer with the mask probability and the masked attention mechanism.

[0014] Furthermore, the multi-objective optimization function constructed based on the shortest processing time, minimum energy consumption and lowest load balancing value of task processing includes: obtaining the task transmission rate based on the transmission power, channel gain, channel interference and channel bandwidth between the edge node and the terminal device; obtaining the task transmission time based on the task transmission rate and the task data size; obtaining the task calculation time based on the computing power of the edge node; and obtaining the processing time based on the task transmission time and the task calculation time.

[0015] Furthermore, constructing a multi-objective optimization function based on the shortest processing time, minimum energy consumption and lowest load balancing value of task processing also includes: obtaining a load balancing value based on the difference between the load rate of a single edge node and the average load rate of all edge nodes.

[0016] Furthermore, the construction of an abnormal data detection model based on prior association and serial association includes: constructing an abnormal data detection model infrastructure based on the Anomaly Transformer algorithm; obtaining historical distributed new energy operation data and training the abnormal data detection model infrastructure based on the detection loss function to obtain an abnormal data detection model.

[0017] The beneficial effects of this application are as follows: 1. The cloud computing center rationally distributes real-time collected data to edge nodes for processing based on the task processing capabilities of edge nodes, reducing output transmission delays, improving the analysis efficiency of distributed renewable energy operation data, and reducing the computing and bandwidth pressure of the cloud computing center. At the same time, the abnormal data detection model and abnormal data repair model are used to perform abnormal data identification and correction on distributed renewable energy, improving the accuracy of distributed renewable energy operation data entering subsequent scheduling calculations.

[0018] 2. By constraining the standard deviation of the load rate, we ensure that the ratio of each edge node's load task to its own performance tends to be consistent. This avoids the situation where some edge nodes with better performance are unable to effectively carry new tasks due to task overload or other reasons, and the tasks are assigned to other edge nodes with poorer performance. This leads to a significant increase in latency costs, excessive pressure on some nodes, and thus affects the stability of the entire edge server system. This application effectively improves the overall stability and reliability of edge nodes.

[0019] 3. Encryption parameters are incorporated into the training of the anomaly detection model, with the embedding layer serving as the model's input layer. The encryption parameters are also treated as trainable variables. During training, the encryption parameters are dynamically updated using the backpropagation algorithm based on the anomaly detection model's loss function. This not only allows the terminal's limited computing resources to be used to encrypt data for privacy protection, but also allows encryption parameters to be updated after each training iteration of the anomaly detection model based on data characteristics and model performance, ensuring that each encryption process uses different encryption parameters, thus achieving better privacy protection.

[0020] 4. Set the concentration of the probability distribution according to the anomaly score attention threshold, so as to encourage the model to focus on positions with higher anomaly scores, thereby preventing outliers from interfering with normal pattern learning and improving data reconstruction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1This is a flow chart of the distributed new energy data scheduling and processing method based on cloud-edge-end collaboration in this application.

[0022] Figure 2 This is a schematic diagram of the cloud-edge-end three-layer collaborative framework of the distributed new energy data scheduling and processing method based on cloud-edge-end collaboration in this application.

[0023] Figure 3 This is a schematic diagram of the anomaly detection model structure of the distributed new energy data scheduling and processing method based on cloud-edge-end collaboration in this application.

[0024] Figure 4 This is a schematic diagram of the abnormal repair model structure of the distributed new energy data scheduling and processing method based on cloud-edge-end collaboration in this application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of this application more clear, the application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific implementation method described here is only an optimal embodiment of this application, which is only used to explain this application and does not limit the scope of protection of this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0026] like Figure 1 、 Figure 2 As shown in FIG, the distributed new energy data scheduling and processing method based on cloud-edge-end collaboration includes the following steps:

[0027] The cloud computing center outputs a task scheduling strategy adapted to the task processing capabilities of each edge node to the terminal device based on the edge node information, terminal device information, and the edge node task scheduling model;

[0028] The terminal device uploads the distributed new energy operation data to the corresponding edge node according to the task scheduling strategy and encryption strategy;

[0029] The edge node performs abnormal data identification and correction on distributed new energy data based on the abnormal data detection model and the abnormal data repair model.

[0030] In this embodiment, the cloud computing center rationally distributes real-time collected data to edge nodes for processing based on their task processing capabilities. This reduces output transmission delays, improves the efficiency of analyzing distributed renewable energy operating data, and reduces computing and bandwidth pressure on the cloud computing center. Furthermore, anomaly detection and repair models are used to identify and correct abnormal data in distributed renewable energy systems, improving the accuracy of distributed renewable energy operating data before it enters subsequent scheduling calculations.

[0031] Trusted central servers serve as cloud computing centers, responsible for integrated computing for large-scale offline tasks. Servers close to end devices serve as edge nodes, responsible for executing local real-time computing tasks. Metering devices, such as smart meters and inverters, serve as end devices, responsible for collecting and reporting distributed renewable energy data.

[0032] The cloud computing center outputs a task scheduling strategy adapted to the task processing capabilities of each edge node to the terminal device based on edge node information, terminal device information, and the edge node task scheduling model, including:

[0033] Construct a multi-objective optimization function based on the shortest processing time, minimum energy consumption and lowest load balancing value of task processing;

[0034] Construct edge node task scheduling model based on multi-objective optimization objective function and constraint conditions;

[0035] The cloud computing center obtains edge node information and terminal device information, performs optimization solution based on the edge node task scheduling model, and outputs the task scheduling strategy to the terminal device.

[0036] By constructing a multi-objective optimization function based on the shortest processing time, minimum energy consumption and lowest load balancing value of task processing, and solving the multi-objective optimization function, a task scheduling strategy that takes into account task execution efficiency, resource utilization and system stability is obtained. This can not only improve the effective utilization of edge node clusters, but also enable the closest edge nodes to take on the processing of data uploaded by the corresponding terminal devices, reduce transmission time, share computing pressure, and improve computing efficiency.

[0037] Specifically, the multi-objective optimization function based on the shortest processing time, minimum energy consumption and lowest load balancing value of task processing includes:

[0038] Obtain the task transmission rate based on the transmission power, channel gain, channel interference, and channel bandwidth between the edge node and the terminal device;

[0039] Get the task transmission time based on the task transmission rate and task data size;

[0040] Get the task computing time based on the edge node computing power;

[0041] The processing time is obtained by taking the task transmission time and the task calculation time.

[0042] According to the transmission power, channel gain, channel interference and channel bandwidth between the edge node and the terminal device, the task transmission rate is obtained as follows:

[0043] ;

[0044] ;

[0045] in, Indicates the tasks of the terminal device Transmit to edge node The transmission rate, represents the channel bandwidth, Indicates the tasks of the terminal device Transmit to edge node The transmission power, Indicates the tasks of the terminal device Transmit to edge node The channel gain, Indicates the tasks of the terminal device Use shared channels with edge nodes The channel interference between them, N represents the total number of edge nodes, Indicates terminal device tasks Transmit to edge node The transmission power, Represents an edge node Whether to perform terminal device tasks The task of the channel is unloaded, if Then the task is unloaded, otherwise .

[0046] It's understood that transmission power determines the power of the transmitted signal, channel gain reflects changes in signal strength during transmission, channel interference refers to the impact of interference signals other than the desired signal on the receiving end, and channel bandwidth refers to the frequency range of signals that a channel can transmit. By comprehensively evaluating task transmission rates based on transmission power, channel gain, channel interference, and channel bandwidth, we rationally allocate and schedule tasks based on the actual channel conditions between edge nodes and terminal devices. This helps avoid scheduling deviations caused by interference from complex communication environments, avoid unnecessary high-power transmissions, reduce energy loss, and improve energy efficiency.

[0047] The task transmission time is obtained based on the task transmission rate and task data size:

[0048] ;

[0049] in, Indicates the tasks of the terminal device Transmit to edge node The task transfer time, Indicates the tasks of the terminal device Contains the data size.

[0050] The task calculation time obtained based on the edge node computing power is:

[0051] ;

[0052] in, Indicates the tasks of the terminal device At the edge node The task calculation time, Indicates completion of the task The total number of CPUs required, Represents an edge node The number of CPU cycles per second.

[0053] In this embodiment, the computing capacity of the edge node is represented by the number of cycles per second. The processing time is obtained by taking the task transmission time and the task calculation time as:

[0054] ;

[0055] in, Indicates the tasks of the terminal device Via edge nodes The processing time of the execution.

[0056] The multi-objective optimization function based on the shortest processing time, minimum energy consumption and lowest load balancing value of task processing also includes:

[0057] The load balancing value is obtained based on the difference between the load rate of a single edge node and the average load rate of all edge nodes.

[0058] Specifically, the load balancing value is used to quantify the computing load balancing of each edge node. The smaller the load balancing value, the more balanced the overall load of the edge node. The load balancing value is obtained based on the difference between the load rate of a single edge node and the average load rate of all edge nodes:

[0059] ;

[0060] ;

[0061] Among them, B represents the load balancing value, Represents an edge node The load factor, Indicates the average load rate of all edge nodes.

[0062] The multi-objective optimization function based on the shortest processing time, minimum energy consumption and lowest load balancing value of task processing is constructed as follows:

[0063] ;

[0064] ;

[0065] in, represents the processing time weight, represents the energy consumption weight, Indicates the load balancing weight. Indicates the tasks of the terminal device Via edge nodes The processing energy consumed by the execution.

[0066] In this embodiment, the processing time weight, energy consumption weight, and load balancing weight are equal. In other embodiments, the weights may be adaptively allocated based on the actual edge node situation, processing efficiency requirements, and energy consumption considerations.

[0067] In this embodiment, by constraining the standard deviation of the load ratio, the ratio of each edge node's load task to its own performance is ensured to be consistent. This prevents some edge nodes with better performance from being unable to effectively handle new tasks due to task overload or other reasons, forcing the tasks to be assigned to other edge nodes with poorer performance. This can lead to a significant increase in latency costs, excessive pressure on some nodes, and thus affect the stability of the entire edge server system. This embodiment adopts load balancing considerations to effectively improve the overall stability and reliability of edge nodes.

[0068] The edge node task scheduling model constructed with multi-objective optimization objective functions and constraints also includes:

[0069] Build constraints based on scheduling decisions, node uniqueness, node capacity, and task completion time.

[0070] The first constraint condition constructed based on the scheduling decision is:

[0071] ;

[0072] Indicates whether to assign the terminal device's task Dispatching to edge nodes Calculate, when When the terminal device is tasked Dispatching to edge nodes Calculate, when When the terminal device is not Dispatching to edge nodes Perform calculations.

[0073] The second constraint condition based on node uniqueness is:

[0074] ;

[0075] The second constraint condition represents the task of the terminal device Must be assigned to and can only be assigned to one edge node.

[0076] The third constraint condition is constructed based on the node capacity:

[0077] ;

[0078] The third constraint represents the edge node capacity limitation.

[0079] The fourth constraint condition is constructed based on the task completion time:

[0080] ;

[0081] Indicates the tasks of the terminal device The processing deadline.

[0082] An edge node task scheduling model is constructed based on a multi-objective optimization function and constraints, and solved using a genetic algorithm or particle swarm optimization algorithm. In this embodiment, edge node information includes at least the number of CPU cycles per second, edge node location, edge node capacity, edge node computing energy consumption, edge node transmission power, channel gain, and channel bandwidth. Terminal device information includes at least the task processing deadline, task data size, and the total number of CPUs required to complete the task.

[0083] The terminal device uploads distributed new energy operation data to the corresponding edge node according to the task scheduling strategy and encryption strategy, including:

[0084] The terminal device obtains the embedded representation based on the encrypted parameters according to the collected distributed new energy operation data, and uploads the embedded representation to the corresponding edge node according to the task scheduling strategy.

[0085] The distributed renewable energy operation data collected by the terminal device is a typical time series data. The distributed renewable energy operation data is encrypted and reconstructed through the embedding layer. In this embodiment, the embedding layer consists of a linear layer, and the encryption parameters consist of a weight matrix and a bias vector:

[0086] ;

[0087] in, represents the embedded representation learned after the embedding layer, X represents the distributed new energy operation data, W represents the weight matrix of the embedding layer, and b represents the bias vector.

[0088] Distributed new energy operation data is first embedded and learned in the terminal device, converted into embedded representation and transmitted to the corresponding edge node for data anomaly analysis and processing, thereby avoiding data leakage during transmission and improving data security.

[0089] The edge node performs abnormal data identification and correction on distributed new energy data based on the abnormal data detection model and the abnormal data repair model, including:

[0090] Build an abnormal data detection model based on prior association and sequence association;

[0091] Build an abnormal data repair model based on the relative or absolute position information of time series data;

[0092] The edge node performs abnormal data identification on the distributed new energy data based on the abnormal data detection model, uses the identification results as the input of the abnormal data repair model, and outputs the correction value of the abnormal data.

[0093] In some cases, after receiving the embedded representation output by the terminal device, the edge node first decrypts the embedded representation and then performs data preprocessing and abnormal data identification. Data preprocessing includes at least: data standardization and normalization. As a feasible embodiment, the distributed new energy operation data is preprocessed using the Z-score standardization and Min-Max normalization method:

[0094] Use the Z-score standardization method to standardize the distributed new energy operation data:

[0095] ;

[0096] in, It represents the standardized distributed renewable energy operation data, which obeys the distribution with a mean of 0 and a standard deviation of 1. Represents the mean of the original distributed new energy operation data, Represents the standard deviation of the original distributed new energy operation data.

[0097] Use the Min-Max normalization method to normalize the distributed new energy operation data:

[0098] ;

[0099] in: Represents the normalized distributed new energy operation data, Indicates the minimum value of the original distributed new energy operation data, Indicates the maximum value of the original distributed new energy operation data.

[0100] Building anomaly data detection models based on prior association and sequence association includes:

[0101] Build an abnormal data detection model infrastructure based on the Anomaly Transformer algorithm;

[0102] Obtain historical distributed new energy operation data and train the abnormal data detection model infrastructure based on the detection loss function to obtain the abnormal data detection model.

[0103] In this embodiment, the Anomaly Transformer algorithm is introduced to obtain the prior correlation and sequence correlation of distributed new energy data. Figure 3 As shown in Figure 2, the Anomaly Transformer algorithm replaces the self-attention module in the Transformer algorithm with the Anomaly attention module, while modeling the prior association and data association of the data:

[0104] ;

[0105] ;

[0106] in, Indicates the Hidden representation of the layer, Representation layer normalization, Indicates the The output of the layer, Indicates the The output of the layer.

[0107] The Anomaly Attention Module uses a learnable Gaussian kernel to model the prior association of each data point and learns the sequence association from the original sequence. The anomaly data detection model quantifies the distance between the prior association and the sequence association at each time point to represent the distinguishability between normal and abnormal data, and uses this as a criterion for time series anomaly detection.

[0108] Among them, The corresponding query of the layer definition ,key ,value , and additionally define the standard deviation matrix for prior correlation calculation :

[0109] ;

[0110] ;

[0111] ;

[0112] ;

[0113] in, Indicates the The query weight of the layer, Indicates the The key weights of the layer, Indicates the The value weight of the layer, Indicates the Standard deviation matrix weights for the layer.

[0114] No. The Anomaly attention mechanism of the layer is expressed as:

[0115] ;

[0116] ;

[0117] ;

[0118] in, Indicates the The prior correlation of the layer, Indicates the Sequential association of layers, represents the standard deviation matrix for time step m, represents the distance between time step m and time step g, represents the feature dimension, represents the transpose of the key matrix, Represents the reconstruction feature.

[0119] After calculating the prior correlation and serial correlation, the symmetric KL divergence is used to formalize the correlation difference between the prior correlation and the serial correlation, which represents the information gain between the two distributions. The correlation differences of the layers are averaged to combine the correlations of multiple layers of features into a more informative metric:

[0120] ;

[0121] in, , M represents the total number of time steps, and L represents the total number of model layers; Indicates the The feature vector of the prior association at the layer and the mth time step; express The feature vector of the sequence association at the layer and the mth time step; Indicates the The feature vector of the prior association at the layer and the Mth time step; express The feature vector of the sequence association at the layer and the Mth time step.

[0122] This gives the associated difference , for an outlier point, The mth row represents the Gaussian distance between it and other time points. The closer the Gaussian distance, the larger the corresponding value. Since this point is abnormal, the attention mechanism will only focus on the points near the abnormal point, which leads to and The more similar they are, the smaller the KL divergence value is, so the AssDis value is smaller than that of normal points, which makes it possible to distinguish normal points from abnormal points.

[0123] Then, the detection loss function is constructed as:

[0124] ;

[0125] in, Represents the reconstructed sequence of distributed new energy operation data, represents the balance term, and F represents the Frobenius norm.

[0126] In this embodiment, historical distributed new energy operation data is obtained to train the abnormal data detection model infrastructure based on the loss function. Obtaining the abnormal data detection model also includes:

[0127] A two-stage loss function is constructed based on minimization and maximization.

[0128] In the minimization phase, the prior correlation is optimized , let the prior association Approximate the serial correlation learned from the original time series This process will adapt the prior association to different time series patterns, while also preventing the scale parameters of the prior association from being too small. In the maximization stage, the sequence association is optimized. , to maximize the difference between associations. This process will make the sequence association pay more attention to non-adjacent points, making the reconstruction of abnormal points more difficult. Integrating the reconstruction loss results in a two-stage loss function:

[0129] Minimization phase: ;

[0130] Maximization phase: ;

[0131] in, represents the total loss function, represents the sequence association features of gradient separation, Represents the prior correlation features for gradient separation.

[0132] Construct an anomaly data detection model based on the Anomaly Transformer algorithm and the two-stage loss function:

[0133] ;

[0134] in, Represents the characteristic vector of the original distributed new energy operation data at the mth time step, Represents the characteristic vector of the reconstructed sequence of distributed new energy operation data under m time steps, Represents element multiplication, AnomalyScore represents the anomaly score of the input sequence. The smaller the association difference AssDis, the larger the anomaly score; the larger the reconstruction error, the larger the anomaly score, and thus the more likely the distributed new energy operation data is to be abnormal.

[0135] like Figure 4 As shown in the figure, constructing an abnormal data repair model based on the relative or absolute position information of time series data includes:

[0136] Construct a repair embedding layer based on the high-dimensional embedding map;

[0137] Constructing a position coding layer based on the relative or absolute position information corresponding to the position coding;

[0138] Constructing the Transformer decoder layer based on the masked attention mechanism;

[0139] Construct a repair loss function based on the reconstruction error to establish a feedforward layer;

[0140] The abnormal data repair model is constructed with the repair embedding layer, position encoding layer, Transformer decoder layer and feedforward layer.

[0141] The repair embedding layer is constructed based on the high-dimensional embedding map:

[0142] ;

[0143] Among them, E represents the repair embedding representation, represents the first repair embedding weight matrix, represents the second repair embedding weight matrix, represents the first bias vector, Represents the second bias vector.

[0144] First, the time series values ​​are mapped to a high-dimensional space through the repair embedding layer. Then, position encoding is added to the repair embedding representation based on the relative or absolute position information in the time series data to obtain a joint representation containing position information:

[0145] ;

[0146] ;

[0147] ;

[0148] in, represents the positional encoding of even dimensions, represents an even dimension, represents the dimension of the position encoding, represents the positional encoding of odd dimensions, represents odd dimensions, Represents the joint representation after adding position encoding.

[0149] The resulting representation vector matrix is ​​passed into the Transformer decoder layer, and the dependencies are captured using a masked attention mechanism. Unlike conventional Transformer models, in this embodiment, the mask matrix is ​​determined by the position of the outliers in the input sequence. When the anomaly score output by the anomaly detection model for the input sequence is greater than the preset anomaly score, the corresponding element of the mask matrix is ​​0; when the anomaly score output by the anomaly detection model for the input sequence is less than the preset anomaly score, the corresponding element of the mask matrix is ​​1.

[0150] Then, the repair value is generated through the output head of the feedforward layer to replace the abnormal data. The repair loss function is constructed based on the reconstruction error of the abnormal part:

[0151] ;

[0152] in, Indicates repair loss, represents the set of mask positions, Indicates the The true value of the distributed new energy operation data at the mask position, Indicates the The inpainted value at the mask position.

[0153] In this embodiment, a distributed new energy acquisition and processing architecture with cloud-edge collaboration is established, which sinks real-time tasks to edge nodes for processing, significantly reduces data transmission delays, improves the efficiency of real-time collection and analysis of distributed new energy operation data, and effectively alleviates cloud computing and bandwidth pressure. At the same time, the terminal-to-edge data transmission technology using soft encryption algorithms realizes the encryption and privacy protection of the operation data of new energy terminals such as inverters, ensures the end-to-end security of sensitive information during transmission, and reduces the risk of data leakage. In addition, a multi-objective optimization scheduling model that comprehensively considers task latency, energy consumption and load balancing is established to intelligently allocate computing resources, maximize the collaborative efficiency of edge nodes and cloud computing centers, and reduce the overall operating cost of the system. An abnormal data detection and abnormal data repair model based on deep learning algorithms is established, combined with the attention mechanism to achieve efficient detection and context-aware repair of outliers in massive operation data, improve data quality and availability, and provide reliable support for subsequent regulatory decisions.

[0154] As a second embodiment of the present application, the distributed new energy data scheduling and processing method based on cloud-edge-end collaboration also includes:

[0155] Build an abnormal data detection model based on the Anomaly attention mechanism, and train the abnormal data detection model based on historical distributed new energy operation data and encrypted parameters;

[0156] Deploy the linear layer of the abnormal data detection model on the terminal device as the embedding layer of the terminal device;

[0157] When the terminal device outputs the encrypted distributed new energy operation data to the corresponding edge node, the corresponding edge node performs abnormal data identification and encryption parameter update according to the abnormal data detection model, and feeds back the updated encryption parameters to the terminal device. The terminal device performs the next encryption according to the updated encryption parameters.

[0158] In this embodiment, encryption parameters are incorporated into the training of anomaly detection models, with the embedding layer serving as the model's input layer. The encryption parameters are also treated as trainable variables. During training, the encryption parameters are dynamically updated using a backpropagation algorithm based on the anomaly detection model's loss function. This not only allows data to be encrypted to protect privacy using the terminal's limited computing resources, but also allows encryption parameters to be updated based on data characteristics and model performance after each training iteration of the anomaly detection model, ensuring that each encryption process uses different encryption parameters, thereby achieving even better privacy protection.

[0159] In this embodiment, constructing a Transformer decoder layer based on the masked attention mechanism also includes:

[0160] The mask probability is constructed based on the anomaly score attention threshold, and the Transformer decoder layer is constructed with the mask probability and mask attention mechanism.

[0161] The mask probability is constructed based on the anomaly score attention threshold as:

[0162] ;

[0163] in, represents the mask probability; The scaling hyperparameter for the anomaly score controls the impact of the anomaly score on the probability, allowing the model to focus more on locations with higher anomaly scores based on actual needs. Represents the data point at the mth time step The anomaly score.

[0164] The concentration of the probability distribution is set according to the anomaly score attention threshold, thereby prompting the model to pay attention to positions with higher anomaly scores, mask the input sequence according to the mask probability, and construct the corresponding mask matrix. For abnormal data, the corresponding elements of the mask matrix are negative infinity (a very large negative number is taken in actual calculation, such as 10 to the power of -9), indicating that the corresponding information is shielded. For normal data, the corresponding elements of the mask matrix are 0, indicating that the corresponding information is retained. This prevents outliers from interfering with normal pattern learning and improves data reconstruction accuracy. It can be understood that in this embodiment, the anomaly score attention threshold is represented by a scaling hyperparameter of the anomaly score. The scaling hyperparameter can be selected according to the interval of anomaly score attention, so that the model pays more attention to the anomalies in the interval position, achieving improved adaptability under different reconstruction accuracies.

[0165] The specific implementation method described above is a preferred implementation method of the distributed new energy data scheduling and processing method based on cloud-edge-end collaboration of this application, and is not intended to limit the specific implementation scope of this application. The scope of this application includes but is not limited to this specific implementation method. Any equivalent changes made in accordance with the shape and structure of this application are within the protection scope of this application.

Claims

1. A distributed new energy data scheduling and processing method based on cloud-edge-end collaboration is characterized by: The steps include: The cloud computing center outputs a task scheduling strategy adapted to the task processing capabilities of each edge node to the terminal device based on the edge node information, terminal device information, and the edge node task scheduling model; The terminal device uploads the distributed new energy operation data to the corresponding edge node according to the task scheduling strategy and encryption strategy; The edge node performs abnormal data identification and correction on distributed new energy data based on the abnormal data detection model and abnormal data repair model; Among them, an abnormal data detection model is built based on the Anomaly attention mechanism, and the abnormal data detection model is trained based on historical distributed new energy operation data and encrypted parameters; Deploy the linear layer of the abnormal data detection model on the terminal device as the embedding layer of the terminal device; When the terminal device outputs the encrypted distributed new energy operation data to the corresponding edge node, the corresponding edge node performs abnormal data identification and encryption parameter update according to the abnormal data detection model, and feeds back the updated encryption parameters to the terminal device. The terminal device performs the next encryption according to the updated encryption parameters.

2. The distributed new energy data scheduling and processing method based on cloud-edge-end collaboration according to claim 1 is characterized by: The cloud computing center outputs a task scheduling strategy adapted to the task processing capabilities of each edge node to the terminal device based on edge node information, terminal device information, and the edge node task scheduling model, including: Construct a multi-objective optimization function based on the shortest processing time, minimum energy consumption and lowest load balancing value of task processing; Construct edge node task scheduling model based on multi-objective optimization objective function and constraint conditions; The cloud computing center obtains edge node information and terminal device information, performs optimization solution based on the edge node task scheduling model, and outputs the task scheduling strategy to the terminal device.

3. The distributed new energy data scheduling and processing method based on cloud-edge-end collaboration according to claim 2 is characterized by: The edge node task scheduling model constructed by using multi-objective optimization objective functions and constraint conditions also includes: Build constraints based on scheduling decisions, node uniqueness, node capacity, and task completion time.

4. The distributed new energy data scheduling and processing method based on cloud-edge-end collaboration according to claim 1 is characterized by: The edge node performs abnormal data identification and correction on the distributed new energy data based on the abnormal data detection model and the abnormal data repair model, including: Build an abnormal data detection model based on prior association and sequence association; Build an abnormal data repair model based on the relative or absolute position information of time series data; The edge node performs abnormal data identification on the distributed new energy data based on the abnormal data detection model, uses the identification results as the input of the abnormal data repair model, and outputs the correction value of the abnormal data.

5. The distributed new energy data scheduling and processing method based on cloud-edge-end collaboration according to claim 4 is characterized by: The construction of an abnormal data repair model based on the relative or absolute position information of the time series data includes: Construct a repair embedding layer based on the high-dimensional embedding map; Constructing a position coding layer based on the relative or absolute position information corresponding to the position coding; Constructing the Transformer decoder layer based on the masked attention mechanism; Construct a repair loss function based on the reconstruction error to establish a feedforward layer; The abnormal data repair model is constructed with the repair embedding layer, position encoding layer, Transformer decoder layer and feedforward layer.

6. The distributed new energy data scheduling and processing method based on cloud-edge-end collaboration according to claim 5 is characterized by: The Transformer decoder layer constructed based on the mask attention mechanism also includes: The mask probability is constructed based on the anomaly score attention threshold, and the Transformer decoder layer is constructed with the mask probability and mask attention mechanism.

7. The distributed new energy data scheduling and processing method based on cloud-edge-end collaboration according to claim 2 is characterized by: The multi-objective optimization function constructed based on the shortest processing time, minimum energy consumption and lowest load balancing value of task processing includes: Obtain the task transmission rate based on the transmission power, channel gain, channel interference, and channel bandwidth between the edge node and the terminal device; Get the task transmission time based on the task transmission rate and task data size; Get the task computing time based on the edge node computing power; The processing time is obtained by taking the task transmission time and the task calculation time.

8. The distributed new energy data scheduling and processing method based on cloud-edge-end collaboration according to claim 2 is characterized by: The multi-objective optimization function constructed based on the shortest processing time, minimum energy consumption and lowest load balancing value of task processing also includes: The load balancing value is obtained based on the difference between the load rate of a single edge node and the average load rate of all edge nodes.

9. The distributed new energy data scheduling and processing method based on cloud-edge-end collaboration according to claim 4 is characterized by: The abnormal data detection model constructed based on prior association and sequence association includes: Build an abnormal data detection model infrastructure based on the Anomaly Transformer algorithm; Obtain historical distributed new energy operation data and train the abnormal data detection model infrastructure based on the detection loss function to obtain the abnormal data detection model.

Images