Data processing method and device of virtual power plant, electronic equipment and storage medium

Abstract

This invention discloses a data processing method, apparatus, electronic device, and storage medium for a virtual power plant, addressing the technical problem that traditional data processing methods struggle to meet the real-time data analysis and response requirements of virtual power plants in multiple scenarios. The method includes: acquiring a multimodal raw signal dataset from the virtual power plant; performing feature extraction and pattern recognition-based data processing on the multimodal raw signal dataset to obtain a feature vector matrix, sparsity evaluation values, and inter-feature correlation values; when the sparsity evaluation values ​​and inter-feature correlation values ​​meet preset conditions, constructing a joint sparse representation model based on the multimodal raw signal dataset and the feature vector matrix; introducing a multimodal adaptive measurement matrix; and obtaining a compressed observation fusion model by performing compressed sensing modeling on the joint sparse representation model; and optimizing and solving the compressed observation fusion model to obtain a fully reconstructed multimodal signal from the multimodal raw signal dataset.

Description

Technical Field

[0001] This invention relates to the field of virtual power plant data analysis technology, and in particular to a data processing method, apparatus, electronic device, and storage medium for a virtual power plant. Background Technology

[0002] Currently, virtual power plant access resources are widely dispersed, large in scale, and diverse in type. The high-dimensional heterogeneous data generated by these access resources (including real-time measurements, status monitoring, and environmental sensing) poses a severe challenge to data transmission bandwidth, storage pressure, and processing efficiency.

[0003] In the operation of virtual power plants, traditional data processing methods fall short in handling the massive, heterogeneous, multimodal, and multidimensional sparse power data processing demands. First, traditional methods struggle to efficiently process large-scale heterogeneous data, resulting in processing speeds that cannot meet real-time response requirements. Second, in various application scenarios, current technologies, while ensuring data integrity, struggle to achieve rapid data transmission and processing. Communication resource utilization is low, leading to inefficient data transmission. Furthermore, traditional data processing methods often fail to efficiently integrate and analyze complex, multidimensional, sparse data, further exacerbating the conflict between processing speed and data quality.

[0004] In summary, traditional technologies have significant limitations in data processing speed, communication efficiency, and multi-dimensional data integration. They are unable to meet the real-time data analysis and response needs of virtual power plants in various scenarios. Summary of the Invention

[0005] This invention provides a data processing method, apparatus, electronic device, and storage medium for a virtual power plant, which solves or partially solves the technical problem that traditional data processing methods are unable to meet the real-time data analysis and response needs of virtual power plants in multiple scenarios.

[0006] This invention provides a data processing method for a virtual power plant, the method comprising:

[0007] A multimodal raw signal dataset of a virtual power plant is acquired, and data processing based on feature extraction and pattern recognition is performed on the multimodal raw signal dataset to obtain a feature vector matrix, sparsity evaluation value, and correlation value between features.

[0008] When the sparsity evaluation value and the correlation value between features meet the preset conditions, a joint sparse representation model is constructed based on the multimodal original signal dataset and the feature vector matrix.

[0009] By introducing a multimodal adaptive measurement matrix and performing compressed sensing modeling on the joint sparse representation model, a compressed observation fusion model is obtained.

[0010] The compressed observation fusion model is optimized and solved to obtain the multimodal complete reconstructed signal of the multimodal original signal dataset.

[0011] Optionally, the step of performing data processing based on feature extraction and pattern recognition on the multimodal raw signal dataset to obtain feature vector matrices, sparsity evaluation values, and inter-feature correlation values ​​includes:

[0012] Feature extraction is performed on the original multimodal signal dataset to obtain a feature vector matrix;

[0013] The sparsity of the eigenvectors within the eigenvector matrix under a specified transform domain is identified to obtain a sparsity evaluation value.

[0014] The correlation between eigenvectors within the eigenvector matrix is ​​quantitatively analyzed to obtain the correlation value between features.

[0015] Optionally, when the sparsity evaluation value and the correlation value between features meet preset conditions, a joint sparse representation model is constructed based on the multimodal original signal dataset and the feature vector matrix, including:

[0016] When the sparsity evaluation value is greater than the preset sparsity evaluation threshold, and the correlation value between features is greater than the preset correlation threshold between features, the multimodal original signal dataset is stacked into a multimodal high-dimensional vector matrix by data stacking.

[0017] Based on the multimodal high-dimensional vector matrix, and considering both the specified transform domain and the sparse coefficient vector, a joint sparse representation model is constructed.

[0018] The specified transform domain is obtained by optimizing a joint dictionary learning algorithm based on a pre-designed sparse basis of different modes.

[0019] Optionally, the introduction of a multimodal adaptive measurement matrix, and the obtaining of a compressed observation fusion model by compressed sensing modeling the joint sparse representation model, includes:

[0020] Based on the feature vector matrix, an adaptive sampling rate model is constructed through an adaptive linear mapping;

[0021] Extract the original signal data and the length of each mode from the multimodal original signal dataset;

[0022] For each mode, the original signal data length is adaptively compressed using the adaptive sampling rate model in conjunction with the sampling rate enhancement mechanism based on data mutation, to obtain the compressed signal data length. Based on the original signal data length and the compressed signal data length, an adaptive measurement matrix corresponding to the mode is constructed.

[0023] For each of the aforementioned modes, the original signal data is simultaneously subjected to distributed cooperative sampling based on the adaptive measurement matrix to obtain the compressed observation vector matrix corresponding to each of the aforementioned modes.

[0024] Based on the compressed observation vector matrix of each mode, a global low-dimensional observation vector matrix is ​​constructed by stacking observation values.

[0025] Based on the low-dimensional observation vector matrix, global compressed sensing modeling is performed on the joint sparse representation model to obtain a compressed observation fusion model.

[0026] Optionally, optimizing the compressed observation fusion model to obtain the complete multimodal reconstructed signal of the original multimodal signal dataset includes:

[0027] The data reconstruction problem of the compressed observation fusion model is transformed into the optimal sparsity problem of the sparse coefficient vector in the joint sparse representation model;

[0028] Combining a fast iterative optimization algorithm, the alternating direction multiplier method is used to iteratively solve the data reconstruction problem and the optimal sparsity problem to obtain the converged optimal sparse coefficient vector.

[0029] The joint sparse representation model is reconstructed based on the optimal sparse coefficient vector to obtain the multimodal complete reconstructed signal of the original multimodal signal dataset.

[0030] Optionally, the method further includes:

[0031] Collect multimodal sparse raw data from a virtual power plant;

[0032] The multimodal sparse raw data is first processed for missing values ​​and outliers, and then standardized and normalized to obtain the multimodal raw signal dataset of the virtual power plant.

[0033] Optionally, the method further includes:

[0034] The multimodal reconstructed signal is first denoised by discrete wavelet transform, and then enhanced to obtain a multimodal denoised reconstructed signal.

[0035] The present invention also provides a data processing apparatus for a virtual power plant, the apparatus comprising:

[0036] The feature extraction and pattern recognition unit is used to acquire the multimodal raw signal dataset of the virtual power plant, and to perform data processing based on feature extraction and pattern recognition on the multimodal raw signal dataset to obtain the feature vector matrix, sparsity evaluation value and correlation value between features;

[0037] The joint sparse representation model construction unit is used to construct a joint sparse representation model based on the multimodal original signal dataset and the feature vector matrix when the sparsity evaluation value and the correlation value between features meet the preset conditions.

[0038] The compressed sensing modeling unit is used to introduce a multimodal adaptive measurement matrix and obtain a compressed observation fusion model by performing compressed sensing modeling on the joint sparse representation model.

[0039] The model optimization and solution unit is used to optimize and solve the compressed observation fusion model to obtain the multimodal complete reconstructed signal of the multimodal original signal dataset.

[0040] The present invention also provides an electronic device, the device comprising a processor and a memory:

[0041] The memory is used to store program code and transmit the program code to the processor;

[0042] The processor is configured to execute the data processing method for the virtual power plant as described above, according to the instructions in the program code.

[0043] The present invention also provides a computer-readable storage medium for storing program code for performing the data processing method of the virtual power plant as described in any of the preceding claims.

[0044] As can be seen from the above technical solutions, the present invention has the following advantages:

[0045] This paper presents a data processing method for a virtual power plant. The first step involves acquiring a multimodal raw signal dataset from the virtual power plant and performing feature extraction and pattern recognition-based data processing to obtain a feature vector matrix, sparsity evaluation values, and inter-feature correlation values. This feature extraction and pattern recognition process identifies the sparsity and correlation of the data while preventing overfitting, providing a basis for optimizing compressed sensing algorithms. The second step involves constructing a joint sparse representation model based on the multimodal raw signal dataset and the feature vector matrix when the sparsity evaluation values ​​and inter-feature correlation values ​​meet preset conditions. This joint sparse representation allows for the establishment of a unified sparse dictionary for multimodal data, uncovering intermodal correlations, and achieving compact representation and efficient compression of heterogeneous data. Furthermore, by optimizing the algorithm design, compressed sensing technology improves the efficiency and performance of processing all resource data from the virtual power plant. The third step introduces a multimodal adaptive measurement matrix and uses compressed sensing modeling on the joint sparse representation model to obtain a compressed observation fusion model. Therefore, by introducing a multimodal adaptive measurement matrix, adaptive measurement design and dynamic optimization are adopted during data sampling. This allows for real-time adjustment of the sampling matrix parameters and sparse basis based on data characteristics, automatically increasing the sampling rate when data mutations occur, and balancing information content and transmission cost. The fourth step involves optimizing the compressed observation fusion model to obtain a complete multimodal reconstructed signal from the original multimodal signal dataset. Thus, by studying feature fusion and model construction of multimodal data, an integrated fusion-reconstruction model is constructed, embedding multimodal features, decisions, and model fusion into the reconstruction optimization problem, achieving cross-modal data compression and reconstruction. Finally, joint solving achieves high-precision reconstruction and low-latency processing, improving data transmission efficiency and communication resource utilization. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a schematic diagram of the structure of a data processing system for a virtual power plant.

[0048] Figure 2 A flowchart illustrating the steps of a data processing method for a virtual power plant;

[0049] Figure 3 A flowchart of a distributed compressed sensing data processing algorithm;

[0050] Figure 4A schematic diagram of the overall process of a data processing method for a virtual power plant;

[0051] Figure 5 This is a structural block diagram of a data processing device for a virtual power plant. Detailed Implementation

[0052] This invention provides a data processing method, apparatus, electronic device, and storage medium for a virtual power plant, which solves or partially solves the technical problem that traditional data processing methods are unable to meet the real-time data analysis and response needs of virtual power plants in multiple scenarios.

[0053] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0054] As an example, in the operation of virtual power plants, traditional data processing methods fall short in handling the massive, heterogeneous, multimodal, and multidimensional sparse power data processing demands. First, traditional methods struggle to efficiently process large-scale heterogeneous data, resulting in processing speeds that cannot meet real-time response requirements. Second, in various application scenarios, current technologies, while ensuring data integrity, struggle to achieve rapid data transmission and processing. Communication resource utilization is low, leading to inefficient data transmission. Furthermore, traditional data processing methods often fail to efficiently integrate and analyze complex, multidimensional, sparse data, further exacerbating the conflict between processing speed and data quality.

[0055] In summary, traditional technologies have significant limitations in data processing speed, communication efficiency, and multi-dimensional data integration. They are unable to meet the real-time data analysis and response needs of virtual power plants in various scenarios.

[0056] Further analysis reveals that compressed sensing (CS) technology possesses the characteristic of "sparse sampling-precise reconstruction," providing a new approach for the efficient processing of massive resource data in virtual power plants. This technology captures the sparse characteristics of signals in the transform domain through random observations at a rate far lower than the Nyquist sampling rate. This significantly reduces data acquisition, transmission, and storage overhead while ensuring no loss of critical information. It is particularly suitable for high-dimensional, heterogeneous virtual power plant data environments with stringent real-time requirements.

[0057] Therefore, one of the core inventive points of this invention is: focusing on high-speed processing technology for all resource data of virtual power plants based on compressed sensing, and conducting research from the dimensions of algorithm optimization, multimodal data modeling, and data acceleration, to provide a more effective virtual power plant data processing method based on compressed sensing, aiming to break through the accuracy bottleneck of virtual power plant data compression and reconstruction, and achieve efficient fusion processing of cross-modal data. First, the massive, diverse, and heterogeneous resource data of the virtual power plant is preprocessed to ensure data quality, consistency, and applicability, meeting the needs of subsequent analysis. Second, by optimizing algorithm design, compressed sensing technology is used to improve the efficiency and performance of virtual power plant all resource data processing. Finally, by studying feature fusion and model construction of multimodal data, cross-modal data compression and reconstruction are achieved, improving data transmission efficiency and communication resource utilization.

[0058] Reference Figure 1 The diagram shows a structural schematic of a data processing system for a virtual power plant provided in an embodiment of the present invention.

[0059] Combination Figure 1 The virtual power plant data processing system provided by the present invention mainly includes a preprocessing module 101, a machine learning module 102, a distributed compressed sensing data processing module 103, and a postprocessing module 104.

[0060] The preprocessing module 101 preprocesses the collected multimodal data sources in parallel computing and hardware acceleration mode, and transmits the preprocessed multimodal data to the machine learning module 102 and the distributed compressed sensing data processing module 103.

[0061] The machine learning module 102 is used to perform data processing based on feature extraction and pattern recognition on the preprocessed multimodal data, and transmits the feature extraction and pattern recognition results to the distributed compressed sensing data processing module 103.

[0062] The distributed compressed sensing data processing module 103, based on the preprocessed multimodal data and the results of feature extraction and pattern recognition, uses a distributed compressed sensing algorithm to achieve high-precision compression and reconstruction of cross-modal data through feature fusion and efficient modeling, thereby obtaining multimodal reconstructed data. The data processing based on feature extraction and pattern recognition in the machine learning module 102, and the distributed compressed sensing data processing flow in the distributed compressed sensing data processing module 103, form an efficient combination of reconstruction algorithms, enabling more real-time and effective processing of massive amounts of multimodal data from virtual power plants.

[0063] The post-processing module 104 further processes the multimodal reconstructed data using post-processing optimization techniques. For example, it uses Discrete Wavelet Transform (DWT) to denoise and enhance the multimodal reconstructed data, outputting high-quality and reliable data.

[0064] Through the collaborative work of the aforementioned modules, a low-power, high-dimensional data processing solution can be provided for large-scale, full-resource data of virtual power plants. The virtual power plant data processing system and method provided in this embodiment of the invention can be adapted to data processing and information integration and display modules in various scenarios, providing technical support for real-time, comprehensive, and multi-dimensional perception of all resources in virtual power plants.

[0065] Reference Figure 2 The diagram illustrates a flowchart of a data processing method for a virtual power plant according to an embodiment of the present invention, which may specifically include the following steps:

[0066] Step 201: Obtain the multimodal raw signal dataset of the virtual power plant, and perform data processing based on feature extraction and pattern recognition on the multimodal raw signal dataset to obtain the feature vector matrix, sparsity evaluation value and correlation value between features;

[0067] In this embodiment of the invention, the true starting point of the multimodal data processing flow of the virtual power plant is the data preprocessing of the multimodal sparse raw data. Specifically, for the multimodal sparse raw data collected by the system, missing values ​​are first processed by direct deletion or interpolation, then outliers are identified and processed by statistical methods, and finally the data is standardized and normalized to complete the data preprocessing and obtain the multimodal raw signal dataset. .

[0068] In terms of outlier handling, for each data dimension The Z-score is calculated using the following formula to identify outliers that deviate significantly:

[0069] ;

[0070] in, Represents a multimodal raw signal dataset The Middle The data in the first The value of the dimension; and The first Mean and standard deviation of the dimensional data.

[0071] like (usually taken) ), then see Outliers are identified and removed or corrected (replaced with the mean / median, interpolation, etc.).

[0072] Regarding standardization and normalization, the data is standardized and normalized separately. First, Z-score standardization is used to make the data conform to zero mean and unit variance:

[0073] ;

[0074] Then, the data is scaled to the [0,1] interval using Min-Max normalization:

[0075] ;

[0076] After the above processing, a well-organized original signal dataset is obtained. .

[0077] Therefore, in the specific implementation, the first step is to collect multimodal sparse raw data from the virtual power plant. Then, the multimodal sparse raw data undergoes missing value and outlier handling, followed by standardization and normalization to obtain the multimodal raw signal dataset of the virtual power plant, which can be used in subsequent processes. This preprocessing of the massive, diverse, and heterogeneous resource data from the virtual power plant ensures data quality, consistency, and applicability, meeting the needs of subsequent analysis.

[0078] Based on this, the multimodal raw signal dataset is processed using feature extraction and pattern recognition to obtain feature vector matrix, sparsity evaluation value and correlation value between features.

[0079] Specifically, machine learning algorithms can be used to extract features and recognize patterns from multimodal raw signal datasets, identifying the sparsity and correlation of data while preventing overfitting, thus providing a basis for optimizing compressed sensing algorithms.

[0080] In terms of feature extraction, this embodiment of the invention employs Principal Component Analysis (PCA) to extract features from the multimodal raw signal dataset. Perform dimensionality reduction and extract the pre-extraction data. The principal components are used as the eigenvector matrix. :

[0081] ;

[0082] in, express It is a collection There are 10 samples, each sample has 100 samples. A data matrix with real-valued features; Indicates the preceding The projection matrix formed by the eigenvectors corresponding to the largest eigenvalues, and satisfying Feature Dimension The selection satisfies the cumulative contribution rate:

[0083] ;

[0084] In the above formula, For the first One eigenvalue; It can be understood as a dataset In Before taking each sample The sample feature matrix formed by the number of features is denoted as the feature vector matrix.

[0085] In terms of sparsity identification and analysis, this invention introduces the Gini index or Ratio quantitatively evaluates the signal in a specific transform domain Sparsity under:

[0086] ;

[0087] in, This is the coefficient vector of the signal in the transform domain; This refers to the eigenvector matrix mentioned earlier. The data point signal, i.e., the feature vector. This value... The closer a number is to 1, the stronger the sparsity. (usually taken) When determining the signal in the current transform domain, It exhibits strong sparsity, enabling high compression rate sampling strategies.

[0088] In correlation analysis, the Pearson correlation coefficient between different modes or features can be calculated using the following formula to quantify their association:

[0089] ;

[0090] in, These represent two different modes or feature vectors; Its sample mean; highly correlated modes ( , usually take In joint sparse representation, sparse modes can be shared, providing a basis for designing cross-modal fusion strategies.

[0091] For adaptive measurement matrix optimization, a feature vector to measurement matrix is ​​established based on the feature extraction and pattern recognition results. Adaptive mapping of parameters. The adaptive sampling rate function is defined as:

[0092] ;

[0093] in, The Sigmoid activation function maps linear combinations to the (0,1) interval; This is the weight vector; The bias term is obtained through training with historical data; , These are the preset minimum and maximum sampling rates (usually set to...). , (corresponding to compression ratios of 20:1 to 5:1).

[0094] Then the first Compressed data length of each modality for:

[0095] ;

[0096] in, For the first The original data length for each modality; [*] indicates rounding up.

[0097] Furthermore, embodiments of the present invention can dynamically adjust the local sampling rate based on the data mutation detection results, ensuring improved sampling accuracy when data changes dynamically, in order to capture key transient features. A data mutation index is defined. :

[0098] ;

[0099] when (Pick When a data mutation is detected, the following sampling rate boosting mechanism is triggered:

[0100] ;

[0101] in, This is the sampling rate boost factor (which can usually be taken as...). ); This represents the maximum allowed number of samples.

[0102] Therefore, by adopting adaptive measurement design and dynamic optimization during data sampling, the sampling matrix parameters and sparse basis can be adjusted in real time according to the data characteristics. When data mutations occur, the sampling rate can be automatically increased, balancing the amount of information and the transmission cost.

[0103] Based on the preceding discussion, in a specific implementation, the process of performing feature extraction and pattern recognition-based data processing on a multimodal raw signal dataset to obtain a feature vector matrix, sparsity evaluation values, and inter-feature correlation values ​​can include: extracting features from the multimodal raw signal dataset to obtain a feature vector matrix; identifying the sparsity of the feature vectors within the feature vector matrix in a specified transform domain to obtain a sparsity evaluation value; and performing quantitative analysis on the correlation between feature vectors within the feature vector matrix to obtain inter-feature correlation values.

[0104] Step 202: When the sparsity evaluation value and the correlation value between features meet the preset conditions, a joint sparse representation model is constructed based on the multimodal original signal dataset and the feature vector matrix.

[0105] In this embodiment of the invention, distributed compressed sensing technology is used to perform joint sparse representation on massive sparse multimodal data in the system, and the complete multimodal data is reconstructed through optimization. (Refer to...) Figure 3 The diagram shows a flowchart of a distributed compressed sensing data processing algorithm provided by an embodiment of the present invention.

[0106] Combination Figure 3 Assume that the multimodal raw signal dataset has a total of Types of modes (e.g.) Figure 3 There are 3 modes. For each mode, design or learn a suitable sparse basis. By using a joint dictionary learning algorithm, an overcomplete transformation domain is optimized. This makes the sparse coefficient vector of all modal data in this transform domain... It satisfies the sparsity property.

[0107] The original multimodal signal dataset is stacked into a high-dimensional vector. A joint sparse representation model is constructed. Its joint sparse representation is:

[0108] ;

[0109] in, It represents a sparse coefficient vector, and its non-zero elements exhibit a cross-modal block structure.

[0110] Based on the preceding discussion, when the sparsity evaluation value and the inter-feature correlation value meet preset conditions, a joint sparse representation model is constructed based on the multimodal original signal dataset and the feature vector matrix. This process specifically includes: when the sparsity evaluation value is greater than a preset sparsity evaluation threshold, and the inter-feature correlation value is greater than a preset inter-feature correlation threshold, stacking the multimodal original signal dataset into a multimodal high-dimensional vector matrix through data stacking; based on the multimodal high-dimensional vector matrix, and considering both the specified transform domain and the sparse coefficient vector, constructing a joint sparse representation model; wherein the specified transform domain is obtained by optimizing a pre-designed sparse basis for different modes using a joint dictionary learning algorithm.

[0111] This allows for the establishment of a unified sparse dictionary across current, images, and text through multimodal joint sparse representation, enabling the mining of intermodal correlations and achieving compact representation and efficient compression of heterogeneous data. Furthermore, by optimizing algorithm design and utilizing compressed sensing technology, the efficiency and performance of processing all resource data from a virtual power plant are improved.

[0112] Step 203: Introduce a multimodal adaptive measurement matrix, and obtain a compressed observation fusion model by performing compressed sensing modeling on the joint sparse representation model;

[0113] For each mode Design a Adaptive measurement matrix .in, Indicates the first The length of the data obtained after compression sampling of each mode, i.e., the number of data points actually transmitted; Indicates the first The original data length of each modality satisfies This invention employs a partially randomized Hadamard matrix to satisfy the requirements of the corresponding sparse basis. The irrelevance.

[0114] Based on the optimization criteria obtained from feature extraction and pattern recognition of the virtual power plant data in the aforementioned steps, The parameters are dynamically adjusted to ensure the matrix's adaptability. For example, when a sudden data change is detected, the sampling rate needs to be automatically increased based on a sampling rate boosting mechanism. The local sampling rate is obtained. .

[0115] This invention uses distributed cooperative sampling for sampling. That is, in the first... Multiple edge nodes, simultaneously collecting data Data of various modes The nodes compute compressed observations locally to obtain the modal data. The corresponding compressed observation vector matrix:

[0116] ;

[0117] Each node will only use the low-dimensional observation vector matrix. The current measurement matrix identifier is uploaded to the cloud for centralized processing. Combined with the previously collected edge node information, this constructs an edge-cloud collaborative processing architecture. In this architecture, edge nodes perform lightweight compressed sampling, while the cloud centrally performs fusion and reconstruction, balancing real-time performance, accuracy, and system scalability.

[0118] Stack the observations from all nodes in the system into a global low-dimensional observation vector matrix. Based on this, the global compressed sensing process is modeled as follows:

[0119] ;

[0120] in, The global measurement matrix in block diagonal form is defined as follows:

[0121] ;

[0122] in, This represents the total number of edge nodes. The total compression dimension for a single node; This represents the total original dimension of a single node.

[0123] Based on the preceding discussion, after constructing the joint sparse representation model through step 202, a multimodal adaptive measurement matrix can be introduced. By performing compressed sensing modeling on the joint sparse representation model, a compressed observation fusion model can be obtained. In a specific implementation, this process can further include: constructing an adaptive sampling rate model based on the feature vector matrix using adaptive linear mapping; extracting the original signal data and length of each modality from the multimodal original signal dataset; for each modality, combining a sampling rate enhancement mechanism based on data mutation, adaptively compressing the length of the original signal data using the adaptive sampling rate model to obtain the compressed signal data length, and constructing the corresponding adaptive measurement matrix based on the original and compressed signal data lengths; for each modality, simultaneously performing distributed collaborative sampling on the original signal data based on the adaptive measurement matrix to obtain the corresponding compressed observation vector matrix for each modality; constructing a global low-dimensional observation vector matrix based on the compressed observation vector matrices of each modality using an observation stacking method; and performing global compressed sensing modeling on the joint sparse representation model based on the low-dimensional observation vector matrix to obtain the compressed observation fusion model.

[0124] Step 204: Optimize and solve the compressed observation fusion model to obtain the multimodal complete reconstructed signal of the multimodal original signal dataset.

[0125] This step mainly involves optimizing the compressed observation fusion model to obtain a fully reconstructed multimodal signal from the original multimodal signal dataset.

[0126] Specifically, when solving the problem, the data reconstruction problem of the compressed observation fusion model can first be transformed into the problem of solving for the sparsest coefficient vector:

[0127] ;

[0128] in, The coefficient represents the trade-off between sparsity penalty and graph regularization, controlling the balance between sparsity and modal consistency of the reconstructed signal. To reconstruct the error tolerance (noise level), satisfy ; The graph Laplace regularization term penalizes the difference in coefficients between associated modes:

[0129] ;

[0130] in, For modality and The correlation weights between them are determined by the correlation analysis results of the preceding steps: when , ,otherwise ; Representing modes The corresponding sparse coefficient subvector.

[0131] The Alternating Direction Method of Multipliers (ADMM) is used to iteratively solve the reconstruction and fusion subproblems alternately. The above convex optimization problem is solved by combining optimization algorithms such as FISTA (Fast Iterative Shrinkage-Thresholding Algorithm). The iterative steps are briefly described as follows:

[0132] ;

[0133] in, Index for iteration count; Step size; gradient The Lipschitz constant; This is a soft thresholding proximal operator; This represents element-wise multiplication; Let be the objective function; the gradient is calculated as follows: ;here Let be the graph Laplace matrix.

[0134] The final reconstructed signal is:

[0135] ;

[0136] in, This is the optimal sparse coefficient vector after convergence.

[0137] Based on the preceding content, the implementation process for optimizing the compressed observation fusion model to obtain the fully reconstructed multimodal signal from the original multimodal signal dataset can include: transforming the data reconstruction problem of the compressed observation fusion model into an optimal sparsity problem of the sparse coefficient vector in the joint sparse representation model; using a fast iterative optimization algorithm and the alternating direction multiplier method to iteratively solve the data reconstruction problem and the optimal sparsity problem to obtain the converged optimal sparse coefficient vector; and reconstructing the joint sparse representation model based on the optimal sparse coefficient vector to obtain the fully reconstructed multimodal signal from the original multimodal signal dataset.

[0138] By studying feature fusion and model building of multimodal data, an integrated fusion-reconstruction model was constructed. This model embeds multimodal features, decisions, and models into the reconstruction optimization problem, achieving cross-modal data compression and reconstruction. Joint solution enables high-precision reconstruction and low-latency processing, improving data transmission efficiency and communication resource utilization.

[0139] In some embodiments, the reconstructed signal data can be further processed using post-processing optimization techniques. Specifically, Discrete Wavelet Transform (DWT) is performed on the reconstructed signal, and soft thresholding is applied to the detail coefficients to suppress noise.

[0140] ;

[0141] in, For the first Scale No. Wavelet coefficients of position; The threshold (usually set based on the Donoho-Johnstein criterion, taking...) (This is for estimating the noise standard deviation). An inverse transform after processing yields the denoised signal. Post-processing optimization techniques for denoising and enhancing the data can significantly improve its quality and usability.

[0142] Therefore, in the specific implementation, after obtaining the multimodal complete reconstructed signal through data processing, the multimodal complete reconstructed signal can be further denoised and then enhanced by discrete wavelet transform to obtain the multimodal denoised reconstructed signal.

[0143] This invention provides a virtual power plant data processing method based on compressed sensing. First, the massive, diverse, and heterogeneous resource data of the virtual power plant is preprocessed to ensure data quality, consistency, and applicability, meeting subsequent analysis requirements. Second, by optimizing algorithm design, compressed sensing technology is used to improve the efficiency and performance of processing all resource data of the virtual power plant. Finally, by studying feature fusion and model construction of multimodal data, cross-modal data compression and reconstruction are achieved, improving data transmission efficiency and communication resource utilization.

[0144] The technical solution provided by this invention, on the one hand, significantly reduces the amount of data by employing compressed sensing sparse sampling for the massive, heterogeneous, multimodal, and multidimensional sparse resource data of virtual power plants, achieving high-speed processing and real-time response, and significantly improving data processing efficiency and speed. On the other hand, by constructing a multimodal processing model, heterogeneous data is effectively integrated, supporting cross-modal data fusion and reconstruction, and improving information integrity and decision accuracy. Simultaneously, during the data compression sampling process, a dynamic adaptive optimization mechanism is adopted, which can automatically adjust parameters according to the real-time data stream, making it more adaptable to different scenarios and enhancing system flexibility and robustness. Furthermore, this invention employs a high compression ratio to reduce bandwidth consumption, lowering transmission and storage pressure and improving system economy.

[0145] For better explanation, refer to Figure 4 This diagram illustrates the overall flow of a data processing method for a virtual power plant according to an embodiment of the present invention. It should be noted that this embodiment only provides a brief description of the general flow of data processing in a virtual power plant. The specific implementation process of each step can be understood by referring to the relevant content in the foregoing embodiments, and will not be elaborated upon here. It is understood that the present invention does not impose any limitations on this.

[0146] Step 401: Collect multimodal sparse raw data of the virtual power plant, and first process the missing value and outlier value of the multimodal sparse raw data, and then perform standardization and normalization to obtain the multimodal raw signal dataset.

[0147] Step 402: Perform data processing based on feature extraction and pattern recognition on the multimodal raw signal dataset to obtain the feature vector matrix, sparsity evaluation value and correlation value between features;

[0148] Step 403: When the sparsity evaluation value and the correlation value between features meet the preset conditions, construct a joint sparse representation model based on the multimodal original signal dataset and feature vector matrix;

[0149] Step 404: Introduce a multimodal adaptive measurement matrix, and obtain a compressed observation fusion model by performing compressed sensing modeling on the joint sparse representation model;

[0150] Step 405: Optimize and solve the compressed observation fusion model to obtain the multimodal complete reconstructed signal of the multimodal original signal dataset;

[0151] Step 406: First, perform denoising on the multimodal complete reconstructed signal using discrete wavelet transform, and then perform enhancement processing to obtain the multimodal denoised reconstructed signal.

[0152] Reference Figure 5 The diagram illustrates a structural block diagram of a data processing device for a virtual power plant according to an embodiment of the present invention, which may specifically include:

[0153] The feature extraction and pattern recognition unit 501 is used to acquire the multimodal raw signal dataset of the virtual power plant, and to perform data processing based on feature extraction and pattern recognition on the multimodal raw signal dataset to obtain the feature vector matrix, sparsity evaluation value and correlation value between features;

[0154] The joint sparse representation model construction unit 502 is used to construct a joint sparse representation model based on the multimodal original signal dataset and the feature vector matrix when the sparsity evaluation value and the correlation value between features meet the preset conditions.

[0155] The compressed sensing modeling unit 503 is used to introduce a multimodal adaptive measurement matrix and obtain a compressed observation fusion model by performing compressed sensing modeling on the joint sparse representation model.

[0156] The model optimization and solution unit 504 is used to optimize and solve the compressed observation fusion model to obtain the multimodal complete reconstructed signal of the multimodal original signal dataset.

[0157] In one optional embodiment, the feature extraction and pattern recognition unit 501 includes:

[0158] The feature extraction unit is used to extract features from the multimodal raw signal dataset to obtain a feature vector matrix;

[0159] A sparsity identification unit is used to identify the sparsity of the feature vectors in the feature vector matrix under a specified transformation domain and obtain a sparsity evaluation value.

[0160] The correlation quantification analysis unit is used to quantify the correlation between feature vectors within the feature vector matrix and obtain the correlation value between features.

[0161] In one alternative embodiment, the joint sparse representation model construction unit 502 includes:

[0162] A multimodal high-dimensional vector matrix construction unit is used to stack the original multimodal signal dataset into a multimodal high-dimensional vector matrix by data stacking when the sparsity evaluation value is greater than a preset sparsity evaluation threshold and the correlation value between features is greater than a preset correlation threshold between features.

[0163] A joint sparse representation model construction subunit is used to construct a joint sparse representation model based on the multimodal high-dimensional vector matrix, while considering the specified transform domain and sparse coefficient vector.

[0164] The specified transform domain is obtained by optimizing a joint dictionary learning algorithm based on a pre-designed sparse basis of different modes.

[0165] In one alternative embodiment, the compressed sensing modeling unit 503 includes:

[0166] An adaptive sampling rate model construction unit is used to construct an adaptive sampling rate model based on the feature vector matrix through an adaptive linear mapping.

[0167] The raw signal data extraction unit is used to extract the raw signal data of each mode and the length of the raw signal data from the multimodal raw signal dataset;

[0168] An adaptive measurement matrix construction unit is used to, for each mode, combine a sampling rate enhancement mechanism based on data mutation, use the adaptive sampling rate model to adaptively compress the length of the original signal data to obtain the compressed signal data length, and construct the adaptive measurement matrix corresponding to the mode based on the original signal data length and the compressed signal data length.

[0169] A distributed cooperative sampling unit is used to perform distributed cooperative sampling of the original signal data for each of the modes, based on the adaptive measurement matrix, to obtain the compressed observation vector matrix corresponding to each of the modes.

[0170] The low-dimensional observation vector matrix construction unit is used to construct a global low-dimensional observation vector matrix by stacking observation values ​​based on the compressed observation vector matrices of each mode.

[0171] The global compressed sensing modeling subunit is used to perform global compressed sensing modeling on the joint sparse representation model based on the low-dimensional observation vector matrix to obtain a compressed observation fusion model.

[0172] In one optional embodiment, the model optimization solution unit 504 includes:

[0173] The problem transformation unit is used to transform the data reconstruction problem of the compressed observation fusion model into the optimal sparsity problem of the sparse coefficient vector in the joint sparse representation model;

[0174] The alternating iterative solution unit is used to combine a fast iterative optimization algorithm and use the alternating direction multiplier method to alternately iteratively solve the data reconstruction problem and the optimal sparsity problem to obtain the converged optimal sparse coefficient vector.

[0175] The signal reconstruction unit is used to reconstruct the joint sparse representation model based on the optimal sparse coefficient vector to obtain the multimodal complete reconstructed signal of the multimodal original signal dataset.

[0176] In one alternative embodiment, the device further includes:

[0177] A multimodal sparse raw data acquisition unit is used to acquire multimodal sparse raw data from a virtual power plant.

[0178] The data preprocessing unit is used to first process the missing value and outlier value of the multimodal sparse raw data, and then perform standardization and normalization processing to obtain the multimodal raw signal dataset of the virtual power plant.

[0179] In one alternative embodiment, the device further includes:

[0180] The data post-processing unit is used to first denoise the multimodal complete reconstructed signal through discrete wavelet transform, and then enhance it to obtain a multimodal denoised reconstructed signal.

[0181] As the device embodiment is basically similar to the method embodiment, it is described in a relatively simple way. For relevant details, please refer to the description of the method embodiment above.

[0182] This invention also provides an electronic device, which includes a processor and a memory:

[0183] The memory is used to store program code and transfer the program code to the processor;

[0184] The processor is used to execute the data processing method of the virtual power plant according to the instructions in the program code of any embodiment of the present invention.

[0185] This invention also provides a computer-readable storage medium for storing program code for executing the data processing method of a virtual power plant according to any embodiment of this invention.

[0186] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0187] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0188] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0189] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0190] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0191] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0192] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data processing method of a virtual power plant, characterized by, include: A multimodal raw signal dataset of a virtual power plant is acquired, and data processing based on feature extraction and pattern recognition is performed on the multimodal raw signal dataset to obtain a feature vector matrix, sparsity evaluation value, and correlation value between features. When the sparsity evaluation value and the correlation value between features meet the preset conditions, a joint sparse representation model is constructed based on the multimodal original signal dataset and the feature vector matrix. By introducing a multimodal adaptive measurement matrix and performing compressed sensing modeling on the joint sparse representation model, a compressed observation fusion model is obtained. The compressed observation fusion model is optimized and solved to obtain the multimodal complete reconstructed signal of the multimodal original signal dataset.

2. The data processing method of a virtual power plant according to claim 1, wherein, The data processing based on feature extraction and pattern recognition on the multimodal raw signal dataset to obtain feature vector matrices, sparsity evaluation values, and inter-feature correlation values ​​includes: Feature extraction is performed on the original multimodal signal dataset to obtain a feature vector matrix; The sparsity of the eigenvectors within the eigenvector matrix under a specified transform domain is identified to obtain a sparsity evaluation value. The correlation between eigenvectors within the eigenvector matrix is ​​quantitatively analyzed to obtain the correlation value between features. 3.The data processing method of a virtual power plant according to claim 1 or 2, characterized in that, When the sparsity evaluation value and the correlation value between features meet preset conditions, a joint sparse representation model is constructed based on the multimodal original signal dataset and the feature vector matrix, including: When the sparsity evaluation value is greater than the preset sparsity evaluation threshold, and the correlation value between features is greater than the preset correlation threshold between features, the multimodal original signal dataset is stacked into a multimodal high-dimensional vector matrix by data stacking. Based on the multimodal high-dimensional vector matrix, and considering both the specified transform domain and the sparse coefficient vector, a joint sparse representation model is constructed. The specified transform domain is obtained by optimizing a joint dictionary learning algorithm based on a pre-designed sparse basis of different modes. 4.The data processing method of a virtual power plant according to claim 1 or 2, characterized in that, The introduction of a multimodal adaptive measurement matrix, and the obtaining of a compressed observation fusion model through compressed sensing modeling of the joint sparse representation model, includes: Based on the feature vector matrix, an adaptive sampling rate model is constructed through an adaptive linear mapping; Extract the original signal data and the length of each mode from the multimodal original signal dataset; For each mode, the original signal data length is adaptively compressed using the adaptive sampling rate model in conjunction with the sampling rate enhancement mechanism based on data mutation, to obtain the compressed signal data length. Based on the original signal data length and the compressed signal data length, an adaptive measurement matrix corresponding to the mode is constructed. For each of the aforementioned modes, the original signal data is simultaneously subjected to distributed cooperative sampling based on the adaptive measurement matrix to obtain the compressed observation vector matrix corresponding to each of the aforementioned modes. Based on the compressed observation vector matrix of each mode, a global low-dimensional observation vector matrix is ​​constructed by stacking observation values. Based on the low-dimensional observation vector matrix, global compressed sensing modeling is performed on the joint sparse representation model to obtain a compressed observation fusion model. 5.The data processing method of a virtual power plant according to claim 1 or 2, characterized in that, The optimization and solution of the compressed observation fusion model to obtain the multimodal complete reconstructed signal of the multimodal original signal dataset includes: The data reconstruction problem of the compressed observation fusion model is transformed into the optimal sparsity problem of the sparse coefficient vector in the joint sparse representation model; Combining a fast iterative optimization algorithm, the alternating direction multiplier method is used to iteratively solve the data reconstruction problem and the optimal sparsity problem to obtain the converged optimal sparse coefficient vector. The joint sparse representation model is reconstructed based on the optimal sparse coefficient vector to obtain the multimodal complete reconstructed signal of the original multimodal signal dataset. 6.The data processing method of a virtual power plant according to claim 1, wherein, Also includes: Collect multimodal sparse raw data from a virtual power plant; The multimodal sparse raw data is first processed for missing values ​​and outliers, and then standardized and normalized to obtain the multimodal raw signal dataset of the virtual power plant.

7. The data processing method of a virtual power plant according to claim 1, wherein, Also includes: The multimodal reconstructed signal is first denoised by discrete wavelet transform, and then enhanced to obtain a multimodal denoised reconstructed signal.

8. A data processing apparatus of a virtual power plant, characterized by, include: The feature extraction and pattern recognition unit is used to acquire the multimodal raw signal dataset of the virtual power plant, and to perform data processing based on feature extraction and pattern recognition on the multimodal raw signal dataset to obtain the feature vector matrix, sparsity evaluation value and correlation value between features; The joint sparse representation model construction unit is used to construct a joint sparse representation model based on the multimodal original signal dataset and the feature vector matrix when the sparsity evaluation value and the correlation value between features meet the preset conditions. The compressed sensing modeling unit is used to introduce a multimodal adaptive measurement matrix and obtain a compressed observation fusion model by performing compressed sensing modeling on the joint sparse representation model. The model optimization and solution unit is used to optimize and solve the compressed observation fusion model to obtain the multimodal complete reconstructed signal of the multimodal original signal dataset.

9. An electronic device, comprising: The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the data processing method of the virtual power plant according to any one of claims 1-7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for executing the data processing method of the virtual power plant according to any one of claims 1-7.

Images