A carbon trading data prediction method and system based on big data

Abstract

The present invention discloses a carbon trading data prediction method and system based on big data, which relates to the field of carbon trading prediction technology. The method includes: obtaining the geographical location characteristics and historical carbon trading matrix of each virtual power plant; performing cluster analysis on the virtual power plants according to the geographical location characteristics and the historical carbon trading matrix, and dividing all virtual power plants into several federated learning sub-nodes; obtaining historical climate characteristic data, and constructing a global prediction model and a local prediction model of each federated learning sub-node according to the historical climate characteristic data and the historical carbon trading matrix; inputting the climate characteristic data and carbon trading matrix obtained in real time into the local prediction model of the corresponding federated learning sub-node to obtain the carbon trading data prediction result. This application can provide a data basis for the operation and decision-making of virtual power plants, and realize effective resource allocation and environmental management.

Description

Technical Field

[0001] The present invention relates to the technical field of carbon trading prediction, and in particular to a carbon trading data prediction method and system based on big data. Background Art

[0002] The power industry faces immense pressure from carbon emissions, making carbon trading data forecasting crucial. In recent years, academia and industry have proposed new methods to improve the accuracy and efficiency of carbon trading forecasting. For example, big data analysis techniques and machine learning methods have been applied to carbon trading forecasting, mining vast amounts of historical trading data to build more complex forecasting models.

[0003] However, existing approaches still face several problems and challenges. First, limited computing resources make accurate predictions difficult when processing the vast and widely distributed carbon trading data. Second, due to the significant differences in carbon trading models between different companies, existing approaches struggle to personalize and manage data from these different models, resulting in inadequate operational efficiency and decision-making accuracy. Summary of the Invention

[0004] In order to solve the problems existing in the above background technology, the present invention adopts the following technical solutions:

[0005] A carbon trading data prediction method based on big data, comprising the steps of:

[0006] Obtain the geographical location characteristics and historical carbon trading matrix of each virtual power plant; the historical carbon trading matrix includes the power supply transaction matrix between the virtual power plant and each power supply terminal and the power sales transaction matrix of each power user terminal; the power supply transaction matrix stores power supply carbon trading data with energy type and sampling time as index; the power sales transaction matrix stores power sales carbon trading data with customer type and sampling time as index;

[0007] Cluster analysis of virtual power plants is performed based on geographical location characteristics and historical carbon trading matrices, and all virtual power plants are divided into several federated learning sub-nodes; each federated training sub-node includes one or more virtual power plants;

[0008] Obtain historical climate characteristic data, and build a global prediction model and local prediction models for each federated learning sub-node based on the historical climate characteristic data and the historical carbon trading matrix;

[0009] The real-time climate characteristic data and carbon trading matrix are input into the local prediction model of the corresponding federated learning sub-node to obtain the carbon trading data prediction results.

[0010] As a preferred solution, the cluster analysis of virtual power plants based on geographical location characteristics and historical carbon trading matrix includes the following steps:

[0011] S21. Extracting the geographical location characteristics of each virtual power plant and the standardized numerical characteristics of the historical carbon trading matrix;

[0012] S22, initialize the K value of the K-means algorithm and select K virtual power plants as the initial cluster centers;

[0013] S23, assigning each virtual power plant to the nearest center point in the cluster according to the distance metric value to form K clusters;

[0014] S24, updating the center point within each cluster according to the mean value of the standard numerical characteristics of the virtual power plant within each cluster;

[0015] S25. Repeat steps S23 and S24 until the change amplitude of the center points in all clusters is less than the preset threshold; each cluster is recorded as a federated learning sub-node.

[0016] As a preferred solution, the extraction of geographical location characteristics of each virtual power plant and standard numerical characteristics of the historical carbon trading matrix specifically includes:

[0017] Standardize the power supply carbon trading data and power sales carbon trading data based on the geographical location characteristics and historical carbon trading matrix to obtain standard geographical location characteristics, standard power supply trading matrix, and standard power sales trading matrix.

[0018] The basic statistical features of each data column of the standard power supply transaction matrix and the standard power sales transaction matrix are calculated, and the standard geographical location features and basic statistical features are merged into standard numerical features.

[0019] As a preferred solution, the K value of the initialization K-means algorithm specifically includes:

[0020] Set the K value range, calculate the total square error SSE corresponding to each K value within the K value range and draw the K-SSE relationship curve; the K value range is , where N is the number of virtual power plants, Express Round up;

[0021] The K value corresponding to the second-order difference value in the K-SSE relationship curve being greater than the preset second-order difference threshold is selected as the K value of the K-means algorithm;

[0022] The total error sum of squares is expressed as:

[0023] ,

[0024] Where K represents the number of clusters, represents the i-th cluster, represents the virtual power plant in the i-th cluster, represents the center point in the i-th cluster, express and The distance metric of .

[0025] As a preferred solution, the second-order difference value is expressed as:

[0026] ,

[0027] ,

[0028] in, represents the total error sum of squares corresponding to the K value, Indicates the rate of decrease of the total square error corresponding to the K value.

[0029] As a preferred solution, the step of selecting K virtual power plants as the initial cluster centers is specifically selecting K virtual power plants as the initial cluster centers by using a maximum distance method, including the following steps:

[0030] S2221. Select the two virtual power plants with the largest distance measurement values ​​of the standard numerical features among all virtual power plants as the center points within the cluster;

[0031] S2222. Allocate the remaining virtual power plants to the corresponding clusters according to their distance measurement values ​​from the cluster center points, and update the cluster center points;

[0032] S2223, in each cluster, find the virtual power plant with the largest distance measurement value from the cluster center point as the added cluster center point;

[0033] S2224. Repeat steps S2222 and S2223 until the number of center points in the cluster reaches K.

[0034] As a preferred solution, the construction of a global prediction model and a local prediction model of each federated learning sub-node based on historical climate characteristic data and a historical carbon trading matrix includes cyclically executing the following steps:

[0035] The global model parameters of the global prediction model are transferred to the federated learning sub-node, and the local model parameters corresponding to the federated learning sub-node are set according to the global model parameters;

[0036] Train the local prediction model based on the historical climate characteristic data and historical carbon trading matrix of the federated learning sub-node;

[0037] Regularly transmit the updated local model parameters of each federated learning sub-node to the central server;

[0038] The local model parameters of each federated learning sub-node are aggregated through the federated averaging algorithm to update the global prediction model.

[0039] As a preferred solution, the federated averaging algorithm is expressed as:

[0040]

[0041] in, are the model parameters of the updated global prediction model, For the The model parameters of the local prediction model, is the number of local prediction models.

[0042] As a preferred solution, the global prediction model and the local prediction model both include a personalized hidden layer and a global hidden layer;

[0043] The transmitting of the updated local model parameters of each federated learning sub-node to the central server is specifically transmitting the updated local model parameters corresponding to the global hidden layer of each federated learning sub-node to the central server;

[0044] The parameter aggregation of the local model parameters of each federated learning sub-node by using the federated averaging algorithm is specifically: parameter aggregation of the local model parameters corresponding to the global hidden layer of each federated learning sub-node by using the federated averaging algorithm.

[0045] The present invention also provides a carbon trading data prediction system based on big data, which applies the carbon trading data prediction method based on big data as described above, including:

[0046] The data acquisition module is used to obtain the geographical location characteristics and historical carbon trading matrix of each virtual power plant, as well as real-time climate characteristic data and carbon trading matrix; the historical carbon trading matrix includes the power supply transaction matrix and the power sales transaction matrix;

[0047] Cluster analysis module, used to perform cluster analysis on virtual power plants based on geographical location characteristics and historical carbon trading matrix, and divide all virtual power plants into several federated learning sub-nodes;

[0048] The model building module is used to obtain historical climate characteristic data and build a global prediction model and a local prediction model for each federated learning sub-node based on the historical climate characteristic data and the historical carbon trading matrix;

[0049] The real-time prediction module is used to input the real-time climate characteristic data and carbon trading matrix into the local prediction model of the corresponding federated learning sub-node to obtain the carbon trading data prediction results.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] This application distributes the prediction model training process to multiple federated learning sub-nodes through a federated learning architecture to fully utilize computing resources and reduce the computing pressure on a single server; through cluster analysis, virtual power plants with similar geographical location characteristics and historical carbon trading characteristics are aggregated to form federated learning sub-nodes. The virtual power plants within each federated learning sub-node have similar climate characteristics and carbon trading patterns. By sharing training data sets within the federated learning sub-nodes, the overfitting problem that is prone to occur in the local prediction model in the early stage of training due to the small amount of data can be avoided, and the efficiency of personalized learning can be improved, avoiding the waste of resources caused by personalized training; by inputting real-time climate characteristic data and carbon trading matrix into the local prediction model to obtain personalized carbon trading data prediction results, a data basis can be provided for the operation and decision-making of the virtual power plant, thereby realizing effective resource allocation and environmental management.

[0052] This application extracts the geographical location characteristics of each virtual power plant and the standard numerical characteristics of the historical carbon trading matrix, and performs cluster analysis on virtual power plants through the K-means algorithm. It can effectively perform cluster analysis on virtual power plants with wide distribution and complex characteristics, and can effectively capture the commonalities of virtual power plants with similar geographical location characteristics and carbon trading models. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0055] Figure 1 A schematic diagram of the process of the carbon trading data prediction method based on big data provided in this embodiment;

[0056] Figure 2 A schematic diagram of the process for extracting the geographical location characteristics of each virtual power plant and the standard numerical characteristics of the historical carbon trading matrix provided in this embodiment;

[0057] Figure 3 A schematic diagram of the process of building a global prediction model and local prediction models for each federated learning sub-node based on historical climate characteristic data and a historical carbon trading matrix provided in this embodiment;

[0058] Figure 4 A schematic diagram of the structure of a carbon trading data prediction system based on big data provided in this embodiment;

[0059] Figure 5 This is a schematic structural diagram of an electronic device provided in this embodiment. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0061] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0062] In addition, the descriptions of "first", "second", etc. in the present invention are for descriptive purposes only and should not be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0063] The power industry faces immense pressure from carbon emissions, making carbon trading data forecasting crucial. In recent years, academia and industry have proposed new methods to improve the accuracy and efficiency of carbon trading forecasting. For example, big data analysis techniques and machine learning methods have been applied to carbon trading forecasting, mining vast amounts of historical trading data to build more complex forecasting models.

[0064] However, existing approaches still face several problems and challenges. First, limited computing resources make accurate predictions difficult when processing the vast and widely distributed carbon trading data. Second, due to the significant differences in carbon trading models between different companies, existing approaches struggle to personalize and manage data from these different models, resulting in inadequate operational efficiency and decision-making accuracy.

[0065] The present invention provides a carbon trading data prediction method and system based on big data. The specific implementation scheme of the present invention will be described in detail below.

[0066] Example 1

[0067] like Figure 1 As shown, a carbon trading data prediction method based on big data includes the following steps:

[0068] S1. Obtain the geographical location characteristics and historical carbon trading matrix of each virtual power plant; the historical carbon trading matrix includes the power supply trading matrix between the virtual power plant and each power supply terminal and the power sales trading matrix of each power user; the power supply trading matrix stores power supply carbon trading data with energy type and sampling time as indexes; the power sales trading matrix stores power sales carbon trading data with customer type and sampling time as indexes;

[0069] S2. Perform cluster analysis on virtual power plants based on their geographical location characteristics and historical carbon trading matrix, and divide all virtual power plants into several federated learning sub-nodes; each federated training sub-node includes one or more virtual power plants;

[0070] S3. Obtain historical climate characteristic data, and build a global prediction model and a local prediction model for each federated learning sub-node based on the historical climate characteristic data and the historical carbon trading matrix;

[0071] S4. Input the real-time climate characteristic data and carbon trading matrix into the local prediction model of the corresponding federated learning sub-node to obtain the carbon trading data prediction result.

[0072] This application distributes the prediction model training process to multiple federated learning sub-nodes through a federated learning architecture to fully utilize computing resources and reduce the computing pressure on a single server; through cluster analysis, virtual power plants with similar geographical location characteristics and historical carbon trading characteristics are aggregated to form federated learning sub-nodes. The virtual power plants within each federated learning sub-node have similar climate characteristics and carbon trading patterns. By sharing training data sets within the federated learning sub-nodes, the overfitting problem that is prone to occur in the local prediction model in the early stage of training due to the small amount of data can be avoided, and the efficiency of personalized learning can be improved, avoiding the waste of resources caused by personalized training; by inputting real-time climate characteristic data and carbon trading matrix into the local prediction model to obtain personalized carbon trading data prediction results, a data basis can be provided for the operation and decision-making of the virtual power plant, thereby realizing effective resource allocation and environmental management.

[0073] Specifically, the steps of the carbon trading data prediction method based on big data of the present invention are described in detail through the following content:

[0074] A carbon trading data prediction method based on big data, comprising the steps of:

[0075] S1. Obtain the geographical location characteristics and historical carbon trading matrix of each virtual power plant; the historical carbon trading matrix includes the power supply trading matrix between the virtual power plant and each power supply terminal and the power sales trading matrix of each power user; the power supply trading matrix stores power supply carbon trading data with energy type and sampling time as indexes; the power sales trading matrix stores power sales carbon trading data with customer type and sampling time as indexes;

[0076] In this embodiment, the power supply transaction matrix is ​​used to record and analyze energy transactions between the virtual power plant and the power supply end. The power supply transaction matrix uses energy type and sampling time as indexes, for example, with energy type as the row and sampling time as the column, to store power supply carbon trading data.

[0077] The electricity sales transaction matrix is ​​used to record and analyze the energy transactions between the virtual power plant and the electricity user. The power supply transaction matrix uses energy type and sampling time as indexes, for example, customer type as row and sampling time as column to store electricity sales carbon transaction data.

[0078] In one embodiment, the geographical location characteristics include longitude, latitude, climate type and economic development indicators.

[0079] In one embodiment, the power supply carbon trading data and the power sales carbon trading data are both arrays consisting of electricity trading volume (MWh), carbon emissions (tons) and average transaction price (yuan / ton).

[0080] S2. Perform cluster analysis on virtual power plants based on their geographical location characteristics and historical carbon trading matrix, and divide all virtual power plants into several federated learning sub-nodes; each federated training sub-node includes one or more virtual power plants;

[0081] Furthermore, the cluster analysis of virtual power plants based on geographical location characteristics and historical carbon trading matrix includes the following steps:

[0082] S21. Extracting the geographical location characteristics of each virtual power plant and the standardized numerical characteristics of the historical carbon trading matrix;

[0083] Among them, the values ​​of the extracted standardized numerical features are distributed in the same range to eliminate the influence of dimension;

[0084] S22, initialize the K value of the K-means algorithm and select K virtual power plants as the initial cluster centers;

[0085] S23, assigning each virtual power plant to the nearest center point in the cluster according to the distance metric value to form K clusters;

[0086] The distance metric represents the Euclidean distance between the virtual power plant in the selected cluster and the standardized numerical features of the center point in the cluster;

[0087] S24, updating the center point within each cluster according to the mean value of the standard numerical characteristics of the virtual power plant within each cluster;

[0088] S25. Repeat steps S23 and S24 until the change amplitude of the center points in all clusters is less than the preset threshold; each cluster is recorded as a federated learning sub-node.

[0089] This embodiment extracts the geographical location characteristics of each virtual power plant and the standard numerical features of the historical carbon trading matrix, and performs cluster analysis on the virtual power plants using the K-means algorithm. This can effectively perform cluster analysis on virtual power plants with widespread distribution and complex characteristics, and can effectively capture the commonalities of virtual power plants with similar geographical location characteristics and carbon trading patterns. For example, virtual power plants located in the same climate zone face similar climate changes, which can enable the model to better understand and utilize these regional characteristics and improve prediction accuracy. The federated learning subnodes formed by the clustering results provide a structured foundation for subsequent federated learning to improve the training effect of the local prediction model.

[0090] In one embodiment, please refer to Figure 2 In step S21, the extraction of the geographical location characteristics of each virtual power plant and the standard numerical characteristics of the historical carbon trading matrix specifically includes:

[0091] Standardize the power supply carbon trading data and the power sales carbon trading data (including electricity trading volume, carbon emissions, and average transaction price) based on the geographic location characteristics and historical carbon trading matrix to obtain standard geographic location characteristics, a standard power supply trading matrix, and a standard power sales trading matrix. Z-score normalization or Min-Max normalization can be used for standardization.

[0092] Calculate the basic statistical features for each data column of the standard power supply transaction matrix and the standard power sales transaction matrix, and combine the standard geographic location features and basic statistical features into standard numerical features. The basic statistical features of each data column in the standard power supply transaction matrix represent the mean, standard deviation, maximum, and minimum values ​​corresponding to each energy type within a certain statistical time period; the basic statistical features of each data column in the standard power sales transaction matrix represent the mean, standard deviation, maximum, and minimum values ​​corresponding to each customer type within a certain statistical time period. In the standard numerical features, each basic statistical quantity and standard geographic location feature is considered a dimension of the standard numerical features.

[0093] In one embodiment, the initialization of the K value of the K-means algorithm specifically includes:

[0094] S2211, set the K value range, calculate the total square error SSE corresponding to each K value within the K value range and draw a K-SSE relationship curve; the K value range is , where N is the number of virtual power plants, Express Round up. The total error square sum is expressed as:

[0095] ,

[0096] Where K represents the number of clusters, represents the i-th cluster, represents the virtual power plant in the i-th cluster, represents the center point in the i-th cluster, express and The distance metric of .

[0097] Furthermore, the distance metric is the Euclidean distance between the standard numerical features of the virtual power plant in the cluster and the center point in the cluster. The distance metric is expressed as:

[0098] ,

[0099] in, represents the dimension of the standard numerical features, represents the jth eigenvalue of the standard numerical characteristics of the virtual power plant, Represents the jth eigenvalue of the standardized numerical feature of the center point in the cluster.

[0100] S2212. Select the K value corresponding to when the second-order difference value in the K-SSE relationship curve is greater than a preset second-order difference threshold as the K value of the K-means algorithm.

[0101] In the K-SSE relationship curve, when a smaller K value is selected, increasing K can significantly improve the similarity within the cluster (reduce SSE), and the curve drops rapidly at this time. After exceeding the optimal K value, increasing the K value has little effect on improving SSE, and the curve tends to be flat. Therefore, in order to make the initialized K value both obvious in the difference between clusters and not cause overfitting problems due to too many clusters, this embodiment needs to select a K value with a significantly reduced slope in the K-SSE relationship curve as the K value of the K-means algorithm, which is specifically calculated through the relationship between the second-order difference value and the preset second-order difference threshold. Among them, the second-order difference value is expressed as:

[0102] ,

[0103] ,

[0104] in, represents the total error sum of squares corresponding to the K value, Indicates the rate of decrease of the total square error corresponding to the K value.

[0105] K virtual power plants are selected as the initial cluster centers, and subsequent virtual power plant division is performed accordingly. The selection method of the initial cluster centers is random in some embodiments, and the maximum distance method is used in some embodiments.

[0106] Based on the foregoing, in one embodiment, K virtual power plants are selected as initial cluster centers. Specifically, the K virtual power plants are selected as initial cluster centers using a maximum distance method, including the following steps:

[0107] S2221. Select the two virtual power plants with the largest distance measurement values ​​of the standard numerical features among all virtual power plants as the center points within the cluster;

[0108] S2222. Allocate the remaining virtual power plants to the corresponding clusters according to their distance measurement values ​​from the cluster center points, and update the cluster center points;

[0109] S2223, in each cluster, find the virtual power plant with the largest distance measurement value from the cluster center point as the added cluster center point;

[0110] S2224. Repeat steps S2222 and S2223 until the number of center points in the cluster reaches K.

[0111] In this embodiment, the maximum distance method ensures that the initially selected center points are widely distributed in the feature space and can more fully represent the different feature areas in the data set; since the initial center points are better distributed, the selection based on the maximum distance method can reduce the number of unnecessary iterations, thereby accelerating the convergence speed of the clustering algorithm, reducing the consumption of computing resources, and improving computing efficiency.

[0112] S3. Obtain historical climate characteristic data, and build a global prediction model and a local prediction model for each federated learning sub-node based on the historical climate characteristic data and the historical carbon trading matrix;

[0113] Further, see Figure 3 The process of constructing a global prediction model and a local prediction model for each federated learning sub-node based on historical climate characteristic data and a historical carbon trading matrix includes looping through the following steps:

[0114] S31, transmitting the global model parameters of the global prediction model to the federated learning sub-node, and setting the local model parameters corresponding to the federated learning sub-node according to the global model parameters;

[0115] At the beginning of the first cycle, the central server initializes and builds a global prediction model, then transmits its global model parameters to the federated learning child node. The global prediction model can employ models such as polynomial regression models, recurrent neural networks (RNNs), long short-term memory networks (LSTMs), and deep neural networks (DNNs). In a preferred embodiment, the global prediction model employs a long short-term memory network (LSTM), a type of neural network suitable for processing time series data. Due to its unique network structure, the LSTM network can capture long-term dependencies between climate characteristic data and the carbon trading matrix.

[0116] In subsequent cycles, the central server regularly publishes the latest global model parameters to each federated learning sub-node. After receiving the global model parameters, each federated learning sub-node initializes or updates its local model parameters to ensure consistency with the global model.

[0117] S32. Train a local prediction model based on the local historical climate characteristic data and historical carbon trading matrix of the federated learning child node;

[0118] Each federated learning node uses its own local dataset for training, including historical climate data and a historical carbon trading matrix. During training, it uses appropriate training algorithms (such as gradient descent and the Adam optimizer) to iterate and train the prediction model over multiple rounds. Model hyperparameters (such as the learning rate and batch size) are adjusted as needed.

[0119] S33, regularly transmitting the updated local model parameters of each federated learning sub-node to the central server;

[0120] At preset intervals, each federated learning node completes local training, generates an updated local prediction model, and transmits the corresponding local model parameters (such as weights and bias values) to the central server. This transmission process can use an encrypted transmission protocol to ensure data security and privacy protection.

[0121] S34. Aggregate the local model parameters of each federated learning sub-node using a federated averaging algorithm to update the global prediction model.

[0122] The federated averaging algorithm is expressed as:

[0123]

[0124] in, are the model parameters of the updated global prediction model, For the The model parameters of the local prediction model, is the number of local prediction models.

[0125] In this step, after receiving the local model parameters from all child nodes, the central server aggregates them using the federated averaging algorithm. This algorithm calculates the weighted average of each child node's model to update the global model parameters. After aggregation, the central server generates a new global prediction model and broadcasts it back to each node for the next round of training.

[0126] This embodiment uses federated averaging and parameter aggregation to continuously optimize the global model based on the characteristics of each sub-node, thereby improving the prediction performance of the global prediction model. Without exchanging actual data, parameter transmission and aggregation are used to achieve data privacy protection and privacy requirements between federated learning sub-nodes. The distributed computing method balances computing and data storage pressures, improving system scalability and response speed.

[0127] As a preferred embodiment, the global prediction model and the local prediction model both include a personalized hidden layer and a global hidden layer;

[0128] The transmitting of the updated local model parameters of each federated learning sub-node to the central server is specifically transmitting the updated local model parameters corresponding to the global hidden layer of each federated learning sub-node to the central server;

[0129] The parameter aggregation of the local model parameters of each federated learning sub-node by using the federated averaging algorithm is specifically: parameter aggregation of the local model parameters corresponding to the global hidden layer of each federated learning sub-node by using the federated averaging algorithm.

[0130] For example, based on the standard LSTM architecture, personalized layers are added. These layers are trained only on specific users or data sources. Therefore, the parameters of the personalized hidden layers are updated only locally, improving the model's ability to fit local data. The personalized hidden layers are designed to capture the unique data characteristics of each federated learning child node. Using the node's training data, the personalized layers are optimized through backpropagation, accelerating their adaptation to local data characteristics.

[0131] Global hidden layers, on the other hand, contain parameters shared across all nodes and are responsible for extracting common patterns in the data. During training, if parts of the global hidden layers are important (such as low-level features) and stable, these layers can be frozen, and only the layers needed to adapt to changing data patterns can be updated. If the global layers need to be more responsive to local data, all layers can be updated simultaneously. This is often used when the data distribution has changed significantly.

[0132] S4. Input the real-time climate characteristic data and carbon trading matrix into the local prediction model of the corresponding federated learning sub-node to obtain the carbon trading data prediction result.

[0133] Among them, the carbon trading data prediction results include predicted power supply carbon trading data and predicted power sales carbon trading data. Both the predicted power supply carbon trading data and the predicted power sales carbon trading data include the corresponding predicted electricity trading volume, predicted carbon emissions and predicted average transaction price, to help decision makers evaluate the economic and environmental impacts of carbon trading from multiple perspectives and optimize operational strategies.

[0134] Example 2

[0135] like Figure 4 As shown, a carbon trading data prediction system based on big data includes:

[0136] The data acquisition module is used to obtain the geographical location characteristics and historical carbon trading matrix of each virtual power plant, as well as real-time climate characteristic data and carbon trading matrix; the historical carbon trading matrix includes the power supply transaction matrix and the power sales transaction matrix;

[0137] Cluster analysis module, used to perform cluster analysis on virtual power plants based on geographical location characteristics and historical carbon trading matrix, and divide all virtual power plants into several federated learning sub-nodes;

[0138] The model building module is used to obtain historical climate characteristic data and build a global prediction model and a local prediction model for each federated learning sub-node based on the historical climate characteristic data and the historical carbon trading matrix;

[0139] The real-time prediction module is used to input the real-time climate characteristic data and carbon trading matrix into the local prediction model of the corresponding federated learning sub-node to obtain the carbon trading data prediction results.

[0140] It should be understood that the disclosed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the modules is merely a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, each functional module may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0141] Example 3

[0142] An electronic device 2, such as Figure 5 As shown, a processor 21 and a memory 22, the memory 22 is used to store computer program code, the computer program code includes computer instructions, when the processor 21 executes the computer instructions, the electronic device executes the above-mentioned carbon trading data prediction method based on big data.

[0143] The electronic device 2 includes a processor 21, a memory 22, an output device 23, and an input device 24. The processor 21, memory 22, input device 24, and output device 23 are coupled via a connector, which may include various interfaces, transmission lines, or buses, etc., although this is not limited in the present embodiment. It should be understood that in various embodiments of the present invention, coupling refers to interconnection in a specific manner, including direct connection or indirect connection through other devices, such as various interfaces, transmission lines, buses, etc.

[0144] The processor 21 may be one or more graphics processing units (GPUs). If the processor 21 is a GPU, the GPU may be a single-core GPU or a multi-core GPU. Alternatively, the processor 21 may be a processor group consisting of multiple GPUs, with the multiple processors coupled to each other via one or more buses. Alternatively, the processor may be another type of processor, and this is not limited in this embodiment of the present invention.

[0145] The memory 22 can be used to store computer program instructions and various computer program codes, including program codes for executing the embodiments of the present invention. Optionally, the memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), and is used for related instructions and data.

[0146] The input device 24 is used to input data and / or signals, and the output device 23 is used to output data and / or signals. The output device 23 and the input device 24 can be independent devices or an integrated device.

[0147] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but is intended to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A carbon trading data prediction method based on big data, characterized by: Including steps: Obtain the geographical location characteristics and historical carbon trading matrix of each virtual power plant; the historical carbon trading matrix includes the power supply transaction matrix between the virtual power plant and each power supply terminal and the power sales transaction matrix of each power user terminal; the power supply transaction matrix stores power supply carbon trading data with energy type and sampling time as index; the power sales transaction matrix stores power sales carbon trading data with customer type and sampling time as index; Cluster analysis of virtual power plants is performed based on geographical location characteristics and historical carbon trading matrices, and all virtual power plants are divided into several federated learning sub-nodes; each federated training sub-node includes one or more virtual power plants; Obtain historical climate characteristic data, and build a global prediction model and local prediction models for each federated learning sub-node based on the historical climate characteristic data and the historical carbon trading matrix; Input the real-time climate characteristic data and carbon trading matrix into the local prediction model of the corresponding federated learning sub-node to obtain the carbon trading data prediction results; The cluster analysis of virtual power plants based on geographical location characteristics and historical carbon trading matrices includes the steps of: extracting geographical location characteristics of each virtual power plant and standard numerical characteristics of the historical carbon trading matrix; The extraction of the geographical location characteristics of each virtual power plant and the standard numerical characteristics of the historical carbon trading matrix specifically includes: Standardize the power supply carbon trading data and power sales carbon trading data based on the geographical location characteristics and historical carbon trading matrix to obtain standard geographical location characteristics, standard power supply trading matrix, and standard power sales trading matrix. Calculate the basic statistical features of each data column of the standard power supply transaction matrix and the standard power sales transaction matrix, and merge the standard geographical location features and basic statistical features into standard numerical features; The process of constructing a global prediction model and a local prediction model for each federated learning sub-node based on historical climate characteristic data and a historical carbon trading matrix includes looping through the following steps: The global model parameters of the global prediction model are transferred to the federated learning sub-node, and the local model parameters corresponding to the federated learning sub-node are set according to the global model parameters; Train the local prediction model based on the historical climate characteristic data and historical carbon trading matrix of the federated learning sub-node; Regularly transmit the updated local model parameters of each federated learning sub-node to the central server; The local model parameters of each federated learning sub-node are aggregated through the federated averaging algorithm to update the global prediction model; The global prediction model and the local prediction model both include a personalized hidden layer and a global hidden layer; The transmitting of the updated local model parameters of each federated learning sub-node to the central server is specifically transmitting the updated local model parameters corresponding to the global hidden layer of each federated learning sub-node to the central server; The parameter aggregation of the local model parameters of each federated learning sub-node by using the federated averaging algorithm is specifically: parameter aggregation of the local model parameters corresponding to the global hidden layer of each federated learning sub-node by using the federated averaging algorithm.

2. The carbon trading data prediction method based on big data according to claim 1, characterized in that: The cluster analysis of virtual power plants based on geographical location characteristics and historical carbon trading matrix also includes the following steps: S22, initialize the K value of the K-means algorithm and select K virtual power plants as the initial cluster centers; S23, assigning each virtual power plant to the nearest center point in the cluster according to the distance metric value to form K clusters; S24, updating the center point within each cluster according to the mean value of the standard numerical characteristics of the virtual power plant within each cluster; S25. Repeat steps S23 and S24 until the change amplitude of the center points in all clusters is less than the preset threshold; each cluster is recorded as a federated learning sub-node.

3. The carbon trading data prediction method based on big data according to claim 2, characterized in that: The K value of the K-means algorithm is initialized, specifically including: Set the K value range, calculate the total square error SSE corresponding to each K value within the K value range and draw the K-SSE relationship curve; the K value range is , where N is the number of virtual power plants, Express Round up; The K value corresponding to the second-order difference value in the K-SSE relationship curve being greater than the preset second-order difference threshold is selected as the K value of the K-means algorithm; The total error sum of squares is expressed as: ， Where K represents the number of clusters, represents the i-th cluster, represents the virtual power plant in the i-th cluster, represents the center point in the i-th cluster, express and The distance metric of .

4. The carbon trading data prediction method based on big data according to claim 3 is characterized by: The second-order difference value is expressed as: ， ， in, represents the total error sum of squares corresponding to the K value, Indicates the rate of decrease of the total square error corresponding to the K value.

5. The carbon trading data prediction method based on big data according to claim 2, characterized in that: The step of selecting K virtual power plants as the initial cluster centers is specifically selecting K virtual power plants as the initial cluster centers by using a maximum distance method, including the following steps: S2221. Select the two virtual power plants with the largest distance measurement values ​​of the standard numerical features among all virtual power plants as the center points within the cluster; S2222. Allocate the remaining virtual power plants to the corresponding clusters according to their distance measurement values ​​from the cluster center points, and update the cluster center points; S2223, in each cluster, find the virtual power plant with the largest distance measurement value from the cluster center point as the added cluster center point; S2224. Repeat steps S2222 and S2223 until the number of center points in the cluster reaches K.

6. The carbon trading data prediction method based on big data according to claim 1, characterized in that: The federated averaging algorithm is expressed as: ， in, are the model parameters of the updated global prediction model, For the The model parameters of the local prediction model, is the number of local prediction models.

7. A carbon trading data prediction system based on big data, characterized by: Applying the carbon trading data prediction method based on big data according to any one of claims 1 to 6, comprising: The data acquisition module is used to obtain the geographical location characteristics and historical carbon trading matrix of each virtual power plant, as well as real-time climate characteristic data and carbon trading matrix; the historical carbon trading matrix includes the power supply transaction matrix and the power sales transaction matrix; Cluster analysis module, used to perform cluster analysis on virtual power plants based on geographical location characteristics and historical carbon trading matrix, and divide all virtual power plants into several federated learning sub-nodes; The model building module is used to obtain historical climate characteristic data and build a global prediction model and a local prediction model for each federated learning sub-node based on the historical climate characteristic data and the historical carbon trading matrix; The real-time prediction module is used to input the real-time climate characteristic data and carbon trading matrix into the local prediction model of the corresponding federated learning sub-node to obtain the carbon trading data prediction results.

Images