Carbon transaction data prediction method and system based on big data

Abstract

The invention discloses a carbon transaction data prediction method and system based on big data, and relates to the technical field of carbon transaction prediction. The method comprises the following steps: acquiring geographic position features and historical carbon transaction matrixes of each virtual power plant; performing clustering analysis on the virtual power plants according to the geographic position features and the historical carbon transaction matrix, and dividing all the virtual power plants to a plurality of federated learning sub-nodes; acquiring historical climate characteristic data, and constructing a global prediction model and a local prediction model of each federated learning sub-node according to the historical climate characteristic data and the historical carbon transaction matrix; and inputting the climate characteristic data and the carbon transaction matrix which are acquired in real time into the local prediction model of the corresponding federated learning sub-node to obtain a carbon transaction data prediction result. According to the invention, a data basis can be provided for operation and decision-making of the virtual power plant, and effective resource configuration and environment management are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of carbon trading prediction, and in particular to a carbon trading data prediction method and system based on big data. Background Art

[0002] The power industry is facing huge pressure of carbon emissions, and the prediction of carbon trading data is crucial. In recent years, academia and industry have proposed some new methods to improve the accuracy and efficiency of carbon trading prediction. For example, big data analysis technology and machine learning methods are applied to carbon trading prediction, and more complex prediction models are constructed by mining a large amount of historical trading data.

[0003] However, the existing methods still face some problems and challenges. First, when processing large amounts of widely distributed carbon trading data, the limitation of computing resources is an important reason why the existing methods are difficult to achieve accurate predictions; second, due to the large differences in carbon trading models between different companies, the existing methods are difficult to achieve personalized prediction and management of data from different carbon trading models, resulting in insufficient corporate operating efficiency and decision-making accuracy. Summary of the invention

[0004] In order to solve the problems existing in the above background technology, the present invention adopts the following technical solutions: A carbon trading data prediction method based on big data, comprising the steps of: Obtain the geographical location characteristics and historical carbon trading matrix of each virtual power plant; the historical carbon trading matrix includes the power supply trading matrix between the virtual power plant and each power supply end and the power sales trading matrix of each power consumption end; the power supply trading matrix stores power supply carbon trading data with energy type and sampling time as index; the power sales trading matrix stores power sales carbon trading data with customer type and sampling time as index; Cluster analysis of virtual power plants is performed based on geographical location characteristics and historical carbon trading matrices, and all virtual power plants are divided into several federated learning sub-nodes; each federated training sub-node includes one or more virtual power plants; Obtain historical climate characteristic data, and build a global prediction model and a local prediction model for each federated learning sub-node based on the historical climate characteristic data and the historical carbon trading matrix; The real-time climate characteristic data and carbon trading matrix are input into the local prediction model of the corresponding federated learning sub-node to obtain the carbon trading data prediction results.

[0005] As a preferred solution, the cluster analysis of virtual power plants based on geographical location characteristics and historical carbon trading matrix includes the following steps: S21, extracting the geographical location characteristics of each virtual power plant and the standard numerical characteristics of the historical carbon trading matrix; S22, initializing the K value of the K-means algorithm, and selecting K virtual power plants as the initial cluster centers; S23, assigning each virtual power plant to the nearest center point in the cluster according to the distance metric value to form K clusters; S24, updating the center point in each cluster according to the mean value of the standard numerical characteristics of the virtual power plant in each cluster; S25. Repeat steps S23 and S24 until the change range of the center points in all clusters is less than a preset threshold; each cluster is recorded as a federated learning subnode.

[0006] As a preferred solution, the extraction of geographical location characteristics of each virtual power plant and standard numerical characteristics of the historical carbon trading matrix specifically includes: Standardize the power supply carbon trading data and power sales carbon trading data of the geographical location characteristics and historical carbon trading matrix to obtain standard geographical location characteristics, standard power supply trading matrix and standard power sales trading matrix; The basic statistical features of each data column of the standard power supply transaction matrix and the standard power sales transaction matrix are calculated, and the standard geographical location features and basic statistical features are merged into standard numerical features.

[0007] As a preferred solution, the K value of the initialization K-means algorithm specifically includes: Set the K value range, calculate the total square error SSE corresponding to each K value within the K value range and draw the K-SSE relationship curve; the K value range is , where N is the number of virtual power plants, Express Round up; Select the K value corresponding to the second-order difference value in the K-SSE relationship curve that is greater than the preset second-order difference threshold as the K value of the K-means algorithm; The total error sum of squares is expressed as: , Where K represents the number of clusters, represents the i-th cluster, represents the virtual power plant in the ith cluster, represents the center point in the i-th cluster, express and The distance metric of .

[0008] As a preferred solution, the second-order difference value is expressed as: , , in, represents the total error sum of squares corresponding to the K value, Indicates the rate of decrease of the total square error corresponding to the K value.

[0009] As a preferred solution, the step of selecting K virtual power plants as the initial center points within the cluster is specifically selecting K virtual power plants as the initial center points within the cluster by using a maximum distance method, including the steps of: S2221, selecting the two virtual power plants with the largest distance measurement values ​​of standard numerical features among all virtual power plants as the center points within the cluster; S2222, allocating the remaining virtual power plants to corresponding clusters according to the distance measurement values ​​between them and the center points in the clusters, and updating the center points in the clusters; S2223, in each cluster, searching for a virtual power plant with the largest distance measurement value from the center point of the cluster as the added center point of the cluster; S2224. Repeat steps S2222 and S2223 until the number of center points in the cluster reaches K.

[0010] As a preferred solution, the global prediction model and the local prediction model of each federated learning sub-node are constructed according to the historical climate characteristic data and the historical carbon trading matrix, including cyclically executing the following steps: The global model parameters of the global prediction model are transmitted to the federated learning sub-node, and the local model parameters corresponding to the federated learning sub-node are set according to the global model parameters; Train the local prediction model based on the historical climate characteristic data and historical carbon trading matrix of the local federated learning subnode; Regularly transmit the updated local model parameters of each federated learning sub-node to the central server; The local model parameters of each federated learning sub-node are aggregated through the federated averaging algorithm to update the global prediction model.

[0011] As a preferred solution, the federated average algorithm is expressed as:

[0012] in, are the model parameters of the updated global prediction model, For the The model parameters of the local prediction model, is the number of local prediction models.

[0013] As a preferred solution, the global prediction model and the local prediction model both include a personalized hidden layer and a global hidden layer; The transmitting of the updated local model parameters of each federated learning sub-node to the central server specifically includes transmitting the updated local model parameters corresponding to the global hidden layer of each federated learning sub-node to the central server; The method of aggregating the parameters of the local model parameters of each federated learning sub-node by using the federated averaging algorithm specifically includes: aggregating the parameters of the local model parameters corresponding to the global hidden layer of each federated learning sub-node by using the federated averaging algorithm.

[0014] The present invention also provides a carbon trading data prediction system based on big data, which uses the carbon trading data prediction method based on big data as described above, including: The data acquisition module is used to obtain the geographical location characteristics and historical carbon trading matrix of each virtual power plant, as well as the climate characteristic data and carbon trading matrix obtained in real time; the historical carbon trading matrix includes the power supply trading matrix and the power sales trading matrix; The cluster analysis module is used to perform cluster analysis on virtual power plants based on geographical location characteristics and historical carbon trading matrices, and divide all virtual power plants into several federated learning sub-nodes; The model building module is used to obtain historical climate characteristic data and build a global prediction model and a local prediction model for each federated learning sub-node based on the historical climate characteristic data and the historical carbon trading matrix; The real-time prediction module is used to input the real-time climate characteristic data and carbon trading matrix into the local prediction model of the corresponding federated learning sub-node to obtain the carbon trading data prediction results.

[0015] Compared with the prior art, the present invention has the following beneficial effects: This application distributes the prediction model training process to multiple federated learning sub-nodes through a federated learning architecture to fully utilize computing resources and reduce the computing pressure on a single server; through cluster analysis, virtual power plants with similar geographical location characteristics and historical carbon trading characteristics are aggregated to form federated learning sub-nodes. The virtual power plants within each federated learning sub-node have similar climate characteristics and carbon trading patterns. By sharing training data sets within the federated learning sub-nodes, the overfitting problem that is prone to occur in the local prediction model in the early stage of training due to the small amount of data can be avoided, and the efficiency of personalized learning can be improved, avoiding the waste of resources caused by personalized training; by inputting the real-time climate characteristic data and carbon trading matrix into the local prediction model to obtain personalized carbon trading data prediction results, a data basis can be provided for the operation and decision-making of the virtual power plant, thereby realizing effective resource allocation and environmental management.

[0016] This application extracts the geographical location characteristics of each virtual power plant and the standard numerical characteristics of the historical carbon trading matrix, and performs cluster analysis on virtual power plants through the K-means algorithm. It can effectively perform cluster analysis on virtual power plants with wide distribution and complex characteristics, and can effectively capture the commonalities of virtual power plants with similar geographical location characteristics and carbon trading patterns. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 A schematic diagram of the process of the carbon trading data prediction method based on big data provided in this embodiment; Figure 2 A schematic diagram of a process for extracting the geographical location characteristics of each virtual power plant and the standard numerical characteristics of the historical carbon trading matrix provided in this embodiment; Figure 3 A schematic diagram of the process of building a global prediction model and a local prediction model of each federated learning sub-node based on historical climate characteristic data and a historical carbon trading matrix provided in this embodiment; Figure 4 A schematic diagram of the structure of a carbon trading data prediction system based on big data provided in this embodiment; Figure 5 A schematic diagram of the structure of an electronic device provided in this embodiment. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0022] In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0023] The power industry is facing huge pressure of carbon emissions, and the prediction of carbon trading data is crucial. In recent years, academia and industry have proposed some new methods to improve the accuracy and efficiency of carbon trading prediction. For example, big data analysis technology and machine learning methods are applied to carbon trading prediction, and more complex prediction models are constructed by mining a large amount of historical trading data.

[0024] However, the existing methods still face some problems and challenges. First, when processing large amounts of widely distributed carbon trading data, the limitation of computing resources is an important reason why the existing methods are difficult to achieve accurate predictions; second, due to the large differences in carbon trading models between different companies, the existing methods are difficult to achieve personalized prediction and management of data from different carbon trading models, resulting in insufficient corporate operating efficiency and decision-making accuracy.

[0025] The present invention provides a carbon trading data prediction method and system based on big data. The specific implementation scheme of the present invention will be described in detail below.

[0026] Example 1

[0027] like Figure 1 As shown, a carbon trading data prediction method based on big data includes the following steps: S1. Obtain the geographical location characteristics and historical carbon trading matrix of each virtual power plant; the historical carbon trading matrix includes the power supply trading matrix between the virtual power plant and each power supply end and the power sales trading matrix of each power consumption end; the power supply trading matrix stores power supply carbon trading data with energy type and sampling time as index; the power sales trading matrix stores power sales carbon trading data with customer type and sampling time as index; S2. Perform cluster analysis on virtual power plants based on geographical location characteristics and historical carbon trading matrix, and divide all virtual power plants into several federated learning sub-nodes; each federated training sub-node includes one or more virtual power plants; S3. Obtain historical climate characteristic data, and build a global prediction model and a local prediction model for each federated learning sub-node based on the historical climate characteristic data and the historical carbon trading matrix; S4. Input the climate characteristic data and carbon trading matrix obtained in real time into the local prediction model of the corresponding federated learning sub-node to obtain the carbon trading data prediction result.

[0028] This application distributes the prediction model training process to multiple federated learning sub-nodes through a federated learning architecture to fully utilize computing resources and reduce the computing pressure on a single server; through cluster analysis, virtual power plants with similar geographical location characteristics and historical carbon trading characteristics are aggregated to form federated learning sub-nodes. The virtual power plants within each federated learning sub-node have similar climate characteristics and carbon trading patterns. By sharing training data sets within the federated learning sub-nodes, the overfitting problem that is prone to occur in the local prediction model in the early stage of training due to the small amount of data can be avoided, and the efficiency of personalized learning can be improved, avoiding the waste of resources caused by personalized training; by inputting the real-time climate characteristic data and carbon trading matrix into the local prediction model to obtain personalized carbon trading data prediction results, a data basis can be provided for the operation and decision-making of the virtual power plant, thereby realizing effective resource allocation and environmental management.

[0029] Specifically, each step of the carbon trading data prediction method based on big data of the present invention is described in detail through the following content: A carbon trading data prediction method based on big data, comprising the steps of: S1. Obtain the geographical location characteristics and historical carbon trading matrix of each virtual power plant; the historical carbon trading matrix includes the power supply trading matrix between the virtual power plant and each power supply end and the power sales trading matrix of each power consumption end; the power supply trading matrix stores power supply carbon trading data with energy type and sampling time as index; the power sales trading matrix stores power sales carbon trading data with customer type and sampling time as index; In this embodiment, the power supply transaction matrix is ​​used to record and analyze the energy transaction between the virtual power plant and the power supply end. The power supply transaction matrix uses energy type and sampling time as indexes, for example, energy type as rows and sampling time as columns to store power supply carbon trading data.

[0030] The power sales transaction matrix is ​​used to record and analyze the energy transactions between the virtual power plant and the electricity user. The power supply transaction matrix uses energy type and sampling time as indexes, for example, customer type as rows and sampling time as columns to store power sales carbon trading data.

[0031] In one embodiment, the geographical location characteristics include longitude, latitude, climate type and economic development indicators.

[0032] In one embodiment, the power supply carbon trading data and the power sales carbon trading data are both arrays consisting of electricity trading volume (MWh), carbon emissions (tons) and average transaction price (yuan / ton).

[0033] S2. Perform cluster analysis on virtual power plants based on geographical location characteristics and historical carbon trading matrix, and divide all virtual power plants into several federated learning sub-nodes; each federated training sub-node includes one or more virtual power plants; Furthermore, the cluster analysis of virtual power plants based on geographical location characteristics and historical carbon trading matrix includes the following steps: S21, extracting the geographical location characteristics of each virtual power plant and the standard numerical characteristics of the historical carbon trading matrix; Among them, the values ​​of the extracted standardized numerical features are distributed in the same range to eliminate the influence of the dimension; S22, initializing the K value of the K-means algorithm, and selecting K virtual power plants as the initial cluster centers; S23, assigning each virtual power plant to the nearest center point in the cluster according to the distance metric value to form K clusters; Among them, the distance metric value represents the Euclidean distance between the virtual power plant in the selected cluster and the standard numerical feature of the center point in the cluster; S24, updating the center point in each cluster according to the mean value of the standard numerical characteristics of the virtual power plant in each cluster; S25. Repeat steps S23 and S24 until the change range of the center points in all clusters is less than a preset threshold; each cluster is recorded as a federated learning subnode.

[0034] This embodiment extracts the geographical location characteristics of each virtual power plant and the standard numerical characteristics of the historical carbon trading matrix, and performs cluster analysis on virtual power plants through the K-means algorithm. It can effectively perform cluster analysis on virtual power plants with wide distribution and complex characteristics, and can effectively capture the commonalities of virtual power plants with similar geographical location characteristics and carbon trading patterns. For example, virtual power plants located in the same climate zone face similar climate changes, which can enable the model to better understand and utilize these regional characteristics and improve prediction accuracy. The federated learning sub-nodes formed by the clustering results provide a structured basis for subsequent federated learning to improve the training effect of the local prediction model.

[0035] In one embodiment, please refer to Figure 2 , step S21, extracting the geographical location characteristics of each virtual power plant and the standard numerical characteristics of the historical carbon trading matrix, specifically includes: The power supply carbon trading data and power sales carbon trading data (including power trading volume, carbon emissions and average trading price) of the geographical location characteristics and historical carbon trading matrix are standardized to obtain standard geographical location characteristics, standard power supply trading matrix and standard power sales trading matrix. The standardization can be done by Z-score standardization or Min-Max normalization.

[0036] Calculate the basic statistical features of each data column of the standard power supply transaction matrix and the standard power sales transaction matrix, and merge the standard geographical location features and basic statistical features into standard numerical features. Among them, the basic statistical features of each data column of the standard power supply transaction matrix represent the mean, standard deviation, maximum and minimum values ​​corresponding to each energy type within a certain statistical time; the basic statistical features of each data column of the standard power sales transaction matrix represent the mean, standard deviation, maximum and minimum values ​​corresponding to each customer type within a certain statistical time. In the standard numerical features, each basic statistic and standard geographical location feature are used as a dimension of the standard numerical features.

[0037] In one embodiment, the K value of the initialization K-means algorithm specifically includes: S2211, set the K value range, calculate the total square error SSE corresponding to each K value within the K value range and draw a K-SSE relationship curve; the K value range is , where N is the number of virtual power plants, Express Round up. The total error square sum is expressed as: , Where K represents the number of clusters, represents the i-th cluster, represents the virtual power plant in the ith cluster, represents the center point in the i-th cluster, express and The distance metric of .

[0038] Furthermore, the distance metric value is the Euclidean distance between the standard numerical features of the virtual power plant in the cluster and the center point in the cluster. The distance metric value is expressed as: , in, represents the dimension of the standard numerical features, represents the jth eigenvalue of the standard numerical characteristics of the virtual power plant, Represents the jth eigenvalue of the standardized numerical feature of the center point in the cluster.

[0039] S2212. Select the K value corresponding to when the second-order difference value in the K-SSE relationship curve is greater than a preset second-order difference threshold as the K value of the K-means algorithm.

[0040] In the K-SSE relationship curve, when a smaller K value is selected, increasing K can significantly improve the similarity within the cluster (reduce SSE), and the curve drops rapidly at this time. After exceeding the optimal K value, increasing the K value has little effect on improving SSE, and the curve tends to be flat. Therefore, in order to make the initialized K value both obvious in the difference between clusters and not cause overfitting problems due to too many clusters, this embodiment needs to select a K value with a significantly reduced slope in the K-SSE relationship curve as the K value of the K-means algorithm, which is specifically calculated through the relationship between the second-order difference value and the preset second-order difference threshold. Among them, the second-order difference value is expressed as: , , in, represents the total error sum of squares corresponding to the K value, Indicates the rate of decrease of the total square error corresponding to the K value.

[0041] K virtual power plants are selected as the initial cluster center points, and the subsequent virtual power plant division is performed accordingly. The selection method of the initial cluster center point is random in some embodiments, and the maximum distance method is used in some embodiments.

[0042] Based on the foregoing, in one embodiment, K virtual power plants are selected as the initial center points in the cluster, specifically, K virtual power plants are selected as the initial center points in the cluster by using the maximum distance method, including the steps of: S2221, selecting the two virtual power plants with the largest distance measurement values ​​of standard numerical features among all virtual power plants as the center points within the cluster; S2222, allocating the remaining virtual power plants to corresponding clusters according to the distance measurement values ​​between them and the center points in the clusters, and updating the center points in the clusters; S2223, in each cluster, searching for a virtual power plant with the largest distance measurement value from the center point of the cluster as the added center point of the cluster; S2224. Repeat steps S2222 and S2223 until the number of center points in the cluster reaches K.

[0043] In this embodiment, the maximum distance method ensures that the initially selected center points are widely distributed in the feature space and can more fully represent the different feature areas in the data set; since the initial center points are better distributed, the selection based on the maximum distance method can reduce the number of unnecessary iterations, thereby accelerating the convergence speed of the clustering algorithm, reducing the consumption of computing resources, and improving computing efficiency.

[0044] S3. Obtain historical climate characteristic data, and build a global prediction model and a local prediction model for each federated learning sub-node based on the historical climate characteristic data and the historical carbon trading matrix; For further information, see Figure 3 The global prediction model and the local prediction model of each federated learning sub-node are constructed according to the historical climate characteristic data and the historical carbon trading matrix, including looping the following steps: S31, transmitting the global model parameters of the global prediction model to the federated learning sub-node, and setting the local model parameters corresponding to the federated learning sub-node according to the global model parameters; At the beginning of the first cycle, the central server initializes and builds a global prediction model, and then transmits the global model parameters of the global prediction model to the federated learning subnode. The global prediction model can adopt models such as multinomial regression model, recurrent neural network (RNN), long short-term memory network (LSTM), deep neural network (DNN), etc. In a preferred embodiment, the global prediction model adopts a long short-term memory network, which is a neural network suitable for processing time series data. Due to its special network structure, it can capture the long-term dependencies between climate characteristic data and carbon trading matrix.

[0045] In the subsequent cycles, the central server regularly publishes the latest global model parameters to each federated learning sub-node. After receiving the global model parameters, each federated learning sub-node initializes or updates its local model parameters to ensure consistency with the global model.

[0046] S32, training a local prediction model based on the local historical climate characteristic data and historical carbon trading matrix of the federated learning subnode; Each federated learning sub-node uses its own local data set for training, including historical climate characteristic data and historical carbon trading matrix. During the training process, appropriate training algorithms (such as gradient descent, Adam optimizer, etc.) are used to train the prediction model through multiple rounds of iterative training, and the model hyperparameters (such as learning rate, batch size, etc.) are adjusted according to actual needs.

[0047] S33, regularly transmitting the updated local model parameters of each federated learning sub-node to the central server; Within the preset time interval, each federated learning sub-node completes local training and obtains an updated local prediction model, and transmits the local model parameters (such as weights and bias values) corresponding to the local prediction model to the central server. The transmission process can use an encrypted transmission protocol to ensure the security and privacy protection of data transmission.

[0048] S34, performing parameter aggregation on the local model parameters of each federated learning sub-node through a federated averaging algorithm, and updating the global prediction model; The federated average algorithm is expressed as:

[0049] in, are the model parameters of the updated global prediction model, For the The model parameters of the local prediction model, is the number of local prediction models.

[0050] In this step, after the central server receives the local model parameters of all child nodes, it uses the federated average algorithm to aggregate the parameters. The federated average algorithm calculates the weighted average of each child node model to update the global model parameters. After the aggregation is completed, the central server generates a new global prediction model and broadcasts it back to each node for the next round of training.

[0051] This embodiment uses federated averaging and parameter aggregation to continuously optimize the global model based on the characteristics of each sub-node, thereby improving the prediction performance of the global prediction model. Without exchanging actual data, data privacy protection and privacy requirements between federated learning sub-nodes are achieved through parameter transmission and aggregation. The distributed computing method balances the computing and data storage pressures and improves the system scalability and response speed.

[0052] As a preferred embodiment, the global prediction model and the local prediction model both include a personalized hidden layer and a global hidden layer; The transmitting of the updated local model parameters of each federated learning sub-node to the central server specifically includes transmitting the updated local model parameters corresponding to the global hidden layer of each federated learning sub-node to the central server; The method of aggregating the parameters of the local model parameters of each federated learning sub-node by using the federated averaging algorithm specifically includes: aggregating the parameters of the local model parameters corresponding to the global hidden layer of each federated learning sub-node by using the federated averaging algorithm.

[0053] For example, based on the standard LSTM architecture, personalized layers are added, which are only trained on specific users or data sources. Therefore, the parameters of the personalized hidden layers are only updated locally to improve the model's ability to fit local data. The personalized hidden layers are designed to capture the unique data characteristics of each federated learning sub-node, and the personalized layers are optimized through back propagation using the node's training data to speed up the adaptation of these layers to local data characteristics.

[0054] Global hidden layers contain parameters shared across all nodes and are responsible for extracting common patterns in the data. During training, if a portion of the global hidden layers are important (such as low-level features) and stable, you can freeze those layers and only update those layers that need to adapt to changing data patterns. If the global layers need to be more responsive to local data, all layers can be updated simultaneously. This is often used when the data distribution changes significantly.

[0055] S4. Input the climate characteristic data and carbon trading matrix obtained in real time into the local prediction model of the corresponding federated learning sub-node to obtain the carbon trading data prediction result.

[0056] Among them, the carbon trading data prediction results include predicted power supply carbon trading data and predicted power sales carbon trading data. Both the predicted power supply carbon trading data and the predicted power sales carbon trading data include the corresponding predicted electricity trading volume, predicted carbon emissions and predicted average transaction price, to help decision makers evaluate the economic and environmental impacts of carbon trading from multiple perspectives and optimize operational strategies.

[0057] Example 2

[0058] like Figure 4 As shown, a carbon trading data prediction system based on big data includes: The data acquisition module is used to obtain the geographical location characteristics and historical carbon trading matrix of each virtual power plant, as well as the climate characteristic data and carbon trading matrix obtained in real time; the historical carbon trading matrix includes the power supply trading matrix and the power sales trading matrix; The cluster analysis module is used to perform cluster analysis on virtual power plants based on geographical location characteristics and historical carbon trading matrices, and divide all virtual power plants into several federated learning sub-nodes; The model building module is used to obtain historical climate characteristic data and build a global prediction model and a local prediction model for each federated learning sub-node based on the historical climate characteristic data and the historical carbon trading matrix; The real-time prediction module is used to input the real-time climate characteristic data and carbon trading matrix into the local prediction model of the corresponding federated learning sub-node to obtain the carbon trading data prediction results.

[0059] It should be understood that the disclosed system can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, each functional module can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules.

[0060] Example 3

[0061] An electronic device 2, such as Figure 5As shown, a processor 21 and a memory 22, the memory 22 is used to store computer program codes, the computer program codes include computer instructions, when the processor 21 executes the computer instructions, the electronic device executes the above-mentioned carbon trading data prediction method based on big data.

[0062] The electronic device 2 includes a processor 21, a memory 22, an output device 23, and an input device 24. The processor 21, the memory 22, the input device 24, and the output device 23 are coupled via a connector, and the connector includes various interfaces, transmission lines, or buses, etc., which are not limited in the embodiments of the present invention. It should be understood that in various embodiments of the present invention, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, for example, through various interfaces, transmission lines, buses, etc.

[0063] The processor 21 may be one or more graphics processing units (GPUs). When the processor 21 is a GPU, the GPU may be a single-core GPU or a multi-core GPU. Optionally, the processor 21 may be a processor group consisting of multiple GPUs, and the multiple processors are coupled to each other via one or more buses. Optionally, the processor may also be other types of processors, etc., which are not limited in the embodiments of the present invention.

[0064] The memory 22 can be used to store computer program instructions and various computer program codes including program codes for executing the scheme of the present invention. Optionally, the memory includes but is not limited to random access memory (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM), or portable read only memory (CD-ROM), which is used for related instructions and data.

[0065] The input device 24 is used to input data and / or signals, and the output device 23 is used to output data and / or signals. The output device 23 and the input device 24 can be independent devices or an integrated device.

[0066] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A carbon trading data prediction method based on big data, characterized by: Includes steps: Obtain the geographical location characteristics and historical carbon trading matrix of each virtual power plant; the historical carbon trading matrix includes the power supply trading matrix between the virtual power plant and each power supply end and the power sales trading matrix of each power consumption end; the power supply trading matrix stores power supply carbon trading data with energy type and sampling time as index; the power sales trading matrix stores power sales carbon trading data with customer type and sampling time as index; Cluster analysis of virtual power plants is performed based on geographical location characteristics and historical carbon trading matrices, and all virtual power plants are divided into several federated learning sub-nodes; each federated training sub-node includes one or more virtual power plants; Obtain historical climate characteristic data, and build a global prediction model and a local prediction model for each federated learning sub-node based on the historical climate characteristic data and the historical carbon trading matrix; The real-time climate characteristic data and carbon trading matrix are input into the local prediction model of the corresponding federated learning sub-node to obtain the carbon trading data prediction results.

2. The carbon trading data prediction method based on big data according to claim 1 is characterized by: The cluster analysis of virtual power plants based on geographical location characteristics and historical carbon trading matrix includes the following steps: S21, extracting the geographical location characteristics of each virtual power plant and the standard numerical characteristics of the historical carbon trading matrix; S22, initializing the K value of the K-means algorithm, and selecting K virtual power plants as the initial cluster centers; S23, assigning each virtual power plant to the nearest center point in the cluster according to the distance metric value to form K clusters; S24, updating the center point in each cluster according to the mean value of the standard numerical characteristics of the virtual power plant in each cluster; S25. Repeat steps S23 and S24 until the change range of the center points in all clusters is less than a preset threshold; each cluster is recorded as a federated learning subnode.

3. The carbon trading data prediction method based on big data according to claim 2 is characterized by: The extraction of the geographical location characteristics of each virtual power plant and the standard numerical characteristics of the historical carbon trading matrix specifically includes: Standardize the power supply carbon trading data and power sales carbon trading data of the geographical location characteristics and historical carbon trading matrix to obtain standard geographical location characteristics, standard power supply trading matrix and standard power sales trading matrix; The basic statistical features of each data column of the standard power supply transaction matrix and the standard power sales transaction matrix are calculated, and the standard geographical location features and basic statistical features are merged into standard numerical features.

4. The carbon trading data prediction method based on big data according to claim 2 is characterized by: The K value of the initialization K-means algorithm specifically includes: Set the K value range, calculate the total square error SSE corresponding to each K value within the K value range and draw the K-SSE relationship curve; the K value range is , where N is the number of virtual power plants, Express Round up; Select the K value corresponding to the second-order difference value in the K-SSE relationship curve that is greater than the preset second-order difference threshold as the K value of the K-means algorithm; The total error sum of squares is expressed as: ， Where K represents the number of clusters, represents the i-th cluster, represents the virtual power plant in the ith cluster, represents the center point in the i-th cluster, express and The distance metric of .

5. The carbon trading data prediction method based on big data according to claim 4 is characterized by: The second-order difference value is expressed as: ， ， in, represents the total error sum of squares corresponding to the K value, Indicates the rate of decrease of the total square error corresponding to the K value.

6. The carbon trading data prediction method based on big data according to claim 2 is characterized by: The method of selecting K virtual power plants as the initial center points in the cluster is specifically to select K virtual power plants as the initial center points in the cluster by using the maximum distance method, and includes the following steps: S2221, selecting the two virtual power plants with the largest distance measurement values ​​of standard numerical features among all virtual power plants as the center points within the cluster; S2222, allocating the remaining virtual power plants to corresponding clusters according to the distance measurement values ​​between them and the center points in the clusters, and updating the center points in the clusters; S2223, in each cluster, searching for a virtual power plant with the largest distance measurement value from the center point of the cluster as the added center point of the cluster; S2224. Repeat steps S2222 and S2223 until the number of center points in the cluster reaches K.

7. The carbon trading data prediction method based on big data according to claim 1 is characterized by: The construction of a global prediction model and a local prediction model of each federated learning sub-node according to the historical climate characteristic data and the historical carbon trading matrix includes cyclically executing the following steps: The global model parameters of the global prediction model are transmitted to the federated learning sub-node, and the local model parameters corresponding to the federated learning sub-node are set according to the global model parameters; Train the local prediction model based on the historical climate characteristic data and historical carbon trading matrix of the local federated learning subnode; Regularly transmit the updated local model parameters of each federated learning sub-node to the central server; The local model parameters of each federated learning sub-node are aggregated through the federated averaging algorithm to update the global prediction model.

8. The carbon trading data prediction method based on big data according to claim 7 is characterized by: The federated averaging algorithm is expressed as: ， in, are the model parameters of the updated global prediction model, For the The model parameters of the local prediction model, is the number of local prediction models.

9. The carbon trading data prediction method based on big data according to claim 7 is characterized by: The global prediction model and the local prediction model both include a personalized hidden layer and a global hidden layer; The transmitting of the updated local model parameters of each federated learning sub-node to the central server specifically includes transmitting the updated local model parameters corresponding to the global hidden layer of each federated learning sub-node to the central server; The parameter aggregation of the local model parameters of each federated learning sub-node by using the federated averaging algorithm is specifically: the parameter aggregation of the local model parameters corresponding to the global hidden layer of each federated learning sub-node is performed by using the federated averaging algorithm.

10. A carbon trading data prediction system based on big data, characterized by: The method for predicting carbon trading data based on big data as claimed in any one of claims 1 to 9 is applied, comprising: The data acquisition module is used to obtain the geographical location characteristics and historical carbon trading matrix of each virtual power plant, as well as the climate characteristic data and carbon trading matrix obtained in real time; the historical carbon trading matrix includes the power supply trading matrix and the power sales trading matrix; The cluster analysis module is used to perform cluster analysis on virtual power plants based on geographical location characteristics and historical carbon trading matrices, and divide all virtual power plants into several federated learning sub-nodes; The model building module is used to obtain historical climate characteristic data and build a global prediction model and a local prediction model for each federated learning sub-node based on the historical climate characteristic data and the historical carbon trading matrix; The real-time prediction module is used to input the real-time climate characteristic data and carbon trading matrix into the local prediction model of the corresponding federated learning sub-node to obtain the carbon trading data prediction results.

Images