Method and device for constructing carbon integral pricing model

Abstract

The invention provides a carbon integral pricing model construction method and device, and belongs to the technical field of reinforcement learning, and the method comprises the steps: constructing a reinforcement learning model through employing a DDPG algorithm; setting an environment required by learning training of the reinforcement learning model; generating a plurality of experience tuples according to continuous interaction between the reinforcement learning model and the environment, and storing the plurality of experience tuples in an experience pool; training the reinforcement learning model based on the experience tuple in the experience pool until the reinforcement learning model converges; and extracting an Actor network from the convergent reinforcement learning model as a carbon point pricing model so as to determine the carbon point pricing for each type of carbon point seller according to the carbon point number of each type of carbon point seller. According to the construction method and device of the carbon point pricing model, a more scientific and reasonable carbon point pricing scheme can be provided, and healthy development of the whole carbon market can be promoted.

Description

Technical Field

[0001] The present invention relates to the field of reinforcement learning technology, and in particular to a method and device for constructing a carbon credit pricing model. Background Art

[0002] Controlling carbon emissions is one of the keys to achieving sustainable development. As an innovative low-carbon incentive model, the carbon credit mechanism has gradually gained widespread attention and application. The carbon credit mechanism encourages low-carbon behaviors of individuals, families and small and micro enterprises, converts them into carbon credits, and connects them with the carbon trading market, aiming to enhance the low-carbon awareness and participation of the whole society. For example, the Ant Forest project obtains carbon credits through users' low-carbon behaviors, and links them with carbon emission rights in the carbon trading market, promoting the public's active participation in low-carbon life.

[0003] In the process of combining carbon credits with carbon trading, carbon emission trading agencies have played an important bridging role. These companies provide carbon emission rights trading agency services to consumers such as enterprises, the public, and small and micro entities (i.e., carbon credit sellers), helping them enter the carbon trading market. Carbon emission trading agencies (i.e., carbon credit pricing parties) purchase carbon credits from consumers according to the set carbon credit pricing, convert them into carbon emission rights, and then sell them on the market to obtain net benefits from carbon emission reduction. If the carbon credit pricing is too high, it will affect the net benefits of carbon emission reduction obtained by the carbon emission trading agency in the future; if the carbon credit pricing is too low, it will affect the enthusiasm of subsequent consumers to obtain carbon credits through low-carbon behavior, and reduce the net benefits of carbon emission reduction expected by the carbon emission trading agency; it can be seen that the net benefits of carbon emission reduction obtained by the carbon emission trading agency are closely related to the carbon credit pricing.

[0004] The current carbon credit pricing strategies are mostly based on human decision-making, which are highly subjective, inefficient, and difficult to predict the net benefits of carbon emission reductions. This is not conducive to accurately reflecting the impact of carbon credit pricing strategies on environmental benefits and market value. Summary of the invention

[0005] The present invention provides a method and device for constructing a carbon credit pricing model, which is used to solve the defects of artificial decision-making pricing in the prior art, which has strong subjectivity, low efficiency and difficult to predict the net benefits of carbon emission reduction.

[0006] In a first aspect, the present invention provides a method for constructing a carbon credit pricing model, comprising: A reinforcement learning model is constructed using the DDPG algorithm; the network architecture of the reinforcement learning model is an Actor-Critic network architecture, including an Actor network and a Critic network; the Actor network is used to generate corresponding action variables under given state variables, and the Critic network is used to estimate and evaluate given state variables and corresponding action variables; wherein the state variable is the number of carbon credits of each type of carbon credit seller, the action variable is the carbon credit pricing for each type of carbon credit seller, and the reward is the net income of the carbon credit pricing party; the net income is the income from the sale of carbon emission rights minus the cost of purchasing all carbon credits from the carbon credit seller, and the carbon emission rights are obtained by the carbon credit pricing party by exchanging all purchased carbon credits; Setting the environment required for the reinforcement learning model to perform learning and training; the environment is a carbon credit quantity prediction model for different types of carbon credit sellers; the carbon credit quantity prediction model is used to output corresponding state variables based on input action variables; According to the continuous interaction between the reinforcement learning model and the environment, multiple experience tuples are generated and stored in the experience pool; each experience tuple includes state variables, action variables, rewards, and state variables after state transfer; The reinforcement learning model is trained based on the experience tuples in the experience pool until the reinforcement learning model converges; An Actor network is extracted from the converged reinforcement learning model as a carbon credit pricing model to determine the carbon credit pricing for each type of carbon credit seller based on the number of carbon credits of each type of carbon credit seller.

[0007] In a second aspect, the present invention further provides a device for constructing a carbon credit pricing model, comprising: The first processing module is used to construct a reinforcement learning model using the DDPG algorithm; the network architecture of the reinforcement learning model is an Actor-Critic network architecture, including an Actor network and a Critic network; the Actor network is used to generate corresponding action variables under given state variables, and the Critic network is used to estimate and evaluate given state variables and corresponding action variables; wherein the state variable is the number of carbon credits of each type of carbon credit seller, the action variable is the carbon credit pricing for each type of carbon credit seller, and the reward is the net income of the carbon credit pricing party; the net income is the income from the sale of carbon emission rights minus the cost of purchasing all carbon credits from the carbon credit seller, and the carbon emission rights are obtained by the carbon credit pricing party by exchanging all the purchased carbon credits; The second processing module is used to set the environment required for the reinforcement learning model to perform learning and training; the environment is a carbon credit quantity prediction model for different types of carbon credit sellers; the carbon credit quantity prediction model is used to output a corresponding state variable according to an input action variable; The third processing module is used to generate multiple experience tuples according to the continuous interaction between the reinforcement learning model and the environment, and store the multiple experience tuples in the experience pool; each experience tuple includes a state variable, an action variable, a reward, and a state variable after state transfer; A fourth processing module is used to train the reinforcement learning model based on the experience tuples in the experience pool until the reinforcement learning model converges; The fifth processing module is used to extract the Actor network from the converged reinforcement learning model as a carbon credit pricing model to determine the carbon credit pricing for each type of carbon credit seller based on the number of carbon credits of each type of carbon credit seller.

[0008] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for constructing a carbon credit pricing model as described in any one of the above are implemented.

[0009] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for constructing a carbon credit pricing model as described in any one of the above.

[0010] In a fifth aspect, the present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method for constructing a carbon credit pricing model as described in any one of the above items.

[0011] The method and device for constructing a carbon credit pricing model provided by the present invention establish a carbon credit pricing model through a reinforcement learning algorithm to solve the defects of human decision-making pricing in the prior art, which is highly subjective, inefficient, and difficult to predict the net benefits of carbon emission reduction. It can not only provide a more scientific and reasonable carbon credit pricing plan, but also promote the healthy development of the entire carbon market. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0013] Figure 1 It is a flow chart of a method for constructing a carbon credit pricing model provided by the present invention; Figure 2 It is a schematic diagram of the structure of the deep reinforcement learning strategy framework provided by the present invention; Figure 3It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0015] It should be noted that, in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0016] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0017] Combine the following Figure 1-Figure 3 The present invention describes a method and device for constructing a carbon credit pricing model provided by an embodiment of the present invention.

[0018] Figure 1 is a flow chart of the method for constructing the carbon credit pricing model provided by the present invention, such as Figure 1 As shown, including but not limited to the following steps: Step 101: Use the DDPG algorithm to build a reinforcement learning model; The network architecture of the reinforcement learning model is an Actor-Critic network architecture, including an Actor network and a Critic network: The Actor network is used to generate corresponding action variables under given state variables, and the Critic network is used to estimate and evaluate given state variables and corresponding action variables, that is, to output a Q value using a Q value function, which represents the expected return after taking the action. Among them, the state variable is the number of carbon credits of each type of carbon credit seller, the action variable is the carbon credit pricing for each type of carbon credit seller, and the reward is the net income of the carbon credit pricing party; the net income is the income from the sale of carbon emission rights minus the cost of purchasing all carbon credits from the carbon credit seller. The carbon emission rights are obtained by the carbon credit pricing party using all the carbon credits purchased.

[0019] Step 102: Set up the environment required for the reinforcement learning model to perform learning and training.

[0020] The environment is a carbon credit quantity prediction model for different types of carbon credit sellers; the carbon credit quantity prediction model is used to output corresponding state variables based on input action variables.

[0021] It is understandable that the carbon credit quantity prediction model is a regression model used to simulate the responses of different types of carbon credit sellers under a given action variable (carbon credit pricing). This model can be trained based on historical data.

[0022] Step 103: Generate multiple experience tuples according to the continuous interaction between the reinforcement learning model and the environment, and store the multiple experience tuples in the experience pool.

[0023] Each experience tuple includes state variables, action variables, rewards, and state variables after state transfer.

[0024] Step 104: Train the reinforcement learning model based on the experience tuples in the experience pool until the reinforcement learning model converges.

[0025] Among them, the convergence of the reinforcement learning model can be judged based on whether the Q value output by the Critic network converges; it can also be judged based on whether the net profit of the carbon credit pricing party converges.

[0026] Step 105: Extract the Actor network from the converged reinforcement learning model as a carbon credit pricing model to determine the carbon credit pricing for each type of carbon credit seller based on the number of carbon credits of each type of carbon credit seller.

[0027] The method for constructing a carbon credit pricing model provided by the present invention establishes a carbon credit pricing model through a reinforcement learning algorithm to solve the defects of human decision-making pricing in the prior art, which is highly subjective, inefficient, and difficult to predict the net benefits of carbon emission reduction. It can not only provide a more scientific and reasonable carbon credit pricing plan, but also promote the healthy development of the entire carbon market.

[0028] Based on the content of the above embodiment, as an optional embodiment, the method for constructing a carbon credit pricing model provided by the present invention, and the method for determining a carbon credit quantity prediction model for different types of carbon credit sellers include but are not limited to the following steps: (1) Extract features from the historical data on low-carbon behaviors and historical data on carbon credit quantity of the carbon credit sellers to construct feature data of the carbon credit sellers.

[0029] The present invention first needs to collect the low-carbon behavior historical data and carbon credit historical quantity data of the carbon credit seller.

[0030] Among them, the historical data on low-carbon behavior and the historical quantity data on carbon credits can be determined based on the low-carbon behavior of the carbon credit seller (consumer).

[0031] For example, if a consumer has walked 20 kilometers, taken public transportation 15 times, recycled 5 kg of recyclable items, and purchased 5 environmentally certified products in the past period of time, the historical data of low-carbon behavior can be expressed as (20, 15, 5, 5). At the same time, a consumer can obtain a certain number of carbon credits through each low-carbon behavior. By summing up the carbon credits in the past period of time, the corresponding historical data of carbon credits can be obtained.

[0032] Optionally, the present invention may perform data cleaning and standardization on the low-carbon behavior historical data and the carbon credit historical quantity data before performing feature extraction.

[0033] Optionally, when the dimensions of the low-carbon behavior historical data and the carbon credit historical quantity data are relatively large, the feature extraction method may be principal component analysis, which can achieve data dimensionality reduction while achieving feature extraction.

[0034] (2) Using a clustering algorithm to perform cluster analysis on the characteristic data of the carbon credit sellers to determine different types of carbon credit sellers.

[0035] The present invention can use K-means, hierarchical clustering, DBSCAN or other suitable clustering algorithms. The choice of which algorithm depends on the characteristics of the data and business requirements.

[0036] Preferably, the present invention selects the mean shift clustering algorithm to achieve clustering of different carbon credit sellers. After selecting the center point and the window radius, clustering is performed according to the consumer low-carbon behavior data in the sliding window, and the new center point is updated to a higher density window. By iteratively activating the center point within the group, consumers within the window radius tend to show similar characteristics. When the final center point is determined, the corresponding consumer grouping can be obtained. Compared with other methods (such as K-means clustering), mean shift clustering does not require a preset number of consumers, but relies on a natural data-driven classification method.

[0037] (3) For any type of carbon credit seller, a corresponding carbon credit quantity prediction model is established.

[0038] For each group obtained by clustering, a carbon credit quantity prediction model is established separately. This can be based on a variety of machine learning methods, such as linear regression, decision tree, random forest, support vector machine (SVM), neural network, etc.

[0039] As an optional embodiment, the method for constructing a carbon credit pricing model provided by the present invention establishes a corresponding carbon credit quantity prediction model for any type of carbon credit seller, including: obtaining the actual pricing of carbon credits of any type of carbon credit seller and the next actual quantity of carbon credits corresponding to the actual pricing of carbon credits; using the actual pricing of carbon credits as a training input sample and the corresponding next actual quantity of carbon credits as an output sample to construct training sample data; using the training sample data to train a preset neural network; and using the trained preset neural network as a carbon credit quantity prediction model.

[0040] The present invention uses the actual pricing of carbon credits as an input feature. If necessary, other features that may affect the number of carbon credits, such as time, seasonal factors, etc., can also be added, and the corresponding actual number of carbon credits is used as the output target.

[0041] In addition, user behavior may change over time, and the present invention can periodically retrain the model to adapt to new situations.

[0042] Through the above process, the present invention can establish a personalized carbon credit quantity prediction model for different types of carbon credit sellers, so as to more accurately predict the future carbon credit supply situation, and further support a more effective carbon credit pricing strategy.

[0043] Based on the contents of the above embodiments, as an optional embodiment, the method for constructing a carbon credit pricing model provided by the present invention generates new experience tuples to update the experience pool during the interaction between the reinforcement learning model and the environment.

[0044] The newly generated experience tuple is added to the experience pool. If the experience pool is full, the experience pool is updated by removing the oldest experience tuple or randomly replacing an existing experience tuple. By continuously updating the experience pool, the model's generalization ability and ability to respond to market changes can be improved.

[0045] In order to more clearly illustrate the implementation of the present invention, as another optional embodiment, further description is given below.

[0046] Figure 2 It is a schematic diagram of the structure of the deep reinforcement learning strategy framework provided by the present invention. Figure 2 As shown, the decision-making process of this method includes five key elements: environment, agent, state, action and reward. In the present invention, the environment refers to the K LSTM models (i.e., carbon credit quantity prediction models) that have been trained, the agent refers to the carbon emission trading agency, the state refers to the carbon credit price and carbon credit quantity of each group, the action refers to the carbon credit price of each group determined by the agent, and the reward refers to the net income of the carbon emission trading agency.

[0047] The optimization decision method relies on two deep neural networks, namely the evaluation network and action network In order to take flexible actions in continuous space, the parameter Used to represent a function approximator to estimate Value function. Evaluation network is used to evaluate the policy, taking a state and actions , and output The estimated value of .

[0048] To evaluate the new strategy, the evaluation network minimizes the loss function get Time Difference (TD) Estimation of the Value Function Action Network Used to generate new state actions , enter a state , and outputs a set of continuous actions. Value function optimizes pricing strategy.

[0049] In order to ease the decision-making process Unstable fluctuations in values, target network and action network They are introduced to evaluate the network and action network The stability of the exploration process is improved by limiting slow changes in the target value.

[0050] Prioritized experience replay is used to improve the efficiency of this pricing method. Considering that the price decision optimization of carbon emission trading agency is not an independent and identically distributed model, an experience buffer is introduced to store previous experience. It stores previous pricing experience and uniformly samples new data in small batches in each new time period. By mixing past and updated experience, the time correlation in the reset experience is reduced, and the action and evaluation ability of the network are improved. Based on the time difference error The absolute value of the sample experience Probability .

[0051] ; ; ; in, It's a reward. , Represents the degree of prioritization, is based on the time difference error, The absolute value of the experience gained 's ranking number, It’s experience The priority of is the discount factor.

[0052] In the specific exploration operation, random Gaussian noise was added In order to fully explore the market environment and provide more options for the agents running carbon emission trading agencies, large noise is used in the early stage to explore more action spaces. As the environment experience accumulates, the noise gradually decreases, and the agent can choose a more appropriate action based on experience.

[0053] ; ; ; where ζ is the decay rate of the exploration noise.

[0054] In the DDPG method, in order to improve computational efficiency, gradients are used to represent the direction of change in the incentive price action, thereby obtaining better optimization values. For a given state , forward propagation generates actions .gradient The evaluation of the evaluation network is completed through back propagation, and the action network is updated.

[0055] Finally, combining the target network and the priority experience method, the improved calculation process of TD evaluation loss and action update gradient in DDPG can be expressed as: ; ; in, is the size of the mini-batch.

[0056] The parameters of the online evaluation network and action network are updated as follows: ; ; in, and are the learning rates of gradient descent respectively. The parameter update of the target network is also as follows: ; ; in, is the update rate.

[0057] The present invention also provides a device for constructing a carbon credit pricing model, comprising: The first processing module is used to construct a reinforcement learning model using the DDPG algorithm; the network architecture of the reinforcement learning model is an Actor-Critic network architecture, including an Actor network and a Critic network; the Actor network is used to generate corresponding action variables under given state variables, and the Critic network is used to estimate and evaluate given state variables and corresponding action variables; wherein the state variable is the number of carbon credits of each type of carbon credit seller, the action variable is the carbon credit pricing for each type of carbon credit seller, and the reward is the net income of the carbon credit pricing party; the net income is the income from the sale of carbon emission rights minus the cost of purchasing all carbon credits from the carbon credit seller, and the carbon emission rights are obtained by the carbon credit pricing party by exchanging all the purchased carbon credits; The second processing module is used to set the environment required for the reinforcement learning model to perform learning and training; the environment is a carbon credit quantity prediction model for different types of carbon credit sellers; the carbon credit quantity prediction model is used to output a corresponding state variable according to an input action variable; The third processing module is used to generate multiple experience tuples according to the continuous interaction between the reinforcement learning model and the environment, and store the multiple experience tuples in the experience pool; each experience tuple includes a state variable, an action variable, a reward, and a state variable after state transfer; A fourth processing module is used to train the reinforcement learning model based on the experience tuples in the experience pool until the reinforcement learning model converges; The fifth processing module is used to extract the Actor network from the converged reinforcement learning model as a carbon credit pricing model to determine the carbon credit pricing for each type of carbon credit seller based on the number of carbon credits of each type of carbon credit seller.

[0058] It should be noted that the device for constructing a carbon credit pricing model provided in an embodiment of the present invention can, when specifically running, execute the method for constructing a carbon credit pricing model described in any of the above embodiments, which will not be elaborated in this embodiment.

[0059] Figure 3 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the method for constructing the carbon credit pricing model.

[0060] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the method for constructing a carbon credit pricing model provided in the above embodiments.

[0061] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method for constructing a carbon credit pricing model provided in the above-mentioned embodiments.

[0062] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a carbon credit pricing model, characterized in that: include: Use the DDPG algorithm to build a reinforcement learning model; The network architecture of the reinforcement learning model is an Actor-Critic network architecture, including an Actor network and a Critic network; the Actor network is used to generate corresponding action variables under given state variables, and the Critic network is used to estimate and evaluate given state variables and corresponding action variables; wherein the state variable is the number of carbon credits of each type of carbon credit seller, the action variable is the carbon credit pricing for each type of carbon credit seller, and the reward is the net income of the carbon credit pricing party; the net income is the income from the sale of carbon emission rights minus the cost of purchasing all carbon credits from the carbon credit seller, and the carbon emission rights are obtained by the carbon credit pricing party by exchanging all the purchased carbon credits; Setting the environment required for the reinforcement learning model to perform learning and training; the environment is a carbon credit quantity prediction model for different types of carbon credit sellers; the carbon credit quantity prediction model is used to output corresponding state variables based on input action variables; According to the continuous interaction between the reinforcement learning model and the environment, multiple experience tuples are generated and stored in the experience pool; each experience tuple includes state variables, action variables, rewards, and state variables after state transfer; The reinforcement learning model is trained based on the experience tuples in the experience pool until the reinforcement learning model converges; An Actor network is extracted from the converged reinforcement learning model as a carbon credit pricing model to determine the carbon credit pricing for each type of carbon credit seller based on the number of carbon credits of each type of carbon credit seller.

2. The method for constructing a carbon credit pricing model according to claim 1, characterized in that: Methods for determining carbon credit quantity prediction models for different types of carbon credit sellers include: Extract features from the historical data of low-carbon behaviors and historical quantity data of carbon credits of the seller of carbon credits, and construct feature data of the seller of carbon credits; Performing cluster analysis on the characteristic data of the carbon credit sellers using a clustering algorithm to determine different types of carbon credit sellers; For any type of carbon credit seller, a corresponding carbon credit quantity prediction model is established.

3. The method for constructing a carbon credit pricing model according to claim 1, characterized in that: For any type of carbon credit seller, a corresponding carbon credit quantity prediction model is established, including: Obtaining the actual carbon credit pricing of any type of carbon credit seller and the actual number of next carbon credits corresponding to the actual carbon credit pricing; The actual pricing of carbon credits is used as the training input sample, and the corresponding actual number of next carbon credits is used as the output sample to construct the training sample data; Using the training sample data to train a preset neural network; The trained preset neural network is used as a carbon credit quantity prediction model.

4. The method for constructing a carbon credit pricing model according to claim 2, characterized in that: The clustering algorithm is a mean shift clustering algorithm.

5. The method for constructing a carbon credit pricing model according to claim 1, characterized in that: During the interaction between the reinforcement learning model and the environment, new experience tuples are generated to update the experience pool.

6. The method for constructing a carbon credit pricing model according to claim 2, characterized in that: The feature extraction method includes: principal component analysis method.

7. A device for constructing a carbon credit pricing model, characterized in that: include: The first processing module is used to build a reinforcement learning model using the DDPG algorithm; The network architecture of the reinforcement learning model is an Actor-Critic network architecture, including an Actor network and a Critic network; the Actor network is used to generate corresponding action variables under given state variables, and the Critic network is used to estimate and evaluate given state variables and corresponding action variables; wherein the state variable is the number of carbon credits of each type of carbon credit seller, the action variable is the carbon credit pricing for each type of carbon credit seller, and the reward is the net income of the carbon credit pricing party; the net income is the income from the sale of carbon emission rights minus the cost of purchasing all carbon credits from the carbon credit seller, and the carbon emission rights are obtained by the carbon credit pricing party by exchanging all the purchased carbon credits; The second processing module is used to set the environment required for the reinforcement learning model to perform learning and training; the environment is a carbon credit quantity prediction model for different types of carbon credit sellers; the carbon credit quantity prediction model is used to output a corresponding state variable according to an input action variable; The third processing module is used to generate multiple experience tuples according to the continuous interaction between the reinforcement learning model and the environment, and store the multiple experience tuples in the experience pool; each experience tuple includes a state variable, an action variable, a reward, and a state variable after state transfer; A fourth processing module is used to train the reinforcement learning model based on the experience tuples in the experience pool until the reinforcement learning model converges; The fifth processing module is used to extract the Actor network from the converged reinforcement learning model as a carbon credit pricing model to determine the carbon credit pricing for each type of carbon credit seller based on the number of carbon credits of each type of carbon credit seller.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method for constructing the carbon credit pricing model as described in any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for constructing a carbon credit pricing model as described in any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for constructing a carbon credit pricing model as described in any one of claims 1 to 6 are implemented.

Images