Reinforcement learning-based bridge life cycle carbon emission reduction cost efficiency optimization method and device, computer equipment and readable storage medium

Abstract

The application discloses a bridge full-life-cycle carbon emission reduction cost efficiency optimization method and device based on reinforcement learning, computer equipment and a readable storage medium, comprising: acquiring carbon emission data, cost data and optional scheme parameters of each link of the full-life-cycle, and constructing a dynamic database; constructing an optimization model based on the database, and defining unit economic carbon reduction efficiency as an objective function; constructing a reinforcement learning environment, initializing a strategy network, and iteratively training through a reinforcement learning algorithm to solve optimal strategies of each link; and running the optimal strategy in a verification set, and outputting total full-life-cycle carbon emission, economic cost and link carbon reduction efficiency analysis results. The application realizes multi-link collaborative optimization by combining dynamic database real-time data updating and reinforcement learning, and provides intelligent and low-carbon full-life-cycle decision support for bridge engineering.

Description

Technical Field

[0001] This invention relates to the field of infrastructure construction, and more specifically, to a method, apparatus, computer equipment, and readable storage medium for optimizing the cost-efficiency of carbon emission reduction throughout the entire life cycle of bridges based on reinforcement learning. Background Technology

[0002] Bridge engineering involves significant carbon emissions and costs throughout its entire lifecycle. Traditional management methods often rely on experience-based decision-making, making it difficult to achieve overall optimization of carbon reduction and cost efficiency at each stage. Existing technologies have the following shortcomings: First, the data is static, failing to consider dynamic factors such as material price fluctuations and carbon emission factor updates; second, the objectives are singular, focusing on emission reduction in localized stages while neglecting coordinated optimization throughout the entire lifecycle; and third, there is a lack of quantitative models, making it difficult to balance the coupling relationship between emission reduction benefits and economic costs. Summary of the Invention

[0003] The purpose of this invention is to provide a method, apparatus, computer equipment, and readable storage medium for optimizing the cost-efficiency of carbon emission reduction throughout the entire life cycle of bridges based on reinforcement learning.

[0004] In a first aspect, embodiments of the present invention provide a method for optimizing the cost-efficiency of carbon emission reduction throughout the entire life cycle of a bridge based on reinforcement learning, including:

[0005] Acquire carbon emission data, cost data, and parameters of alternative solutions at each stage of the bridge's entire life cycle, and construct a dynamic database;

[0006] Based on the dynamic database, a carbon emission reduction cost-efficiency optimization model is constructed, and the unit economic carbon reduction efficiency is defined as the objective function. The unit economic carbon reduction efficiency is the ratio of the carbon emission difference between the alternative scheme and the basic scheme at each stage to the cost difference.

[0007] Based on the carbon emission reduction cost-efficiency optimization model, a reinforcement learning environment is constructed and a policy network is initialized. The initialized policy network is then used to perform iterative training through a reinforcement learning algorithm to solve the optimization model and obtain the optimal policy for each stage output by the policy network. The reinforcement learning environment includes a state space, an action space, a reward function, and state transition rules. The state space takes each stage of the entire life cycle as a time step and includes the basic and alternative scheme attributes for the current stage.

[0008] Based on the optimal strategies for each stage output by the strategy network, the system runs a preset number of times in the verification set and records the optimal path, outputting the corresponding total carbon emissions, economic cost, and carbon reduction efficiency analysis results for each stage throughout the entire life cycle.

[0009] In one possible implementation, the unit economic carbon reduction efficiency is calculated using the following formula:

[0010]

[0011] in, Represents the first in the life cycle Unit economic carbon reduction efficiency at each stage Indicates the resource scale of the alternative solution. Indicates the resource scale of the basic scheme. The carbon emission factor representing the alternative solution. Indicates the carbon emission factor of the basic scheme. This indicates the unit cost of the alternative solution. This represents the unit cost of the basic design. This indicates the total carbon emissions of the alternative solutions. This indicates the total carbon emissions of the basic scheme. This represents the total economic cost of the alternative solution. This represents the total economic cost of the basic design.

[0012] In one possible implementation, the state space also includes the current stage number, cumulative carbon emissions, cumulative economic cost, and the resource scale, carbon emission factor, and unit cost of the basic and alternative schemes for the current stage in the dynamic database;

[0013] The action space is a set of discrete actions. ;in To select a basic scheme, To select an alternative, a decision is made based on the output results of the cumulative carbon emissions, cumulative economic cost, and current stage scheme attributes in the state space, using a policy network.

[0014] In one possible implementation, the reward function is:

[0015]

[0016] The state transition rule is as follows: based on the current state and the action selection result, update the cumulative carbon emissions and cumulative economic cost to obtain the state of the next stage. When all stages are completed, the current optimization process ends.

[0017] In one possible implementation, the step of iteratively training the initialized policy network using a reinforcement learning algorithm to solve the optimization model and obtain the optimal policy for each stage output by the policy network includes:

[0018] The initial policy network is used to iteratively train the optimization model using the PPO algorithm to obtain the optimal policy path output by the policy network, the total carbon emissions and economic cost over the entire life cycle, the stage efficiency comparison, and the sensitivity analysis results.

[0019] Based on the optimal strategy path, the total carbon emissions and economic cost over the entire life cycle, the stage efficiency comparison, and the sensitivity analysis results, the optimal strategies for each stage output by the strategy network are obtained.

[0020] In one possible implementation, the sensitivity analysis results are obtained by means of:

[0021] At least one parameter in the current stage of the dynamic database, namely the resource scale, carbon emission factor and unit cost, is randomly perturbed, and Monte Carlo simulation is performed to record the total carbon emissions and economic cost of the entire life cycle in each simulation.

[0022] Calculate the coefficient of variation of total carbon emissions and economic cost for each simulation throughout the entire life cycle. When the coefficient of variation is less than a preset threshold, the optimal strategy path is determined to be stable under parameter fluctuation scenarios.

[0023] In one possible implementation, the step of acquiring carbon emission data, cost data, and alternative scheme parameters at each stage of the bridge's entire life cycle, and constructing a dynamic database, includes:

[0024] Carbon emission data, cost data, and optional scheme parameters of the bridge at each stage of its entire life cycle are acquired and stored in a dynamic database. The data in the dynamic database is updated in real time based on at least one of the following factors: market fluctuations, policy and standard adjustments, new actual engineering data, or technological iterations, so as to obtain the completed dynamic database.

[0025] Secondly, embodiments of the present invention provide a bridge lifecycle carbon emission reduction cost-efficiency optimization device based on reinforcement learning, comprising:

[0026] The acquisition module is used to acquire carbon emission data, cost data, and optional scheme parameters for each stage of the bridge's entire life cycle and construct a dynamic database. Based on the dynamic database, a carbon emission reduction cost-efficiency optimization model is constructed, and the unit economic carbon reduction efficiency is defined as the objective function. The unit economic carbon reduction efficiency is the ratio of the difference in carbon emissions between the alternative scheme and the basic scheme at each stage to the difference in cost.

[0027] An optimization module is used to construct a reinforcement learning environment and initialize a policy network based on the carbon emission reduction cost-efficiency optimization model. The initialized policy network is then iteratively trained using a reinforcement learning algorithm to solve the optimization model, obtaining the optimal policies for each stage output by the policy network. The reinforcement learning environment includes a state space, an action space, a reward function, and state transition rules. The state space uses each stage of the entire lifecycle as a time step and includes the basic and alternative scheme attributes for the current stage. Based on the optimal policies for each stage output by the policy network, the system runs the algorithm a preset number of times in a validation set, records the optimal path, and outputs the corresponding total carbon emissions, economic cost, and carbon reduction efficiency analysis results for each stage throughout the entire lifecycle.

[0028] Thirdly, embodiments of the present invention provide a computer device, the computer device including a processor and a non-volatile memory storing computer instructions, wherein when the computer instructions are executed by the processor, the computer device performs the method described in the first aspect.

[0029] Fourthly, embodiments of the present invention provide a readable storage medium, the readable storage medium including a computer program, wherein the computer program, when running, controls the computer device where the readable storage medium is located to execute the method described in the first aspect.

[0030] Compared to existing technologies, the beneficial effects of this invention include: employing a reinforcement learning-based method, apparatus, computer equipment, and readable storage medium for optimizing the cost-efficiency of carbon emission reduction throughout the entire lifecycle of bridges, as disclosed in this invention, a dynamic database is constructed by acquiring carbon emission data, cost data, and parameters of possible solutions at each stage of the entire lifecycle; an optimization model is built based on the database, defining the unit economic carbon reduction efficiency as the objective function; a reinforcement learning environment is constructed, a policy network is initialized, and iterative training is performed using a reinforcement learning algorithm to solve for the optimal policy at each stage; the optimal policy is run on a validation set, and the analysis results of total carbon emissions, economic cost, and carbon reduction efficiency at each stage throughout the entire lifecycle are output. This invention updates data in real time through a dynamic database and achieves multi-stage collaborative optimization by combining reinforcement learning, providing intelligent and low-carbon lifecycle decision support for bridge engineering. Attached Figure Description

[0031] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as limiting the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1A flowchart illustrating the steps of the reinforcement learning-based bridge lifecycle carbon emission reduction cost-efficiency optimization method provided in this embodiment of the invention.

[0033] Figure 2 A schematic block diagram of the structure of the bridge life-cycle carbon emission reduction cost-efficiency optimization device based on reinforcement learning provided in an embodiment of the present invention;

[0034] Figure 3 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0036] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0037] In order to solve the technical problems mentioned in the background art Figure 1 This is a flowchart illustrating the reinforcement learning-based method for optimizing the cost-efficiency of carbon emission reduction throughout the entire life cycle of bridges, as provided in this embodiment. The following is a detailed description of this reinforcement learning-based method for optimizing the cost-efficiency of carbon emission reduction throughout the entire life cycle of bridges.

[0038] Step S201: Obtain carbon emission data, cost data, and optional scheme parameters for each stage of the bridge's entire life cycle, and construct a dynamic database;

[0039] Step S202: Based on the dynamic database, construct a carbon emission reduction cost-efficiency optimization model and define the unit economic carbon reduction efficiency as the objective function. The unit economic carbon reduction efficiency is the ratio of the carbon emission difference between the alternative scheme and the basic scheme at each stage to the cost difference.

[0040] Step S203: Based on the carbon emission reduction cost-efficiency optimization model, a reinforcement learning environment is constructed and a policy network is initialized. The initialized policy network is then used to perform iterative training through a reinforcement learning algorithm to solve the optimization model and obtain the optimal policy for each stage output by the policy network. The reinforcement learning environment includes a state space, an action space, a reward function, and state transition rules. The state space takes each stage of the entire life cycle as a time step and includes the basic and alternative scheme attributes for the current stage.

[0041] Step S204: Based on the optimal strategies for each stage output by the strategy network, run the system a preset number of times in the verification set and record the optimal path. Output the corresponding total carbon emissions, economic cost, and carbon reduction efficiency analysis results for each stage throughout the entire life cycle.

[0042] In this embodiment of the invention, for example, the server aims to coordinate the optimization of carbon emission reduction and cost efficiency throughout the entire life cycle of a bridge. It achieves intelligent decision support through four core steps: dynamic database construction, optimized model definition, reinforcement learning environment design and training, and optimal strategy verification. The specific implementation process is as follows:

[0043] The server first conducts multi-source data collection and integration across all stages of the bridge's entire lifecycle to build a dynamic database. The entire lifecycle covers five key stages: material production, transportation, construction, operation and maintenance, and demolition. The server automatically acquires basic and alternative solution data for each stage by connecting to the bridge design unit's BIM system, the material supplier's resource management system, the construction unit's project execution platform, the operation and maintenance unit's monitoring terminal, and the logistics company's transportation management system. For example, in the material production stage, the server collects data on the resource scale, carbon emission factors, and unit cost of basic materials such as steel and cement from traditional building material supplier systems, while simultaneously obtaining corresponding data on recycled steel and alternative materials such as those mixed with industrial waste (e.g., fly ash) from recycling company systems. In the transportation stage, the server collects data on the energy consumption, emission factors, and transportation costs of diesel transport vehicles through logistics platforms, while also obtaining data on the power consumption, regional power grid emission factors, and electricity costs of electric trucks from new energy transportation companies. The collected raw data is standardized by the server, with carbon emission unit set to kgCO2e and cost unit set to yuan. It is stored in a hierarchical structure of "stage-resource type-scheme type". At the same time, a timed synchronization mechanism is set up to update dynamic data such as material price fluctuations and carbon emission factor adjustments in real time, ensuring the timeliness and accuracy of the database and providing reliable input for subsequent model construction.

[0044] Next, the server constructs a carbon reduction cost-efficiency optimization model based on a dynamic database and defines a core objective function. The model uses the choice of solutions at each stage of the entire lifecycle as optimization variables, and the objective function is set as "unit economic carbon reduction efficiency." This means measuring the carbon reduction benefit achieved per unit cost input by quantifying the ratio of the carbon emission difference between the alternative and the basic solution at each stage to the cost difference. Specifically, the server extracts the resource scale, carbon emission factor, and unit cost parameters of the basic and alternative solutions from the database for the current stage. It calculates the total carbon emissions (the product of resource scale and emission factor) and economic cost (the product of resource scale and unit cost) of the basic solution, and calculates the corresponding values ​​for the alternative solutions in the same way. This yields the carbon emission difference (total carbon emissions of the alternative solution minus total carbon emissions of the basic solution, with the absolute value representing the emission reduction) and the cost difference (economic cost of the alternative solution minus economic cost of the basic solution). The ratio of these two values ​​is the unit economic carbon reduction efficiency for that stage. Through this objective function, the server deeply couples carbon reduction with cost control indicators, providing a quantitative basis for reinforcement learning decision-making.

[0045] Subsequently, the server constructs a reinforcement learning environment, initializes the policy network, and iteratively trains and optimizes the model. The design of the reinforcement learning environment revolves around the state space, action space, reward function, and state transition rules: the state space uses each stage of the entire lifecycle as a time step, and the state vector at each time step includes the current stage number, the cumulative carbon emissions up to the previous stage, the cumulative economic cost, and attributes such as resource scale, carbon emission factor, and unit cost of the basic and alternative solutions for the current stage; the action space is defined as a binary decision, i.e., choosing the basic solution (action 0) or the alternative solution (action 1); the reward function is directly related to the objective function. When choosing the alternative solution, the reward value is the unit economic carbon reduction efficiency of that stage; when choosing the basic solution, the reward value is 0, thus guiding the policy network to prioritize the solution with high emission reduction efficiency; the state transition rules update the cumulative carbon emissions and cumulative economic cost according to the current action and proceed to the state of the next stage. The policy network adopts a fully connected neural network structure, with the input being the state vector and the output being the probability of choosing the alternative solution. The network parameters are initialized using a normal distribution, and the learning rate and discount factor are set according to empirical values. The server uses the PPO algorithm for iterative training. In each training round, multiple complete decision paths (i.e., the sequence of options for the entire process from material production to waste disposal) are sampled in the reinforcement learning environment through the current policy network. The network parameters are optimized based on the total reward value of the path (the sum of the unit economic carbon reduction efficiency of each stage). The training stability is ensured by limiting the policy update amplitude. After multiple iterations, the policy network gradually converges and outputs the stable probability distribution of the options for each stage, i.e. the optimal policy for each stage.

[0046] Finally, the server validates the trained policy network on the validation set and outputs the optimal policy. The validation set contains multiple sets of bridge engineering data in different scenarios (such as bridges with different spans, structural types, and regional environments). The server runs the policy network a preset number of times in each scenario, recording the cumulative carbon emissions, cumulative economic costs, and total reward value of all decision paths. The path with the highest total reward value is selected as the optimal policy path for that scenario. Based on the optimal path, the server calculates the total carbon emissions (the sum of carbon emissions from each selected option in each stage) and economic costs (the sum of costs from each selected option in each stage) over the entire life cycle, and ranks the unit economic carbon reduction efficiency of each stage, identifying high-priority emission reduction stages (such as the substitution of recycled materials in the material production stage and the application of new energy transportation tools in the transportation stage). At the same time, the server tests the robustness of the optimal policy by applying a certain degree of perturbation to key parameters in the database (such as material prices and emission factors), verifying the stability of policy selection under parameter fluctuations. The server integrates the above results into a visual report that includes an optimal strategy path map, a carbon emission and cost comparison analysis, a process efficiency ranking, and a sensitivity analysis. This provides bridge engineering decision-makers with a carbon emission reduction and cost control solution covering the entire life cycle, achieving synergistic optimization of low-carbon construction and economic efficiency.

[0047] Through the above implementation process, the server has achieved full-process automation from dynamic collection of multi-source data to intelligent decision output, effectively solving the problems of the separation between carbon emission reduction and cost targets and the reliance on experience in decision-making in traditional bridge life cycle management, and providing technical support for the green and economical full life cycle management of bridge engineering.

[0048] In this embodiment of the invention, the unit economic carbon reduction efficiency is calculated using the following formula:

[0049]

[0050] in, Represents the first in the life cycle Unit economic carbon reduction efficiency at each stage Indicates the resource scale of the alternative solution. Indicates the resource scale of the basic scheme. The carbon emission factor representing the alternative solution. Indicates the carbon emission factor of the basic scheme. This indicates the unit cost of the alternative solution. This represents the unit cost of the basic design. This indicates the total carbon emissions of the alternative solutions. This indicates the total carbon emissions of the basic scheme. This represents the total economic cost of the alternative solution. This represents the total economic cost of the basic design.

[0051] In an embodiment of the invention, for example, when the server calculates the unit economic carbon reduction efficiency during the material production process, it extracts parameters of the basic scheme and the recycled steel alternative scheme for Q345 steel from the dynamic database: In the basic scheme, the resource scale S base,i =12,000 tons, carbon emission factor f base,i =2251.41kgCO2e / ton, unit cost c base,i =4000 yuan / ton; In the alternative plan, the resource scale S alt,i =12,000 tons, carbon emission factor f alt,i =357kgCO2e / ton, unit cost c alt,i =3500 yuan / ton. Server calculation using the formula: E alt,i =S alt,i ×f alt,i =12000×357=4284000kgCO2e,E base,i =S base,i ×f base,i =12000×2251.41=27016920kgCO2e; C alt,i =S alt,i ×c alt,i =12000×3500=42 million yuan, C base,i =S base,i ×c base,i =12000 × 4000 = 48 million yuan. Therefore, the efficiency is... i =(E base,i -E alt,i ) / (C base,i -C alt,i =(27016920-4284000) / (48000000-42000000)=22732920 / 6000000≈3.789kgCO2e / yuan, which means that for every 1 yuan reduction in cost by choosing to recycle steel in this process, emissions can be reduced by approximately 3.789kgCO2e.

[0052] In this embodiment of the invention, the state space also includes the current stage number, cumulative carbon emissions, cumulative economic cost, and the resource scale, carbon emission factor and unit cost of the basic and alternative solutions for the current stage in the dynamic database;

[0053] The action space is a set of discrete actions. ;in To select a basic scheme, To select an alternative, a decision is made based on the output results of the cumulative carbon emissions, cumulative economic cost, and current stage scheme attributes in the state space, using a policy network.

[0054] In an embodiment of the invention, for example, when the server processes the material production stage of a bridge's entire lifecycle (current stage number i=1), the constructed state space includes: cumulative carbon emissions of 0 kgCO2e, cumulative economic cost of 0 yuan, and the scheme attributes of the current stage in the dynamic database. The basic scheme (Q345 steel) has a resource scale of 12,000 tons, a carbon emission factor of 2251.41 kgCO2e / ton, and a unit cost of 4,000 yuan / ton; the alternative scheme (recycled steel) has a resource scale of 12,000 tons, a carbon emission factor of 357 kgCO2e / ton, and a unit cost of 3,500 yuan / ton. The action space is a discrete set {a1}. {0,1}, where a1=0 indicates the selection of the basic option and a1=1 indicates the selection of the alternative option. The policy network receives the cumulative carbon emissions (0), cumulative cost (0), and current stage option attributes (such as the emission factor of the basic option being 2251.41, the unit cost of the alternative option being 3500, etc.) from the state space, and outputs the probability of action a1=1 as 0.85. Based on this, the server makes a decision to select the steel recycling alternative option.

[0055] In this embodiment of the invention, the reward function is:

[0056]

[0057] The state transition rule is as follows: based on the current state and the action selection result, update the cumulative carbon emissions and cumulative economic cost to obtain the state of the next stage. When all stages are completed, the current optimization process ends.

[0058] In an embodiment of the invention, for example, after the server makes a decision to select an alternative (a1=1) in the material production stage (i=1), it calculates an immediate reward r1 based on the reward function: and extracts the carbon emission E of the basic scheme from the dynamic database. base,i =27016920kgCO2e, Economic cost C base,i =48 million yuan, alternative carbon emissions E alt,i =4284000kgCO2e, economic cost C alt,i =42 million yuan, substituting into the formula, we get r1=(E alt,i -E base,i ) / (C alt,i -C base,i= (4284000-27016920) / (42000000-48000000) = 3.789 kgCO2e / yuan. Then, according to the state transition rules, the current cumulative carbon emissions (0 kgCO2e) are updated to 0 + 4284000 = 4284000 kgCO2e, and the cumulative economic cost (0 yuan) is updated to 0 + 42000000 = 42 million yuan. The state transitions to the transportation stage (i=2). At this point, the state space includes i=2, cumulative carbon emissions of 4284000 kgCO2e, cumulative cost of 42 million yuan, and the resource scale, emission factor, and unit cost of the transportation stage's infrastructure and alternative solutions. The next stage decision is then made.

[0059] In this embodiment of the invention, the method of using the initialized policy network to perform iterative training through a reinforcement learning algorithm to solve the optimization model and obtain the optimal policy of each link output by the policy network can be implemented through the following example.

[0060] The initial policy network is used to iteratively train the optimization model using the PPO algorithm to obtain the optimal policy path output by the policy network, the total carbon emissions and economic cost over the entire life cycle, the stage efficiency comparison, and the sensitivity analysis results.

[0061] Based on the optimal strategy path, the total carbon emissions and economic cost over the entire life cycle, the stage efficiency comparison, and the sensitivity analysis results, the optimal strategies for each stage output by the strategy network are obtained.

[0062] In this embodiment of the invention, for example, the server takes the entire life cycle of a cross-river bridge as the object, initializes the policy network as a fully connected network with two hidden layers (128 neurons each), the input is a state vector (including the stage number, cumulative carbon emissions, cumulative cost, and the current stage's scheme attribute), and the output is the probability of choosing an alternative scheme. The network parameters are initialized using a He normal distribution, with a learning rate of 0.0003 and a discount factor of 0.95. The PPO algorithm is used for iterative training for 1000 rounds. In each round, 20 decision paths (including a 5-stage selection sequence from material production to demolition) are sampled from the reinforcement learning environment: for example, in a certain path, the policy network outputs an alternative scheme probability of 0.85 for the material production stage, choosing to recycle steel (action 1), and receiving a reward of 3.789 kgCO2e / yuan (unit economic carbon reduction efficiency); the alternative scheme probability for the transportation stage is 0.8, choosing an electric truck (action 1), with a calculated efficiency of 0.089 kgCO2e / yuan, and receiving the corresponding reward; after completing the 5 stages in sequence, the total cumulative reward is the sum of the efficiencies of each stage. The server uses the PPO pruning objective function to update network parameters and limits the policy update magnitude (ε=0.2). After 1000 iterations, the network converges and the output action probabilities of each stage tend to stabilize (material production 0.9+, transportation 0.85+, construction 0.3-, operation 0.8+, and waste disposal 0.75+).

[0063] After training, the server ran 100 times on the validation set (50 bridge scenarios) and recorded the optimal path: material production (1), transportation (1), construction (0, due to efficiency 0.042 < threshold 0.05), operation and maintenance (1), and demolition (1). The total carbon emissions of this path throughout its entire life cycle were 5.2 million kg CO2e, and the economic cost was 120 million yuan. The efficiency comparison of each stage showed that material production (3.789 kg CO2e / yuan) > transportation (0.089) > operation (0.072) > demolition (0.065) > construction (0.042). Through sensitivity analysis (material price ±10% disturbance), the stability of the strategy was verified (efficiency still higher than the threshold after fluctuation). Finally, based on the optimal path, total amount, efficiency, and sensitivity results, the optimal strategy for each stage was determined: recycled steel for material production, electric trucks for transportation, foundation scheme for construction, photovoltaic power supply for operation, and recycling for demolition.

[0064] In this embodiment of the invention, the sensitivity analysis results are obtained in the following ways, and can be implemented through the following examples.

[0065] At least one parameter in the current stage of the dynamic database, namely the resource scale, carbon emission factor and unit cost, is randomly perturbed, and Monte Carlo simulation is performed to record the total carbon emissions and economic cost of the entire life cycle in each simulation.

[0066] Calculate the coefficient of variation of total carbon emissions and economic cost for each simulation throughout the entire life cycle. When the coefficient of variation is less than a preset threshold, the optimal strategy path is determined to be stable under parameter fluctuation scenarios.

[0067] In an embodiment of the present invention, for example, the server conducts a sensitivity analysis on the optimal strategy path (material production using recycled steel, transportation using electric trucks, etc.) for a certain cross-river bridge. It selects the unit cost parameter (original value 3500 yuan / ton, fluctuation range 3500±10%, i.e. 3150-3850 yuan / ton) of the alternative solution (recycled steel) in the material production link of the dynamic database and applies random perturbation. It generates 1000 sets of perturbation samples through Monte Carlo simulation (each time sampling uniformly from 3150-3850 yuan / ton).

[0068] For each sample, the server recalculates the total carbon emissions (resource scale of 12,000 tons × emission factor of 357 kgCO2e / ton; since the emission factor is not disturbed, the total carbon emissions are fixed at 4,284,000 kgCO2e) and economic cost (12,000 tons × perturbation cost per unit). It also links the fixed parameters of other links such as transportation and construction to obtain the total carbon emissions (all of which are 5.2 million kgCO2e, since only the cost disturbance does not affect carbon emissions) and economic cost (fluctuating with the material cost in the range of 118-122 million yuan).

[0069] The server calculated the average economic cost of the entire life cycle from 1000 simulations to be 120 million yuan, with a standard deviation of 360,000 yuan. The coefficient of variation was calculated as 360,000 yuan / 120 million yuan = 3%. The total carbon emissions remained unchanged, with a coefficient of variation of 0%. The preset threshold was 5%. Since the coefficients of variation for both carbon emissions (0%) and cost (3%) were less than 5%, the optimal strategy path was determined to be stable under the scenario of ±10% fluctuation in the unit cost of materials.

[0070] In this embodiment of the invention, the step of acquiring carbon emission data, cost data, and optional scheme parameters at each stage of the bridge's entire life cycle and constructing a dynamic database can be implemented through the following example.

[0071] Carbon emission data, cost data, and optional scheme parameters of the bridge at each stage of its entire life cycle are acquired and stored in a dynamic database. The data in the dynamic database is updated in real time based on at least one of the following factors: market fluctuations, policy and standard adjustments, new actual engineering data, or technological iterations, so as to obtain the completed dynamic database.

[0072] In this embodiment of the invention, for example, the server obtains data for each stage of the entire lifecycle by connecting to the bridge design unit's BIM system, the material supplier's ERP system, the construction unit's project management platform, and the operation and maintenance unit's monitoring system: In the material production stage, basic Q345 steel data (scale 12,000 tons, emission factor 2251.41 kgCO2e / ton, unit cost 4000-5500 yuan / ton) is collected from the steel supplier system; and data on steel recycling alternatives (scale 12,000 tons, emission factor 3) is collected from the recycling enterprise system. 57-1987 kg CO2e / ton, unit cost 3500-4500 yuan / ton; in the transportation segment, basic diesel truck scheme data (energy consumption 3358723 kg, emission factor 3.16 kg CO2e / L, cost 7.5-8.5 yuan / L) and electric truck alternative scheme data (electricity consumption 16.037 million kWh, emission factor 0.583 kg CO2e / kWh, cost 0.5-1.2 yuan / kWh) are collected from the logistics platform and stored in a dynamic database according to the structure of "segment-resource type-scheme type".

[0073] The server is configured with a real-time update mechanism: based on market fluctuations, the latest material supplier quotations are synchronized daily at 2:00 AM via API (e.g., the unit cost of Q345 steel is adjusted from 4800 yuan / ton to 4950 yuan / ton); according to policy and standard adjustments, when the state releases a new version of the "Provincial Greenhouse Gas Inventory Compilation Guidelines", the diesel emission factor is automatically updated (slightly adjusted from 3.16 kgCO2e / L to 3.15 kgCO2e / L); when adding actual engineering data, the application parameters of recycled concrete for new bridge projects are integrated (supplementing the alternative resource scale of 9800 tons and the emission factor of 4.2 kgCO2e / ton); after technological iteration, data on new low-carbon steel (emission factor reduced to below 2000 kgCO2e / ton) is integrated into the basic scheme candidate library, and through dynamic updates of multi-source data, a dynamic database with both real-time performance and completeness is finally constructed.

[0074] To more clearly describe the solutions provided in the embodiments of the present invention, a more complete implementation method is provided below.

[0075] This invention discloses a reinforcement learning-based bridge lifecycle carbon emission reduction cost-efficiency optimization model that achieves global optimal carbon emission reduction cost-efficiency through multi-stage state perception and strategy iteration. The model includes: First, analyzing the production, construction, and transportation of bridge construction materials, collecting carbon emission data, cost data, and parameters of alternative solutions (basic or low-carbon alternatives) for each stage, and integrating this information into a real-time updated dynamic database. Second, constructing a bridge carbon emission reduction cost-efficiency optimization model, simulating decision-making choices at different stages using an intelligent decision-making algorithm (based on PPO reinforcement learning): For each stage, the algorithm calculates the economic cost difference and carbon emission reduction effect of two solutions, prioritizing the solution with the highest emission reduction cost-effectiveness, and automatically excluding alternatives with excessively high costs or insignificant emission reduction effects. During this process, the algorithm continuously learns from the effects of historical decisions and dynamically adjusts its strategy to ensure that the final selected solution combination achieves the maximum emission reduction within the overall budget. Finally, the system will conduct multi-dimensional reliability verification: simulate real-world interference factors such as material price fluctuations and changes in carbon emission factors, repeatedly test the stability of the decision-making scheme, and finally output a list of optimal schemes that take into account both economic efficiency and emission reduction effect, clearly indicating which links must adopt low-carbon substitutions and which links can retain the basic schemes, and providing global estimates of costs and emission reductions.

[0076] 1) First, construct a dynamic database for bridge construction.

[0077] First, we analyze the production, construction, and transportation of bridge construction materials. We then obtain the resources and energy at each stage of the bridge's life cycle through design documents, including the scale of the basic and alternative schemes, carbon emission factors, and unit cost data. This information is then integrated into a dynamic database that is updated in real time. Please refer to Tables 1 and 2. Table 1 is an example of the basic scheme database for a certain bridge project, and Table 2 is an example of the alternative scheme database for a certain bridge project.

[0078] Table 1

[0079]

[0080] Table 2

[0081]

[0082] 2) Construction of a cost-efficiency optimization model for bridge carbon emission reduction

[0083] Throughout the entire lifecycle of bridge construction, various stages, such as raw material procurement, transportation, construction, and operation and maintenance, generate significant carbon emissions. To address this, this paper constructs a reinforcement learning-based decision optimization model. This model aims to dynamically select whether to adopt low-carbon alternatives at each stage, thereby maximizing carbon reduction benefits per unit of economic cost while ensuring project feasibility and economic viability.

[0084] We employ reinforcement learning (RL) algorithms to implement the optimal carbon emission reduction efficiency strategy throughout the entire lifecycle of bridge construction and construct an optimal carbon emission reduction efficiency model.

[0085] Throughout the entire bridge construction process, including raw material acquisition, transportation, construction, and energy consumption, different alternative materials or energy sources can be used for carbon reduction.

[0086] Each stage can adopt either a "basic solution" or an "alternative solution." Different stages can choose either a "basic solution" or an "alternative solution" to achieve the goal of carbon emission reduction.

[0087] The goal is to dynamically select whether to use alternative solutions at each stage, maximizing the sum of the carbon emission reduction / cost difference. In other words, the objective is to maximize the carbon emission reduction benefit per unit of economic cost while ensuring project feasibility and cost control.

[0088] This can be formally defined as a multi-stage optimization problem, where in each stage... The "unit economic carbon reduction efficiency" is defined as:

[0089]

[0090] in, Represents the first in the life cycle The unit economic carbon reduction efficiency at each stage, where S represents the resource scale. It represents the carbon emission factor (kgCO2e / unit). This indicates the unit cost (yuan / unit). Total carbon emissions The total economic cost.

[0091] Reinforcement learning controls path decisions by selecting "0 = basic" or "1 = alternative" at each stage, and continuously learns to maximize the aforementioned benefits.

[0092] 3) Solving the algorithm for the cost-efficiency optimization model of bridge carbon emission reduction

[0093] To solve the cost-efficiency optimization model for bridge carbon emission reduction within the RL framework, a standard reinforcement learning environment is first constructed, which includes a state space, an action space, a reward function, and a state transition function.

[0094] A reinforcement learning model for optimizing the cost-efficiency of bridge carbon emission reduction is implemented using the Proximal Policy Optimization (PPO) algorithm. PPO is a policy gradient algorithm that is stable in continuous or discrete action spaces, avoids excessive changes when updating the policy, and is suitable for resource optimization problems with clear structures. Especially when the lifecycle is finite and the state information is clear, PPO can effectively converge to the optimal policy.

[0095] 1) Constructing the state space:

[0096] It serves as the input for the bridge carbon emission reduction cost-efficiency optimization and enhancement model. It captures information about the current decision-making environment at each stage to guide action selection.

[0097] The state space configuration determines the environmental information that the reinforcement learning agent can perceive at each stage. To accurately describe the decision-making context of each stage, this invention defines the state as a vector, which includes the current life cycle stage number, the accumulated carbon emissions and economic costs, and the parameter indicators of the current stage's basic and alternative solutions.

[0098] Construction method: Each state Including: Current stage number Cumulative carbon emissions Cumulative construction cost ; Attributes of the available options at the current stage ( combination)

[0099] 2) Constructing the action space:

[0100] The action space is used by the reinforcement learning agent to make decisions about "whether to use an alternative" in each state, thus defining the possible policy paths at each stage. During the bridge construction lifecycle, combinations of basic or alternative solutions applied at multiple stages are formulated, forming the solution set network path space. This space supports the agent in the reinforcement learning algorithm to explore and adjust its strategies in the solution set path space to find the optimal path.

[0101] Construction method: Constructing a set of actions ;in To select a basic scheme, To select an alternative.

[0102] 3) Construct the reward function:

[0103] The reward function is the core basis for policy optimization, guiding the agent to gradually learn the optimal policy path. It quantifies the contribution of each choice to the final goal, guiding the learning direction. In this model, the reward function is designed as the unit economic carbon reduction efficiency, that is, the ratio of the carbon emission reduction benefit of the alternative to the basic solution within a stage. If the basic solution is chosen in a stage (i.e., action 0), the reward is recorded as 0; if the alternative solution is chosen (action 1), the reward is taken from its efficiency expression. Through this reward setting, the model will automatically favor the solution with more significant carbon reduction effect and better economic efficiency, thereby driving the overall path to converge to the optimal solution.

[0104] Construction method: in the first In this phase, the immediate reward is defined as the unit of economic carbon reduction efficiency:

[0105]

[0106]

[0107] 4) Construct state transition rules:

[0108] State transition rules provide environmental feedback for training agents, supporting PPO optimization iterations. They determine how to transition to the next state after an action is chosen. In this invention, state transition rules are based on a statistical inventory sequence of resources and energy at each stage of the bridge's lifecycle, updating cumulative carbon emissions and cumulative construction costs through action decision results. Each new state is determined by the combined effects of the previous state and the current action.

[0109] Construction method: This includes current total carbon emissions, total cost, and stage number. The decision path is assumed to be a Markov process, depending only on the current state and actions.

[0110] 5) PPO training algorithm for reinforcement learning models

[0111] The path constructed through iterative reinforcement learning, along with sampling, replay, and update strategies, efficiently approximates the optimal path combination, ultimately achieving the optimal carbon reduction benefits for the bridge's entire lifecycle strategy. The output of the algorithm in this invention is as follows:

[0112] Output 1: Optimal policy path (ActionSequence). Outputs the action selection at each stage. The specific path for enabling alternative solutions in bridge lifecycle reinforcement learning.

[0113] Output 2: Total life-cycle carbon emissions and economic cost. Calculate and compare the cumulative carbon emissions before and after implementing the strategy. and cumulative cost

[0114] Output 3: Stage Efficiency Comparison. The efficiency of each stage is quantified as follows: This reveals at which stages choosing an alternative is most economical.

[0115] Output 4: Sensitivity Analysis. (For...) , , Monte Carlo perturbation experiments were conducted with input perturbations (±10%); the stability of strategy selection was analyzed; and the robustness of alternative strategies to cost fluctuations or changes in emission factors was revealed.

[0116] After training, the bridge carbon emission reduction cost-efficiency optimization model will output the optimal life-cycle strategy path, i.e., whether to choose the basic or alternative solution at each stage of the bridge's entire life cycle. It will also statistically analyze the total carbon emissions and economic cost over the entire life cycle, providing decision-makers with a total-level assessment reference. By outputting the unit economic carbon reduction efficiency at each stage, key carbon reduction efficiency stages can be identified, providing data support for key control or priority substitution in practical engineering. The model will also conduct sensitivity analysis, evaluating the stability and robustness of the strategy path through simulations of disturbances to input parameters such as resource scale, emission factors, and cost.

[0117] The pseudocode is as follows:

[0118] {start:

[0119] 1. Read resource and energy data at each stage of the bridge's lifecycle, including:

[0120] - Scale, carbon emission factor, and unit cost of the basic and alternative solutions

[0121] - Convert units and standardize all input data.

[0122] 2. Setting up the RL environment:

[0123] - Each stage is a time step, and each step's state includes basic and alternative attributes.

[0124] - The action is whether to substitute (0 or 1), and the reward is an emission reduction efficiency function.

[0125] - Once a stage is completed, proceed to the next stage, until all stages are finished.

[0126] 3. Initialize the Policy Network (PPO):

[0127] - The input is a state vector, and the output is an action probability distribution or Q-value.

[0128] - Initialize the experience pool and set parameters

[0129] 4. Start training and iterate through N path sets:

[0130] For each training path set:

[0131] - Reset the environment to its initial state

[0132] -For each step:

[0133] - Select an action using the current strategy (whether to replace it).

[0134] - Perform an action to obtain the next state and immediate reward.

[0135] -Save State-Action-Reward-New State Information

[0136] - Total lifecycle reward upon completion of all phases

[0137] - Update policy network parameters using the PPO method

[0138] 5. After training:

[0139] - Fixed policy network

[0140] - Run the test multiple times on the validation set and record the optimal path.

[0141] - Outputs the optimal strategy path, total carbon emissions, total cost, and unit efficiency.

[0142] Finish.}

[0143] In summary, compared to traditional static experience-based decision-making, the core improvement of this invention lies in dynamically weighing the cost-effectiveness of carbon emission reduction at each stage. Traditional methods, by ignoring the differences in carbon emissions and costs at different stages, are prone to resource misallocation. This invention uses reinforcement learning algorithms to calculate the "unit economic carbon reduction efficiency" of each stage in real time, prioritizing the use of alternative solutions at stages with efficiency values ​​exceeding a preset threshold, thereby increasing carbon emission reductions within the same budget. Furthermore, traditional methods lack adaptability to price fluctuations and changes in technical parameters, while this invention verifies the robustness of the strategy through Monte Carlo perturbation simulations, ensuring that key decisions remain stable under perturbation scenarios and avoiding solution failure due to market fluctuations. The technical solution closely aligns with the "dual-carbon" strategic needs and possesses significant engineering application value.

[0144] Please refer to the following: Figure 2 , Figure 2 A bridge lifecycle carbon emission reduction cost-efficiency optimization device 110 based on reinforcement learning, provided for embodiments of the present invention, includes:

[0145] The acquisition module 1101 is used to acquire carbon emission data, cost data and optional scheme parameters of each stage of the bridge's entire life cycle, and construct a dynamic database; based on the dynamic database, a carbon emission reduction cost-efficiency optimization model is constructed, and the unit economic carbon reduction efficiency is defined as the objective function, wherein the unit economic carbon reduction efficiency is the ratio of the carbon emission difference between the alternative scheme and the basic scheme at each stage to the cost difference.

[0146] The optimization module 1102 is used to construct a reinforcement learning environment and initialize a policy network based on the carbon emission reduction cost-efficiency optimization model. The initialized policy network is then iteratively trained using a reinforcement learning algorithm to solve the optimization model, obtaining the optimal policies for each stage output by the policy network. The reinforcement learning environment includes a state space, an action space, a reward function, and state transition rules. The state space uses each stage of the entire lifecycle as a time step and includes the basic and alternative scheme attributes for the current stage. Based on the optimal policies for each stage output by the policy network, the system runs the algorithm a preset number of times in a validation set and records the optimal path, outputting the corresponding total carbon emissions, economic cost, and carbon reduction efficiency analysis results for each stage throughout the entire lifecycle.

[0147] It should be noted that the implementation principle of the aforementioned reinforcement learning-based bridge lifecycle carbon emission reduction cost-efficiency optimization device 110 can refer to the implementation principle of the aforementioned reinforcement learning-based bridge lifecycle carbon emission reduction cost-efficiency optimization method, and will not be repeated here. It should be understood that the division of the various modules in the above device is merely a logical functional division; in actual implementation, they can be fully or partially integrated into a single physical entity, or physically separated. Furthermore, these modules can all be implemented in software through processing element calls; they can all be implemented in hardware; or some modules can be implemented through processing element calls to software, and some modules can be implemented in hardware. For example, the reinforcement learning-based bridge lifecycle carbon emission reduction cost-efficiency optimization device 110 can be a separately established processing element, or it can be integrated into a chip within the aforementioned device. Alternatively, it can be stored as program code in the memory of the aforementioned device, and called and executed by a processing element within the aforementioned device. The implementation of other modules is similar. Furthermore, these modules can be fully or partially integrated together, or implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step or module of the above method can be completed by the integrated logic circuit in the hardware of the processor element or by instructions in the form of software.

[0148] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together to implement a system-on-a-chip (SOC).

[0149] This invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned reinforcement learning-based bridge lifecycle carbon emission reduction cost-efficiency optimization device 110. Figure 4 shows a structural block diagram of the computer device 100 provided in this invention. The computer device 100 includes a reinforcement learning-based bridge lifecycle carbon emission reduction cost-efficiency optimization device 110, a memory 111, a processor 112, and a communication unit 113.

[0150] To enable data transmission or interaction, the memory 111, processor 112, and communication unit 113 are electrically connected to each other directly or indirectly. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The reinforcement learning-based bridge lifecycle carbon reduction cost-efficiency optimization device 110 includes at least one software function module that can be stored in the memory 111 or embedded in the operating system (OS) of the computer device 100 in the form of software or firmware. The processor 112 is used to execute the reinforcement learning-based bridge lifecycle carbon reduction cost-efficiency optimization device 110 stored in the memory 111, such as the software function modules and computer programs included in the reinforcement learning-based bridge lifecycle carbon reduction cost-efficiency optimization device 110.

[0151] This invention provides a readable storage medium, which includes a computer program. When the computer program runs, it controls the computer device where the readable storage medium is located to execute the aforementioned reinforcement learning-based bridge life-cycle carbon emission reduction cost-efficiency optimization device 110.

[0152] For illustrative purposes, the foregoing description has been made with reference to specific embodiments. However, the foregoing illustrative discussions are not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed. Numerous modifications and variations are possible in accordance with the foregoing teachings. These embodiments were chosen and described in order to best illustrate the principles of the present disclosure and its practical application, thereby enabling those skilled in the art to best utilize the disclosure and to employ various embodiments with different modifications to suit a particular intended application.

Claims

1. A method for optimizing the cost-efficiency of bridge carbon emission reduction throughout its entire life cycle based on reinforcement learning, characterized in that, include: Acquire carbon emission data, cost data, and parameters of alternative solutions at each stage of the bridge's entire life cycle, and construct a dynamic database; Based on the dynamic database, a carbon emission reduction cost-efficiency optimization model is constructed, and the unit economic carbon reduction efficiency is defined as the objective function. The unit economic carbon reduction efficiency is the ratio of the carbon emission difference between the alternative scheme and the basic scheme at each stage to the cost difference. Based on the carbon emission reduction cost-efficiency optimization model, a reinforcement learning environment is constructed and a policy network is initialized. The initialized policy network is then used to perform iterative training through a reinforcement learning algorithm to solve the optimization model and obtain the optimal policy for each stage output by the policy network. The reinforcement learning environment includes a state space, an action space, a reward function, and state transition rules. The state space takes each stage of the entire life cycle as a time step and includes the basic and alternative scheme attributes for the current stage. Based on the optimal strategies for each stage output by the strategy network, the system runs a preset number of times in the verification set and records the optimal path, outputting the corresponding total carbon emissions, economic cost, and carbon reduction efficiency analysis results for each stage throughout the entire life cycle. The unit economic carbon reduction efficiency is calculated using the following formula: ； in, Represents the first in the life cycle Unit economic carbon reduction efficiency at each stage Indicates the resource scale of the alternative solution. Indicates the resource scale of the basic scheme. The carbon emission factor representing the alternative solution. Indicates the carbon emission factor of the basic scheme. This indicates the unit cost of the alternative solution. This represents the unit cost of the basic design. This indicates the total carbon emissions of the alternative solutions. This indicates the total carbon emissions of the basic scheme. This represents the total economic cost of the alternative solution. This represents the total economic cost of the basic construction plan; The state space also includes the current stage number, cumulative carbon emissions, cumulative economic cost, and the resource scale, carbon emission factor, and unit cost of the basic and alternative schemes for the current stage in the dynamic database; The action space is a set of discrete actions. ;in To select a basic scheme, To select an alternative, a decision is made based on the output results of the cumulative carbon emissions, cumulative economic cost, and current stage scheme attributes in the state space, using a policy network. The reward function is: ； The state transition rule is as follows: based on the current state and the action selection result, update the cumulative carbon emissions and cumulative economic cost to obtain the state of the next stage. When all stages are completed, the current optimization process ends.

2. The method according to claim 1, characterized in that, The process of iteratively training the initialized policy network using a reinforcement learning algorithm to solve the optimization model and obtain the optimal policy for each stage output by the policy network includes: The initial policy network is iteratively trained using the PPO algorithm to solve the optimization model, and the optimal policy path, total life cycle carbon emissions and economic cost, stage efficiency comparison and sensitivity analysis results output by the policy network are obtained. Based on the optimal strategy path, the total carbon emissions and economic cost over the entire life cycle, the stage efficiency comparison, and the sensitivity analysis results, the optimal strategies for each stage output by the strategy network are obtained.

3. The method according to claim 2, characterized in that, The sensitivity analysis results were obtained through the following methods: At least one parameter in the current stage of the dynamic database, namely the resource scale, carbon emission factor and unit cost, is randomly perturbed, and Monte Carlo simulation is performed to record the total carbon emissions and economic cost of the entire life cycle in each simulation. Calculate the coefficient of variation of total carbon emissions and economic cost for each simulation throughout the entire life cycle. When the coefficient of variation is less than a preset threshold, the optimal strategy path is determined to be stable under parameter fluctuation scenarios.

4. The method according to claim 1, characterized in that, The process of acquiring carbon emission data, cost data, and alternative scheme parameters for each stage of the bridge's entire lifecycle, and constructing a dynamic database, includes: Carbon emission data, cost data, and optional scheme parameters of the bridge at each stage of its entire life cycle are acquired and stored in a dynamic database. The data in the dynamic database is updated in real time based on at least one of the following factors: market fluctuations, policy and standard adjustments, new actual engineering data, or technological iterations, so as to obtain the completed dynamic database.

5. A bridge life-cycle carbon emission reduction cost-efficiency optimization device based on reinforcement learning, characterized in that, include: The acquisition module is used to acquire carbon emission data, cost data, and parameters of alternative solutions at each stage of the bridge's entire life cycle, and to build a dynamic database. Based on the dynamic database, a carbon emission reduction cost-efficiency optimization model is constructed, and the unit economic carbon reduction efficiency is defined as the objective function. The unit economic carbon reduction efficiency is the ratio of the carbon emission difference between the alternative scheme and the basic scheme at each stage to the cost difference. An optimization module is used to construct a reinforcement learning environment and initialize a policy network based on the carbon emission reduction cost-efficiency optimization model. The initialized policy network is then iteratively trained using a reinforcement learning algorithm to solve the optimization model, obtaining the optimal policies for each stage output by the policy network. The reinforcement learning environment includes a state space, an action space, a reward function, and state transition rules. The state space uses each stage of the entire lifecycle as a time step and includes the basic and alternative scheme attributes for the current stage. Based on the optimal policies for each stage output by the policy network, the system runs the algorithm a preset number of times in a validation set, records the optimal path, and outputs the corresponding total carbon emissions, economic cost, and carbon reduction efficiency analysis results for each stage throughout the entire lifecycle. The unit economic carbon reduction efficiency is calculated using the following formula: ； in, Represents the first in the life cycle Unit economic carbon reduction efficiency at each stage Indicates the resource scale of the alternative solution. Indicates the resource scale of the basic scheme. The carbon emission factor representing the alternative solution. Indicates the carbon emission factor of the basic scheme. This indicates the unit cost of the alternative solution. This represents the unit cost of the basic design. This indicates the total carbon emissions of the alternative solutions. This indicates the total carbon emissions of the basic scheme. This represents the total economic cost of the alternative solution. This represents the total economic cost of the basic construction plan; The state space also includes the current stage number, cumulative carbon emissions, cumulative economic cost, and the resource scale, carbon emission factor, and unit cost of the basic and alternative schemes for the current stage in the dynamic database; The action space is a set of discrete actions. ;in To select a basic scheme, To select an alternative, a decision is made based on the output results of the cumulative carbon emissions, cumulative economic cost, and current stage scheme attributes in the state space, using a policy network. The reward function is: ； The state transition rule is as follows: based on the current state and the action selection result, update the cumulative carbon emissions and cumulative economic cost to obtain the state of the next stage. When all stages are completed, the current optimization process ends.

6. A computer device, characterized in that, The computer device includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device performs the method according to any one of claims 1-4.

7. A readable storage medium, characterized in that, The readable storage medium includes a computer program, which, when executed, controls the computer device on which the readable storage medium is located to perform the method described in any one of claims 1-4.

Images