Urban water supply and drainage system full life cycle carbon emission statistics and calculation method and system

Abstract

The invention provides an urban water supply and drainage system full life cycle carbon emission statistics and calculation method and system, and the method comprises the steps: building a facility group based on the number of facilities in a control region, and obtaining the operation state information and carbon emission state information of each facility in the facility group at the current moment; processing the operation state information by using a reinforcement learning model to generate an operation action and obtain a first reward value, processing the carbon emission state information to generate a carbon emission optimization action and obtain a second reward value, updating model parameters based on the state information, the action and the reward, and performing iterative optimization until all facilities are optimized, wherein the first reward value is a negative value of the absolute value of the total carbon emission intensity of the system, and the second reward value is calculated based on facility processing efficiency, energy consumption and carbon emission intensity. The carbon emission intensity of the system can be reduced, and the operation efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent control of urban water systems. More specifically, the present invention relates to a method and system for statistics and measurement of carbon emissions throughout the life cycle of an urban water supply and drainage system. Background Art

[0002] In today's rapidly developing urbanization, urban water supply and drainage systems, as a vital component of urban infrastructure, shoulder the critical task of ensuring the supply of domestic water and sewage treatment for urban residents. However, with the continuous expansion of urban scale and population growth, the operation of water supply and drainage systems faces numerous challenges, particularly in terms of energy consumption and carbon emissions. Traditional water supply and drainage systems, during their design and operation, often focus more on meeting basic water supply and drainage needs, while paying insufficient attention to the system's energy efficiency and carbon emissions. In terms of operational modes, most systems adopt fixed operating strategies and lack the ability to dynamically monitor and optimize real-time operating status and carbon emission data. While this operating mode can ensure stable system operation to a certain extent, it struggles to adapt to the complex and changing urban water demand and environmental requirements, resulting in energy waste and increased carbon emissions.

[0003] In terms of technical principles, existing methods for calculating and estimating carbon emissions from water supply and drainage systems are generally simplistic, primarily based on estimates of facility energy consumption data, and lack a comprehensive consideration of carbon emissions throughout the system's lifecycle. These methods often overlook the impact of factors such as facility operating mode switching, equipment aging, and maintenance on carbon emissions, failing to accurately reflect the system's actual carbon emissions level. Furthermore, existing technologies also have shortcomings in optimizing water supply and drainage system operations to reduce carbon emissions. Traditional optimization methods often rely on empirical rules or simple mathematical models, making it difficult to achieve precise optimization control within complex system operating environments.

[0004] In the process of implementing the embodiments of the present invention, the inventors found that there are at least the following problems or defects in the existing technology: First, the existing water supply and drainage system lacks a systematic statistical and measurement method for carbon emissions throughout its life cycle, and is unable to comprehensively evaluate the system's carbon emission level; second, the traditional operating mode is fixed and cannot be dynamically optimized and adjusted according to real-time operating status and carbon emission data; finally, the existing optimization technology cannot effectively balance the system's operating efficiency and carbon emission control, and it is difficult to meet the requirements of urban sustainable development. Summary of the Invention

[0005] The present invention provides a method and system for statistics and measurement of carbon emissions throughout the life cycle of an urban water supply and drainage system.

[0006] In a first aspect of the present invention, a method for counting and calculating carbon emissions over the entire life cycle of an urban water supply and drainage system is provided, comprising:

[0007] constructing a facility group based on the first facility number of each facility within the water supply and drainage system control area;

[0008] Obtain the current status information of each facility in the facility cluster, including operating status information and carbon emission status information. The operating status information includes the current facility's water supply, drainage, and corresponding operating mode; the carbon emission status information includes the current facility's carbon emission intensity, carbon emissions per unit of processing capacity, energy consumption of the current facility, and carbon emission data of other current facilities;

[0009] The reinforcement learning model is used to process operational and carbon emission information to optimize the operation and carbon emission of water supply and drainage systems, including:

[0010] Step 1: Use the reinforcement learning model to process the current running status information to obtain the corresponding running action;

[0011] The operation action includes maintaining the current operation mode or switching to the next operation mode; executing the operation action and obtaining the corresponding first reward value and operation status information at the first moment; the first reward value is the negative value of the absolute value of the sum of the total carbon emission intensity of all facilities in the current water supply and drainage system, and the total carbon emission intensity includes the difference between the carbon emission intensity of water supply facilities and the carbon emission intensity of drainage facilities; the first moment is the moment after the current moment;

[0012] Step 2: Use the reinforcement learning model to process the current carbon emission status information to obtain the corresponding carbon emission optimization action;

[0013] The carbon emission optimization action includes optimizing the operating parameters of the facility to reduce carbon emissions when the system is in a high carbon emission mode; maintaining the current parameters and further optimizing them when the system is in a low carbon emission mode; executing the carbon emission optimization action and obtaining a corresponding second reward value and carbon emission status information at the first moment; the second reward value is obtained based on the processing efficiency, energy consumption, and carbon emission intensity of the facility;

[0014] Step 3: Update the parameters of the reinforcement learning model based on the current operating state information, the first reward value, the operating action, the carbon emission state information, the second reward value and the carbon emission optimization action, as well as the operating state information and carbon emission state information at the first moment;

[0015] Step 4: Based on the parameters of the reinforcement learning model updated in step 3, repeat steps 1 to 3 until all facilities are optimized.

[0016] Furthermore, the method also includes switching the operating mode of the facility within the switching area based on the operating mode switching demand of the facility: receiving the operating mode switching demand of the facility, and determining whether the facility with the switching demand is an intelligent control facility. If it is an intelligent control facility, the facility with the switching demand sends the target parameters to the target facility, and the target facility and its associated facilities adjust the operating parameters based on the target parameters to form an optimized mode that can be accessed by the facility with the switching demand. The target facility is a facility adjacent to and downstream of the facility with the switching demand after the facility with the switching demand completes the switching.

[0017] Furthermore, based on the first number of facilities in each facility within the water supply and drainage system control area, a facility group is constructed, including: obtaining the first number of facilities in each facility within the water supply and drainage system control area; judging whether the first number of facilities is greater than a first threshold value, if it is greater than the first threshold value, the first first threshold number of facilities in the corresponding area form a facility group; otherwise, all facilities in the corresponding area form a facility group; wherein, the facilities in the facility group maintain a fixed priority operation.

[0018] Furthermore, determining the first reward value includes: determining a second facility number of each facility in the water supply and drainage system based on the state information at the current moment;

[0019] Determining first data for the corresponding facility based on the second facility number and the preset facility capacity of each facility, where the first data represents the difference in carbon emission intensity of all imports and exports of the facility;

[0020] The sum of the first data of all facilities in the water supply and drainage system is obtained to determine a first reward value.

[0021] Further, determining the second reward value includes: obtaining a first time when the facility enters the high carbon emission mode, a second time when the facility exits the high carbon emission mode, a second facility number, carbon emission intensity, energy consumption, and facility processing volume of each facility during the execution of the operation action;

[0022] determining an average operating time of the facility based on the first time, the second time, and the corresponding number of second facilities;

[0023] Obtaining a first value based on the carbon emission intensity of the second facility and the carbon emission intensity of the first facility;

[0024] obtaining a second value based on a processing capacity of the first facility and a processing capacity of a second facility, where the second facility is a downstream facility associated with the first facility;

[0025] Obtaining a third value based on a difference between the first value, the second value, and the facility throughput;

[0026] calculating a larger value of a third value and a zero value during the execution of the operating action, and determining a carbon emission optimization time for the facility based on the larger value and the second facility number;

[0027] Calculate the average carbon emissions of a facility based on its carbon emission intensity and energy consumption;

[0028] A second reward value is determined based on the average operating time of the facility, the carbon emission optimization time, and the average carbon emission amount.

[0029] Furthermore, the loss function expression of the critic in the reinforcement learning model includes:

[0030] y t =r t +γQ(s t+1 ,π(s t+1 |φ′))

[0031] L Critic (θ)=(y t -Q(s t ,A t |θ)) 2

[0032] Among them, y t represents the objective function value, r t represents reward information, γ represents discount factor, Q represents action value function, s t+1 represents the state information at time t+1, π represents the policy function, φ′ represents the parameters of the target policy network, θ′ represents the target Q network parameters, and L Critic represents the critic's loss function, θ represents the online Q-network parameters, and s t Represents the state information at time t, A t Indicates action information.

[0033] Furthermore, the loss function expression of the actor in the reinforcement learning model includes:

[0034]

[0035] Among them, L Actor represents the actor’s loss function, θ represents the online Q-network parameters, Indicates the gradient of θ, Q represents the action value function, s t represents the state information at time t, π represents the policy function, and φ represents the online policy network parameters.

[0036] Furthermore, the expression for determining the first reward value includes:

[0037]

[0038] Among them, R trajectory represents the first reward value, W(l,m) represents the difference in carbon emission intensity, b(l) represents the number of second facilities of the lth water supply facility, bmax (l) represents the capacity of the lth water supply facility, b(m) represents the number of second facilities of the mth drainage facility, b max (l) represents the capacity of the mth drainage facility, l represents the number of the water supply facility, m represents the number of the drainage facility, and i represents the set of facilities.

[0039] Furthermore, a second reward value is determined, and its expression includes:

[0040] R′ trajectory =w1·R n +w2·F n +w3·S n

[0041] Among them, R′ trajectory represents the second reward value, w1 represents the weight corresponding to efficiency, E n represents the average running time, w2 represents the weight corresponding to energy consumption, F n represents the average carbon emissions of the facility, w3 represents the weight corresponding to the carbon emission optimization time, S n represents the carbon emission optimization time, N represents the number of the second facility of each facility during the execution of the operation action, and n represents the number of the facility. Indicates the second time a facility exits high carbon emission mode, Indicates the first time that the facility enters high carbon emission mode, v n (t) represents the carbon emission intensity of facility numbered n at time t, v n-1 (t) represents the carbon emission intensity of facility numbered n-1 at time t, x n-1 (t) represents the processing capacity of the facility numbered n-1 at time t, x n (t) represents the processing capacity of the facility numbered n at time t, l represents the baseline value of the facility processing capacity, j1 represents the power coefficient of carbon emission intensity, j2 represents the power coefficient of energy consumption, represents the constant coefficients corresponding to j1 and j2, a n (t) represents the energy consumption of facility numbered n at time t.

[0042] In a second aspect of the present invention, a system for optimizing carbon emissions of water supply and drainage systems based on reinforcement learning is provided, comprising:

[0043] The mode switching module controls the facility to switch its operating mode within the switching area based on the facility's operating mode switching requirements;

[0044] a facility group construction module, which constructs a facility group based on the first facility number of each facility within the water supply and drainage system control area;

[0045] An acquisition module is used to obtain the current status information of each facility in the facility group. The status information includes operating status information and carbon emission status information. The operating status information includes the water supply and drainage volume of the current facility and the corresponding operating mode. The carbon emission status information includes the carbon emission intensity of the current facility, the carbon emission per unit of processing volume, the energy consumption of the current facility, and the carbon emission data of other current facilities.

[0046] The processing module processes the operation status information and carbon emission status information based on the reinforcement learning model to optimize the operation and carbon emission of the water supply and drainage system, including: step one: using the reinforcement learning model to process the operation status information at the current moment to obtain the corresponding operation action; the operation action includes maintaining the current operation mode or switching to the next operation mode; executing the operation action, and obtaining the corresponding first reward value and the operation status information at the first moment; the first reward value is the negative value of the absolute value of the sum of the total carbon emission intensity of all facilities in the current water supply and drainage system, and the total carbon emission intensity includes the difference between the carbon emission intensity of the water supply facilities and the carbon emission intensity of the drainage facilities; the first moment is the next moment of the current moment; step two: using the reinforcement learning model to process the carbon emission status information at the current moment to obtain the corresponding carbon emission release optimization action; the carbon emission optimization action includes optimizing the operating parameters of the facility to reduce carbon emissions when the system is in high carbon emission mode; when the system is in low carbon emission mode, maintaining the current parameters and further optimizing; executing the carbon emission optimization action, and obtaining the corresponding second reward value and the carbon emission status information at the first moment; the second reward value is obtained based on the processing efficiency, energy consumption and carbon emission intensity of the facility; step three: based on the operating status information, the first reward value, the operating action, the carbon emission status information, the second reward value and the carbon emission optimization action at the current moment, as well as the operating status information and carbon emission status information at the first moment, update the parameters of the reinforcement learning model; step four: based on the parameters of the reinforcement learning model updated in step three, repeat steps one to three until all facilities are optimized.

[0047] The above-described embodiments of the present invention have at least the following beneficial effects: The present invention can dynamically optimize the operating mode and carbon emission strategy of a water supply and drainage system based on a reinforcement learning model. By processing the operational status and carbon emission status information of a cluster of facilities in real time, it automatically generates optimal operating actions and carbon emission optimization actions, achieving precise control of system-level carbon emissions. This method comprehensively considers the differences in carbon emission intensity between water supply and drainage facilities. Through a dual feedback mechanism of a first reward value and a second reward value, it continuously optimizes system operating parameters, effectively reducing overall carbon emission intensity while maintaining system processing efficiency.

[0048] This technology can intelligently identify the system's high-carbon emission modes and automatically adjust operating parameters, further optimizing system performance in low-carbon emission modes. Through collaborative control and priority management of a cluster of facilities, the system can maintain stable operation while reducing carbon emissions. Furthermore, this method can automatically handle the need to switch operating modes between facilities. By transferring target parameters and coordinating adjustments across related facilities, seamless switching and global optimization are achieved, ultimately achieving low-carbon and efficient operation of the water supply and drainage system throughout its lifecycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily apparent by reading the following detailed description with reference to the accompanying drawings, in which several embodiments of the present invention are shown by way of example and not limitation, in which:

[0050] Figure 1 A flow chart of a method for calculating and estimating carbon emissions from the entire life cycle of an urban water supply and drainage system provided by one embodiment of the present invention;

[0051] Figure 2 A schematic diagram of the structure of a carbon emission optimization system for a water supply and drainage system based on reinforcement learning provided by one embodiment of the present invention;

[0052] Figure 3 The figure schematically shows the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided solely to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. Rather, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0054] Those skilled in the art will appreciate that the embodiments of the present invention may be implemented as a system, apparatus, device, method, or computer program product. Therefore, the present invention may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or in a combination of hardware and software.

[0055] It should be noted that any number of elements in the drawings is for illustration only and not for limitation, and any naming is only for distinction and does not have any limiting meaning.

[0056] Reference below Figure 1 , Figure 1This is a flow chart of a method for calculating and counting carbon emissions from the entire life cycle of an urban water supply and drainage system according to an embodiment of the present invention. Figure 1 As shown in the figure, a method for counting and calculating carbon emissions of urban water supply and drainage systems throughout their life cycle includes:

[0057] constructing a facility group based on the first facility number of each facility within the water supply and drainage system control area;

[0058] Obtain the current status information of each facility in the facility cluster, including operating status information and carbon emission status information. The operating status information includes the current facility's water supply, drainage, and corresponding operating mode; the carbon emission status information includes the current facility's carbon emission intensity, carbon emissions per unit of processing capacity, energy consumption of the current facility, and carbon emission data of other current facilities;

[0059] The reinforcement learning model is used to process operational and carbon emission information to optimize the operation and carbon emission of water supply and drainage systems, including:

[0060] Step 1: Use the reinforcement learning model to process the current running status information to obtain the corresponding running action;

[0061] The operation action includes maintaining the current operation mode or switching to the next operation mode; executing the operation action and obtaining the corresponding first reward value and operation status information at the first moment; the first reward value is the negative value of the absolute value of the sum of the total carbon emission intensity of all facilities in the current water supply and drainage system, and the total carbon emission intensity includes the difference between the carbon emission intensity of water supply facilities and the carbon emission intensity of drainage facilities; the first moment is the moment after the current moment;

[0062] Step 2: Use the reinforcement learning model to process the current carbon emission status information to obtain the corresponding carbon emission optimization action;

[0063] The carbon emission optimization action includes optimizing the operating parameters of the facility to reduce carbon emissions when the system is in a high carbon emission mode; maintaining the current parameters and further optimizing them when the system is in a low carbon emission mode; executing the carbon emission optimization action and obtaining a corresponding second reward value and carbon emission status information at the first moment; the second reward value is obtained based on the processing efficiency, energy consumption, and carbon emission intensity of the facility;

[0064] Step 3: Update the parameters of the reinforcement learning model based on the current operating state information, the first reward value, the operating action, the carbon emission state information, the second reward value and the carbon emission optimization action, as well as the operating state information and carbon emission state information at the first moment;

[0065] Step 4: Based on the parameters of the reinforcement learning model updated in step 3, repeat steps 1 to 3 until all facilities are optimized.

[0066] It should be noted that the present invention proposes a method for statistics and measurement of carbon emissions throughout the life cycle of an urban water supply and drainage system. The core of the method is to optimize the operating status and carbon emission status of the water supply and drainage system through a reinforcement learning model. The water supply and drainage system refers to the infrastructure network in the city used to supply domestic water and treat sewage, including water supply facilities, drainage facilities and related pipe network systems. A facility cluster refers to a collection of facilities divided according to the number of facilities within the system control area, which is used for centralized management and optimization. Operating status information refers to data such as the water supply, drainage volume and operating mode of the facility at the current moment, which reflects the operating status of the facility; carbon emission status information includes the carbon emission intensity of the facility, carbon emissions per unit of processing volume, energy consumption and carbon emission data of other related facilities, which are used to evaluate the carbon emission level of the facility. The reinforcement learning model is a learning method based on the interaction between the intelligent agent and the environment, which optimizes the decision-making process through continuous trial and error and reward feedback.

[0067] Specifically, facility clusters are constructed based on the number of first facilities within the water supply and drainage system's control area. The number of first facilities refers to the actual number of facilities within the system's control area. The rules for constructing facility clusters are as follows: if the number of first facilities is greater than a preset first threshold, the first threshold number of facilities are selected to form a facility cluster; if the number of first facilities is less than or equal to the first threshold, all facilities within the area are grouped together. Facilities within a facility cluster maintain a fixed operating priority. Priority refers to the importance of facilities during operation and is used to determine the operating sequence and resource allocation of facilities. The water supply and drainage volume in the operating status information refer to the amount of water supplied and sewage discharged by the facility per unit time. The operating mode refers to the facility's operating strategy under different operating conditions, such as normal operation mode, energy-saving mode, or peak operation mode. The carbon emission intensity in the carbon emission status information refers to the carbon dioxide emissions generated by the facility per unit time. The carbon emissions per unit treatment volume refers to the carbon emissions generated when treating a unit of water. Energy consumption refers to the energy consumed by the facility during operation. The carbon emission data for other facilities refers to the carbon emissions of other facilities associated with the current facility.

[0068] Preferably, the construction of the reinforcement learning model includes the following steps: first, define the state space, which is composed of the operating status information and carbon emission status information of the facility, such as the water supply, drainage, carbon emission intensity and other parameters of the facility; second, define the action space, which includes operating actions and carbon emission optimization actions. The operating action refers to maintaining the current operating mode or switching to the next operating mode. The carbon emission optimization action refers to optimizing the facility operating parameters in the high carbon emission mode to reduce carbon emissions, and maintaining the current parameters and further optimizing them in the low carbon emission mode; then, design a reward function, which is used to evaluate the performance of the model in each decision step. The first reward value is the negative value of the absolute value of the sum of the total carbon emission intensities of all facilities in the water supply and drainage system, and the second reward value is based on the processing efficiency, energy consumption and carbon emission intensity of the facility; finally, update the model parameters through continuous trial and error and reward feedback to optimize the operation and carbon emissions of the water supply and drainage system. During the data processing process, after obtaining the status information of each facility in the facility cluster, this information is input into the reinforcement learning model. The model outputs the corresponding operating actions and carbon emission optimization actions based on the current status information. After executing these actions, the reward value is obtained, and the model parameters are updated based on the reward value and the new status information, thereby achieving optimized operation of the system.

[0069] In some embodiments, the method also includes switching the operating mode of the facility within the switching area based on the operating mode switching requirement of the facility: receiving the operating mode switching requirement of the facility, and determining whether the facility with the switching requirement is an intelligent control facility. If it is an intelligent control facility, the facility with the switching requirement sends the target parameters to the target facility, and the target facility and its associated facilities adjust the operating parameters based on the target parameters to form an optimized mode that can be accessed by the facility with the switching requirement. The target facility is a facility adjacent to and downstream of the facility with the switching requirement after the facility with the switching requirement completes the switching.

[0070] It should be noted that the operation of switching the operating mode of the facilities within the switching area based on the operating mode switching requirements of the facilities involved in the present invention is to more flexibly respond to the operating requirements of the water supply and drainage system under different working conditions. Operating mode switching refers to adjusting the facilities from one operating mode to another according to the actual water demand or the operating status of the system. For example, switching from normal operating mode to energy-saving mode or peak operating mode, etc. Intelligent control facilities refer to facilities that have the ability to automatically sense, analyze and adjust operating parameters, and they can automatically adjust their own operating status by receiving target parameters. Target parameters refer to the operating parameter settings that need to be achieved in order to achieve operating mode switching, such as parameters such as flow, pressure or energy consumption. Target facilities refer to facilities that receive target parameters and make adjustments during the operating mode switching process, and associated facilities refer to other facilities that interact with the operating status of the target facilities. In this way, coordinated operation between facilities can be achieved, and the operating efficiency and carbon emission control of the entire system can be optimized.

[0071] Specifically, the need to switch operating modes refers to the system determining whether the current operating mode needs to be adjusted based on real-time monitoring data or preset rules. For example, during peak water usage periods, the system may need to switch some facilities from energy-saving mode to peak operating mode to meet water demand. Intelligent control facilities are typically equipped with sensors and controllers that can receive and process target parameters. The setting of target parameters can be adjusted according to actual needs. For example, during peak hours, target parameters may include increasing the water supply flow rate or increasing the power of the drainage pump. The target facility adjusts its own operating parameters based on the received target parameters, such as adjusting the pump speed or valve opening. Associated facilities then adjust their own operating status accordingly based on the adjustments made by the target facility to ensure stable operation of the entire system. For example, when the target facility increases the water supply flow rate, downstream drainage facilities may need to adjust their drainage capacity accordingly to avoid excessive system pressure or poor drainage.

[0072] Preferably, the specific implementation process of operating mode switching can be further refined into the following steps: First, the system monitoring module collects the facility's operating status information in real time, including parameters such as water supply, drainage, and energy consumption. When a need to switch operating modes is detected, the system determines whether the facility is an intelligent control facility. If it is an intelligent control facility, the system generates target parameters and sends them to the target facility. After receiving the target parameters, the target facility adjusts the operating parameters based on its own operating status and preset control strategy to achieve mode switching. At the same time, the target facility will feedback the adjusted operating status information to the system, and the system will further optimize the target parameter settings based on this feedback information. Based on the adjustments made by the target facility and using preset collaborative operation rules, the associated facilities will automatically adjust their own operating parameters to ensure the operational stability of the entire system and the effectiveness of carbon emission control. During this process, the system will continuously monitor the facility's operating status and carbon emission data to evaluate the effectiveness of the mode switch and make further optimization adjustments as needed.

[0073] In some embodiments, a facility group is constructed based on the first facility number among each facility within the water supply and drainage system control area, including: obtaining the first facility number among each facility within the water supply and drainage system control area; judging whether the first facility number is greater than a first threshold value, if it is greater than the first threshold value, the first first threshold number of facilities in the corresponding area form a facility group; otherwise, all facilities in the corresponding area form a facility group; wherein the facilities in the facility group maintain a fixed priority operation.

[0074] It should be noted that the purpose of constructing a facility group based on the first number of facilities in each facility within the control area of the water supply and drainage system mentioned in the present invention is to better classify, manage and optimize the control of the facilities within the system. The construction of a facility group is to divide the facilities into different management units based on the number and distribution of facilities, so as to more efficiently carry out operation control and carbon emission statistics. The first number of facilities refers to the total number of facilities that actually exist within the control area of the water supply and drainage system, which is the basis for determining the division of facility groups. The first threshold is a preset parameter used to determine whether the number of facilities is large enough to be divided into multiple facility groups. If the number of facilities is large, some facilities can be divided into a facility group for more refined management; if the number of facilities is small, all facilities can be divided into a facility group to simplify the management process.

[0075] Specifically, the first number of facilities refers to the number of facilities actually existing in the control area of the water supply and drainage system, including water supply facilities, drainage facilities and other related auxiliary facilities. The first threshold is a preset value used to determine whether the facilities need to be divided into multiple facility groups. For example, if the first threshold is set to 10, then when the number of facilities is greater than 10, the system will divide the first 10 facilities into a facility group; if the number of facilities is less than or equal to 10, all facilities will be divided into one facility group. The facilities in the facility group maintain a fixed priority operation. The priority refers to the order of importance of the facilities during operation. For example, water supply facilities may take precedence over drainage facilities to ensure the stability of water supply. In this way, it can be ensured that the facilities operate according to the preset priority during operation, thereby improving the operating efficiency and stability of the system.

[0076] Preferably, the specific implementation process of constructing a facility group can be further refined into the following steps: First, the system obtains the actual number of facilities in the control area of the water supply and drainage system through sensors or monitoring equipment, that is, the first number of facilities. Then, the system compares the first number of facilities with the preset first threshold. If the first number of facilities is greater than the first threshold, the system will select the first threshold number of facilities to form a facility group according to the preset rules; if the first number of facilities is less than or equal to the first threshold, all facilities will be grouped into one facility group. After the facility group division is completed, the system will assign a unique identifier to each facility group and set the corresponding priority according to the type and function of the facility. For example, water supply facilities may be given a higher priority to ensure that water supply needs are met first when resources are limited. In addition, the system will dynamically adjust the facility group based on the operating status and carbon emission data of the facility to optimize the overall operating efficiency and carbon emission control effect of the system.

[0077] In some embodiments, determining the first reward value includes: determining a second facility number of each facility in the water supply and drainage system based on state information at a current moment;

[0078] Determining first data for the corresponding facility based on the second facility number and the preset facility capacity of each facility, where the first data represents the difference in carbon emission intensity of all imports and exports of the facility;

[0079] The sum of the first data of all facilities in the water supply and drainage system is obtained to determine a first reward value.

[0080] It should be noted that the method for determining the first reward value mentioned in the present invention is achieved by conducting a comprehensive analysis of the status information of each facility in the water supply and drainage system. The first reward value is an important parameter in the reinforcement learning model, which is used to evaluate the operating status and carbon emission level of the system and guide the optimization direction of the model. The second number of facilities refers to the number of sub-units actually involved in the operation of the facility at a certain moment, such as the number of water pumps, valves, etc. The preset facility capacity refers to the maximum processing capacity that the facility can achieve under design or operating conditions, such as the maximum water supply or maximum drainage. The first data represents the difference in carbon emission intensity of all imports and exports of the facility, reflecting the changes in carbon emissions during the operation of the facility. By calculating the sum of the first data of all facilities, the first reward value can be obtained, thereby providing a basis for the optimization of the system.

[0081] Specifically, the second number of facilities refers to the number of sub-units actually operating inside each facility in the water supply and drainage system. For example, a water supply facility may contain multiple sub-units such as water pumps and filters. The number of these sub-units is the second number of facilities. The preset facility capacity refers to the maximum processing capacity that the facility can achieve under design or operating conditions, such as the maximum water supply of a water supply facility or the maximum drainage of a drainage facility. The first data is the difference obtained by measuring and calculating the carbon emission intensity of all imports and exports of each facility, which reflects the changes in carbon emissions of the facility during operation. For example, if the import carbon emission intensity of a facility is 10 units and the export carbon emission intensity is 8 units, then the first data of the facility is 2 units. By adding the first data of all facilities, the total carbon emission intensity change value of the system can be obtained, and then the first reward value can be determined.

[0082] Preferably, the specific implementation process of determining the first reward value can be further refined into the following steps: First, the system obtains the second facility number of each facility through sensors or monitoring equipment, that is, the number of sub-units actually in operation. Then, based on the preset facility capacity, the first data of each facility is calculated, that is, the difference between the import and export carbon emission intensities. For example, for a water supply facility, the first data can be calculated by measuring the carbon emission intensity of the import and export. Then, the system adds up the first data of all facilities to obtain the total carbon emission intensity change value of the system. Finally, the first reward value is determined based on the total carbon emission intensity change value. For example, if the total carbon emission intensity change value is positive, it means that the carbon emissions of the system have increased. The first reward value can be set to a negative value to encourage the system to optimize the operating mode to reduce carbon emissions; if the total carbon emission intensity change value is negative, it means that the carbon emissions of the system have decreased. The first reward value can be set to a positive value to reward the optimized operation of the system.

[0083] In some embodiments, determining the second reward value includes: obtaining a first time when the facility enters a high carbon emission mode, a second time when the facility exits the high carbon emission mode, a second facility number, carbon emission intensity, energy consumption, and facility processing volume of each facility during the execution of the operation action;

[0084] determining an average operating time of the facility based on the first time, the second time, and the corresponding number of second facilities;

[0085] Obtaining a first value based on the carbon emission intensity of the second facility and the carbon emission intensity of the first facility;

[0086] obtaining a second value based on a processing capacity of the first facility and a processing capacity of a second facility, where the second facility is a downstream facility associated with the first facility;

[0087] Obtaining a third value based on a difference between the first value, the second value, and the facility throughput;

[0088] calculating a larger value of a third value and a zero value during the execution of the operating action, and determining a carbon emission optimization time for the facility based on the larger value and the second facility number;

[0089] Calculate the average carbon emissions of a facility based on its carbon emission intensity and energy consumption;

[0090] A second reward value is determined based on the average operating time of the facility, the carbon emission optimization time, and the average carbon emission amount.

[0091] It should be noted that the method for determining the second reward value mentioned in this invention is based on a comprehensive evaluation of various parameters and status information during the operation of the facility. The second reward value is used to measure the performance of the facility in optimizing carbon emissions and is an important feedback signal in the reinforcement learning model optimization process. It calculates a value that reflects the optimization effect of the facility by analyzing data such as the time when the facility enters and exits high-carbon emission mode, operating parameters, carbon emission intensity, energy consumption, and processing volume. This method can more comprehensively evaluate the carbon emission optimization effect of the facility during operation, thereby providing more accurate guidance for system optimization.

[0092] Specifically, the calculation of the second reward value involves multiple parameters and concepts. The first time refers to the time point when the facility enters the high-carbon emission mode, and the second time refers to the time point when the facility exits the high-carbon emission mode. The second number of facilities refers to the number of sub-units actually operating inside the facility during the execution of the operation action, such as the number of water pumps, filters, etc. Carbon emission intensity refers to the amount of carbon dioxide emissions generated by the facility per unit time, energy consumption refers to the amount of energy consumed by the facility during operation, and processing volume refers to the amount of water processed by the facility per unit time. The first and second values are intermediate results obtained by calculating the changes in carbon emission intensity and processing volume, and are used to evaluate the performance of the facility during the optimization process. Carbon emission optimization time refers to the time when the facility actually operates effectively during the optimization process, which is used to measure the sustainability of the optimization effect. Through the comprehensive calculation of these parameters, the second reward value can be obtained, thereby providing feedback for the optimization of the system.

[0093] Preferably, the specific implementation process for determining the second reward value can be further refined into the following steps: First, the system uses sensors or monitoring equipment to obtain the first time a facility enters high-carbon emission mode and the second time it exits high-carbon emission mode. Then, the average operating time of the facility during the execution of the operation action is calculated, i.e., the difference between the second time and the first time. Next, a first value and a second value are calculated based on the changes in the facility's carbon emission intensity and processing volume. For example, the first value can be obtained by comparing the carbon emission intensity before and after optimization, and the second value can be obtained by comparing the processing volume before and after optimization. Then, based on the first and second values and the difference in the facility's processing volume, a third value is calculated to assess the effectiveness of the optimization. Finally, based on the larger of the third value and zero, combined with the second number of facilities, the facility's carbon emission optimization time is determined. The second reward value is then calculated based on the facility's average carbon emission volume and average operating time. For example, if the carbon emission intensity after optimization is significantly reduced and the processing volume remains stable or increases, the second reward value can be set to a positive value to reward the facility's optimization effect. If the optimization effect is not significant, the second reward value can be set to a smaller positive or negative value.

[0094] In some embodiments, the loss function expression of the critic in the reinforcement learning model includes:

[0095] y t =r t +γQ(s t+1 ,π(s t+1 |φ′))

[0096] L Critic (θ)=(y t -Q(s t ,A t |θ)) 2

[0097] Among them, y t represents the objective function value, r t represents reward information, γ represents discount factor, Q represents action value function, s t+1 represents the state information at time t+1, π represents the policy function, φ′ represents the parameters of the target policy network, θ′ represents the target Q network parameters, and L Critic represents the critic's loss function, θ represents the online Q-network parameters, and s t Represents the state information at time t, A t Indicates action information.

[0098] It should be noted that the loss function expression of the critic in the reinforcement learning model mentioned in the present invention is an important tool for measuring the difference between the predicted value and the actual value of the model during the optimization process. The critic is a component in reinforcement learning that is responsible for evaluating the quality of the current strategy and providing feedback signals to the actor to optimize the strategy. The loss function is calculated by comparing the difference between the objective function value and the actual reward value. Its purpose is to optimize the parameters of the model by minimizing this difference. The discount factor is used to balance the importance of current rewards and future rewards, and the action value function (Q function) is used to evaluate the expected return of taking a certain action in a given state. The target policy network and target Q network parameters are auxiliary network parameters used to stabilize the training process. They are updated regularly to maintain a certain difference from the main network, thereby improving the stability and convergence of the model.

[0099] Specifically, the critic's loss function expression involves several key parameters and concepts. The objective function value is the optimal value that the model expects to achieve, reflecting the performance of the system under optimal operating conditions. Reward information is the feedback the model receives at each decision step, used to measure the effectiveness of the current action. The discount factor is a value between 0 and 1 that adjusts the weight of future rewards in the current decision, with a larger discount factor indicating greater emphasis on future rewards. The action-value function (Q-function) is a key concept in reinforcement learning, representing the expected reward of taking an action in a given state. State information refers to the operating state of the system at a given moment, including facility operating parameters and carbon emission data. Action information refers to the action selected by the model in the current state, such as adjusting the facility's operating mode or optimizing parameters. The critic's loss function measures the model's prediction error by calculating the squared difference between the predicted value and the target value, thereby providing direction for model optimization.

[0100] Preferably, the specific process of constructing the critic loss function can be further refined into the following steps: First, define the objective function value, which is usually the optimal carbon emission intensity or other performance indicator that the system expects to achieve. Then, calculate the target value based on the predicted action value function (Q value) of the model in the current state and the actual reward value. The target value can be calculated by multiplying the actual reward value by the sum of the maximum Q value in the state at the next moment with a discount factor. Next, calculate the critic loss function, which is the square difference between the predicted Q value and the target value. For example, if the predicted Q value is 10 and the target value is 8, the loss function value is (10-8)2=4. Finally, update the parameters of the critic network by minimizing this loss function. In practical applications, gradient descent or other optimization algorithms can be used to adjust the network parameters to gradually reduce the difference between the predicted value and the target value. In addition, to improve the stability and convergence speed of the model, the parameters of the target policy network and the target Q network can be updated regularly so that they gradually approach the parameters of the main network, but maintain a certain difference to avoid oscillation during training.

[0101] In some embodiments, the loss function expression for an actor in a reinforcement learning model includes:

[0102]

[0103] Among them, L Actor represents the actor’s loss function, θ represents the online Q-network parameters, Indicates the gradient of θ, Q represents the action value function, s t represents the state information at time t, π represents the policy function, and φ represents the online policy network parameters.

[0104] It should be noted that the loss function expression for the actor in the reinforcement learning model mentioned in this invention is a key tool for optimizing the actor network. The actor is a component in reinforcement learning responsible for selecting the optimal action based on the current state. The loss function guides the parameter updates of the actor network by calculating the gradient of the action-value function, thereby optimizing the strategy, enabling the system to select the most optimal action in a given state to reduce carbon emissions. This loss function design enables the actor network to dynamically adjust its strategy based on feedback signals provided by the critic network, thereby optimizing system operation.

[0105] Specifically, the actor's loss function expression involves multiple key parameters and concepts. The actor's loss function is defined by calculating the negative gradient of the action-value function (Q-function), with the goal of maximizing the expected reward. The online Q-network parameters refer to the parameters of the Q-network currently being trained, which are used to evaluate the expected reward of taking a certain action in a given state. The policy function is the core part of the actor network. It outputs an action based on the current state information. This action determines the system's operating mode or parameter adjustment. The parameters of the policy function are the weights and biases of the actor network. These parameters are continuously updated during training to optimize the policy. The gradient is the derivative of the loss function with respect to the policy function parameters, which indicates the direction of the parameter update. By calculating the gradient and updating the parameters of the policy function, the actor network can gradually learn a better policy.

[0106] Preferably, the specific process of constructing the actor loss function can be further refined into the following steps: First, define the action-value function (Q-function), which represents the expected reward of taking a certain action in a given state. This function is typically provided by a critic network and is used to evaluate the quality of the current policy. Next, calculate the gradient of the policy function—the derivative of the loss function with respect to the policy function parameters. This gradient, which can be calculated using the backpropagation algorithm, indicates the direction of parameter updates. Next, update the policy function parameters based on the gradient. For example, if the gradient is positive, increase the parameter value; if the gradient is negative, decrease the parameter value. Finally, through continuous iteration of this process, the actor network can gradually learn a more optimal policy, enabling the system to more effectively reduce carbon emissions during operation. In practical applications, a learning rate can be set to control the step size of parameter updates to avoid overly fast or slow convergence. In addition, to improve model stability and convergence speed, momentum optimization algorithms or other advanced optimization methods can be used to adjust the policy function parameters.

[0107] In some embodiments, the expression for determining the first reward value includes:

[0108]

[0109] Among them, R trajectory represents the first reward value, W(l,m) represents the difference in carbon emission intensity, b(l) represents the number of second facilities of the lth water supply facility, b max (l) represents the capacity of the lth water supply facility, b(m) represents the number of second facilities of the mth drainage facility, b max (l) represents the capacity of the mth drainage facility, l represents the number of the water supply facility, m represents the number of the drainage facility, and i represents the set of facilities.

[0110] It should be noted that the method for determining the first reward value mentioned in the present invention is to normalize the difference in carbon emission intensity of each facility in the water supply and drainage system and calculate it in combination with the capacity information of the facility. This method can more intuitively reflect the changes in carbon emissions of the facility during operation, and avoid calculation deviations caused by differences in facility scale through normalization. The numbers of water supply facilities and drainage facilities are used to identify different facilities in the system, while the facility set refers to the set of all related facilities in the system for unified management and calculation. In this way, the overall carbon emission performance of the system can be evaluated more accurately, and a basis for optimization strategies can be provided.

[0111] Specifically, the calculation of the first reward value involves multiple key parameters and concepts. The second facility number of the water supply facility refers to the number of sub-units actually operating in the water supply facility at a certain moment, such as the number of water pumps, filters, etc. The second facility number of the drainage facility refers to the number of sub-units actually operating in the drainage facility. The capacity of water supply facilities and the capacity of drainage facilities refer to the maximum processing capacity that the facilities can achieve under design or operating conditions, such as the maximum water supply or maximum drainage. The difference in carbon emission intensity refers to the difference in carbon emission intensity between the import and export of the facility, reflecting the changes in carbon emissions during the operation of the facility. The water supply facility number and the drainage facility number are unique numbers used to identify different facilities in the system, which facilitates separate calculation and management of each facility. The facility set refers to the set of all relevant facilities in the system, which is used for unified management and calculation. Through the comprehensive calculation of these parameters, the first reward value can be obtained, thereby providing feedback for the optimization of the system.

[0112] Preferably, the specific implementation process of determining the first reward value can be further refined into the following steps: First, the system obtains the second facility number of each water supply facility and drainage facility through sensors or monitoring equipment, that is, the number of sub-units actually in operation. Then, based on the preset facility capacity, the difference in carbon emission intensity of each facility is calculated. For example, for a water supply facility, the difference in carbon emission intensity can be calculated by measuring the carbon emission intensity of the import and export. Next, the difference in carbon emission intensity of each facility is normalized, that is, it is divided by the maximum capacity of the corresponding facility. For example, if the difference in carbon emission intensity of the water supply facility is 10 units and the maximum capacity is 100 units, the normalized value is 0.1. Finally, the normalized values of all facilities are added together to obtain the first reward value. For example, if there are five water supply facilities and three drainage facilities in the system, and the normalized values of each facility are 0.1, 0.2, 0.15, 0.05, and 0.1 (water supply facilities) and 0.2, 0.15, and 0.1 (drainage facilities), then the first reward value is 0.1 + 0.2 + 0.15 + 0.05 + 0.1 + 0.2 + 0.15 + 0.1 = 1.0. In this way, the overall carbon emission performance of the system can be more accurately evaluated and provide a basis for optimization strategies.

[0113] In some embodiments, a second reward value is determined, and its expression includes:

[0114] R′ trajectory =w1·R n +w2·F n +w3·S n

[0115] Among them, R′ trajectory represents the second reward value, w1 represents the weight corresponding to efficiency, E n represents the average running time, w2 represents the weight corresponding to energy consumption, F n represents the average carbon emissions of the facility, w3 represents the weight corresponding to the carbon emission optimization time, S n represents the carbon emission optimization time, N represents the second facility number of each facility during the execution of the operation action, and n represents the number of the facility.

[0116] It should be noted that the method for determining the second reward value mentioned in this invention is based on a weighted calculation of multiple key parameters during facility operation. These parameters include the facility's average operating time, average carbon emissions, and carbon emission optimization time. By assigning weights to each parameter, the reward value calculation can be adjusted based on the contribution of each parameter to system optimization. This method can more comprehensively reflect the optimization effect of the facility during operation, thereby providing more accurate feedback signals to the reinforcement learning model to guide the system's optimization direction.

[0117] Specifically, the calculation of the second reward value involves multiple key parameters and concepts. The weight corresponding to efficiency refers to the proportion of facility operating efficiency when calculating the reward value. The average operating time refers to the actual operating time of the facility during the execution of the operating action, which reflects the operational stability of the facility. The weight corresponding to energy consumption refers to the proportion of facility energy consumption when calculating the reward value. The average carbon emissions of the facility refer to the total carbon emissions per unit time during the operation period, reflecting the carbon emission level of the facility. The weight corresponding to the carbon emission optimization time refers to the proportion of carbon emission optimization time when calculating the reward value. The carbon emission optimization time refers to the time that the facility actually operates effectively during the optimization process, reflecting the actual effect of the optimization measures. By weighted calculation of these parameters, a second reward value that comprehensively reflects the optimization effect of the facility can be obtained.

[0118] Preferably, the specific implementation process of determining the second reward value can be further refined into the following steps: First, the system obtains data such as the operating time, energy consumption and carbon emissions of the facility during the execution of the operation action through sensors or monitoring equipment. Then, based on these data, the average operating time, average carbon emissions and carbon emission optimization time of the facility are calculated. For example, the average operating time can be obtained by dividing the total operating time by the number of operations; the average carbon emissions can be obtained by dividing the total carbon emissions by the operating time; the carbon emission optimization time can be obtained by analyzing the carbon emission data before and after the implementation of the optimization measures. Then, set a weight for each parameter. For example, the weight corresponding to efficiency can be set to 0.4, the weight corresponding to energy consumption can be set to 0.3, and the weight corresponding to carbon emission optimization time can be set to 0.3. Finally, multiply the values of these parameters by the corresponding weights and add them together to obtain the second reward value. For example, if the average operating time of a facility is 10 hours, the average carbon emissions are 50 units, and the carbon emissions optimization time is 5 hours, and the corresponding weights are 0.4, 0.3, and 0.3, respectively, then the second reward value is 10 × 0.4 + 50 × 0.3 + 5 × 0.3 = 4 + 15 + 1.5 = 20.5. In this way, the optimization effect of the facility during operation can be more accurately evaluated and feedback can be provided for system optimization.

[0119] The above-mentioned embodiments of the present invention have the following beneficial effects: the technical solution can realize the intelligent operation control and carbon emission optimization of the urban water supply and drainage system based on the reinforcement learning algorithm. By constructing a dynamic facility group and collecting the operating parameters and carbon emission data of each facility in real time, the system can automatically generate the optimal operation strategy, including operations such as maintaining or switching the operating mode and adjusting the facility operating parameters. The dual reward mechanism can accurately evaluate the carbon emission status of the system, where the first reward value can reflect the difference in carbon emission intensity between water supply and drainage facilities, and the second reward value can comprehensively consider key indicators such as processing efficiency and energy consumption, thereby ensuring that the system maintains efficient operation while reducing carbon emissions.

[0120] This method can intelligently identify high-carbon emission states and automatically trigger optimization procedures. Seamless switching can be achieved through collaborative control and parameter transfer between facilities. The system can automatically divide and prioritize facility clusters based on preset thresholds, and accurately evaluate optimization results by calculating parameters such as the difference in carbon emission intensity between facility imports and exports and the difference in treatment volume. Introducing the actor-critic model from reinforcement learning allows for continuous optimization of control strategies. The critic network evaluates state values, and the actor network generates optimal control instructions, ultimately achieving low-carbon, stable, and efficient operation of the water supply and drainage system throughout its lifecycle.

[0121] like Figure 2 As shown, in some embodiments, a carbon emission optimization system for a water supply and drainage system based on reinforcement learning is provided, the system comprising:

[0122] Mode switching module 201 controls the facility to switch its operating mode within the switching area based on the facility's operating mode switching requirements;

[0123] The facility group construction module 202 constructs a facility group based on the first facility number of each facility in the water supply and drainage system control area;

[0124] Acquisition module 203 acquires the current status information of each facility in the facility group, the status information including operation status information and carbon emission status information. The operation status information includes the water supply and discharge volume of the current facility and the corresponding operation mode; the carbon emission status information includes the carbon emission intensity of the current facility, the carbon emission per unit of processing volume, the energy consumption of the current facility, and the carbon emission data of other facilities.

[0125] The processing module 204 processes the operation status information and carbon emission status information based on the reinforcement learning model to optimize the operation and carbon emission of the water supply and drainage system, including: step 1: using the reinforcement learning model to process the operation status information at the current moment to obtain the corresponding operation action; the operation action includes maintaining the current operation mode or switching to the next operation mode; executing the operation action, and obtaining the corresponding first reward value and the operation status information at the first moment; the first reward value is the negative value of the absolute value of the sum of the total carbon emission intensity of all facilities in the current water supply and drainage system, and the total carbon emission intensity includes the difference between the carbon emission intensity of the water supply facilities and the carbon emission intensity of the drainage facilities; the first moment is the next moment of the current moment; step 2: using the reinforcement learning model to process the carbon emission status information at the current moment to obtain the corresponding Carbon emission optimization action; the carbon emission optimization action includes optimizing the operating parameters of the facility to reduce carbon emissions when the system is in high carbon emission mode; maintaining the current parameters and further optimizing when the system is in low carbon emission mode; executing the carbon emission optimization action, and obtaining the corresponding second reward value and the carbon emission status information at the first moment; the second reward value is obtained based on the processing efficiency, energy consumption and carbon emission intensity of the facility; step three: based on the operating status information, the first reward value, the operating action, the carbon emission status information, the second reward value and the carbon emission optimization action at the current moment, as well as the operating status information and carbon emission status information at the first moment, update the parameters of the reinforcement learning model; step four: based on the parameters of the reinforcement learning model updated in step three, repeat steps one to three until all facilities are optimized.

[0126] It is understandable that the modules and references in this reinforcement learning-based water supply and drainage system carbon emission optimization system are Figure 1The steps in the method for counting and calculating carbon emissions of urban water supply and drainage systems throughout their life cycle correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method for counting and calculating carbon emissions of urban water supply and drainage systems throughout their life cycle are also applicable to the reinforcement learning-based water supply and drainage system carbon emission optimization system and the modules contained therein, and will not be repeated here.

[0127] Reference below Figure 3 , which shows a schematic structural diagram of an electronic device 300 suitable for implementing some embodiments of the present invention. The electronic devices in some embodiments of the present invention may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 3 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0128] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. Various programs and data required for the operation of the electronic device 300 are also stored in the RAM 303. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0129] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as needed.

[0130] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a terminal device such as a computer, a server, a mobile phone, or a tablet.

[0131] The above descriptions are merely some preferred embodiments of the present invention and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also encompass other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by mutually replacing the above-mentioned features with (but not limited to) technical features having similar functions disclosed in the embodiments of the present invention.

Claims

1. A method for counting and calculating carbon emissions over the entire life cycle of an urban water supply and drainage system, characterized in that: include: constructing a facility group based on the first facility number of each facility within the water supply and drainage system control area; Obtain the current status information of each facility in the facility cluster, including operating status information and carbon emission status information. The operating status information includes the current facility's water supply, drainage, and corresponding operating mode; the carbon emission status information includes the current facility's carbon emission intensity, carbon emissions per unit of processing capacity, energy consumption of the current facility, and carbon emission data of other current facilities; The reinforcement learning model is used to process operational and carbon emission information to optimize the operation and carbon emission of water supply and drainage systems, including: Step 1: Use the reinforcement learning model to process the current running status information to obtain the corresponding running action; The operation action includes maintaining the current operation mode or switching to the next operation mode; executing the operation action and obtaining the corresponding first reward value and operation status information at the first moment; the first reward value is the negative value of the absolute value of the sum of the total carbon emission intensity of all facilities in the current water supply and drainage system, and the total carbon emission intensity includes the difference between the carbon emission intensity of water supply facilities and the carbon emission intensity of drainage facilities; the first moment is the moment after the current moment; Step 2: Use the reinforcement learning model to process the current carbon emission status information to obtain the corresponding carbon emission optimization action; The carbon emission optimization action includes optimizing the operating parameters of the facility to reduce carbon emissions when the system is in a high carbon emission mode; maintaining the current parameters and further optimizing them when the system is in a low carbon emission mode; executing the carbon emission optimization action and obtaining a corresponding second reward value and carbon emission status information at the first moment; the second reward value is obtained based on the processing efficiency, energy consumption, and carbon emission intensity of the facility; Step 3: Update the parameters of the reinforcement learning model based on the current operating state information, the first reward value, the operating action, the carbon emission state information, the second reward value and the carbon emission optimization action, as well as the operating state information and carbon emission state information at the first moment; Step 4: Based on the parameters of the reinforcement learning model updated in step 3, repeat steps 1 to 3 until all facilities are optimized.

2. The method for statistics and calculation of carbon emissions of urban water supply and drainage systems throughout their life cycle according to claim 1 is characterized in that: The method also includes switching the operating mode of the facility within the switching area based on the operating mode switching demand of the facility: receiving the operating mode switching demand of the facility, and judging whether the facility with the switching demand is an intelligent control facility. If it is an intelligent control facility, the facility with the switching demand sends the target parameters to the target facility, and the target facility and its associated facilities adjust the operating parameters based on the target parameters to form an optimized mode that can be accessed by the facility with the switching demand. The target facility is a facility adjacent to and downstream of the facility with the switching demand after the facility with the switching demand completes the switching.

3. The method for statistics and calculation of carbon emissions of urban water supply and drainage systems throughout their life cycle according to claim 1 is characterized in that: Based on the first number of facilities in each facility within the water supply and drainage system control area, a facility group is constructed, including: obtaining the first number of facilities in each facility within the water supply and drainage system control area; judging whether the first number of facilities is greater than a first threshold value, if it is greater than the first threshold value, the first first threshold number of facilities in the corresponding area form a facility group; otherwise, all facilities in the corresponding area form a facility group; wherein the facilities in the facility group maintain a fixed priority operation.

4. The method for statistics and calculation of carbon emissions of urban water supply and drainage systems throughout their life cycle according to claim 1 is characterized in that: Determining the first reward value includes: determining a second facility number of each facility in the water supply and drainage system based on the current state information; Determining first data for the corresponding facility based on the second facility number and the preset facility capacity of each facility, where the first data represents the difference in carbon emission intensity of all imports and exports of the facility; The sum of the first data of all facilities in the water supply and drainage system is obtained to determine a first reward value.

5. The method for statistics and calculation of carbon emissions of urban water supply and drainage systems throughout their life cycle according to claim 1 is characterized in that: Determining the second reward value includes: obtaining a first time when the facility enters the high carbon emission mode, a second time when the facility exits the high carbon emission mode, a second facility number of each facility during the execution of the operation action, carbon emission intensity, energy consumption, and facility processing volume; determining an average operating time of the facility based on the first time, the second time, and the corresponding number of second facilities; Obtaining a first value based on the carbon emission intensity of the second facility and the carbon emission intensity of the first facility; Obtaining a second value based on a processing capacity of the first facility and a processing capacity of a second facility, where the second facility is a downstream facility associated with the first facility; Obtaining a third value based on a difference between the first value, the second value, and the facility throughput; calculating a larger value of a third value and a zero value during the execution of the operating action, and determining a carbon emission optimization time for the facility based on the larger value and the second facility number; Calculate the average carbon emissions of a facility based on its carbon emission intensity and energy consumption; A second reward value is determined based on the average operating time of the facility, the carbon emission optimization time, and the average carbon emission amount.

6. The method for statistics and calculation of carbon emissions of urban water supply and drainage systems throughout their life cycle according to claim 1 is characterized in that: The loss function expression of the critic in the reinforcement learning model includes: y t =r t +γQ(s t+1 ,π(s t+1 |φ′)) L Critic (θ)=(y t -Q(s t ,A t |i)) 2 Among them, y t represents the objective function value, r t represents reward information, γ represents discount factor, Q represents action value function, s t+1 represents the state information at time t+1, π represents the policy function, φ′ represents the parameters of the target policy network, θ′ represents the target Q network parameters, and L Critic represents the critic's loss function, θ represents the online Q-network parameters, and s t Represents the state information at time t, A t Indicates action information.

7. The method for statistics and calculation of carbon emissions of urban water supply and drainage systems throughout their life cycle according to claim 1 is characterized in that: The loss function expression for the actor in the reinforcement learning model includes: Among them, L Actor represents the actor’s loss function, θ represents the online Q-network parameters, Indicates the gradient of θ, Q represents the action value function, s t represents the state information at time t, π represents the policy function, and φ represents the online policy network parameters.

8. The method for statistics and calculation of carbon emissions of urban water supply and drainage systems throughout their life cycle according to claim 4 is characterized in that: The expression for determining the first reward value includes: Among them, R trajectory represents the first reward value, W(l,m) represents the difference in carbon emission intensity, b(l) represents the number of second facilities of the lth water supply facility, b max (l) represents the capacity of the lth water supply facility, b(m) represents the number of second facilities of the mth drainage facility, b max (l) represents the capacity of the mth drainage facility, l represents the number of the water supply facility, m represents the number of the drainage facility, and i represents the set of facilities.

9. The method for statistics and calculation of carbon emissions of urban water supply and drainage systems throughout their life cycle according to claim 5 is characterized in that: Determine the second reward value, the expression of which includes: R′ trajectory =w1·E n +w2·F n +w3·S n Among them, R′ trajectory represents the second reward value, w1 represents the weight corresponding to efficiency, E n represents the average running time, w2 represents the weight corresponding to energy consumption, F n represents the average carbon emissions of the facility, w3 represents the weight corresponding to the carbon emission optimization time, S n represents the carbon emission optimization time, N represents the number of the second facility of each facility during the execution of the operation action, and n represents the number of the facility. Indicates the second time a facility exits high carbon emission mode, Indicates the first time that the facility enters high carbon emission mode, v n (t) represents the carbon emission intensity of facility numbered n at time t, v n-1 (t) represents the carbon emission intensity of facility numbered n-1 at time t, x n-1 (t) represents the processing capacity of the facility numbered n-1 at time t, x n (t) represents the processing capacity of the facility numbered n at time t, l represents the baseline value of the facility processing capacity, j1 represents the power coefficient of carbon emission intensity, j2 represents the power coefficient of energy consumption, represents the constant coefficients corresponding to j1 and j2, a n (t) represents the energy consumption of facility numbered n at time t.

10. A carbon emission optimization system for water supply and drainage systems based on reinforcement learning, characterized in that: include: The mode switching module controls the facility to switch its operating mode within the switching area based on the facility's operating mode switching requirements; a facility group construction module, which constructs a facility group based on the first facility number of each facility within the water supply and drainage system control area; An acquisition module is used to obtain the current status information of each facility in the facility group. The status information includes operating status information and carbon emission status information. The operating status information includes the water supply and drainage volume of the current facility and the corresponding operating mode. The carbon emission status information includes the carbon emission intensity of the current facility, the carbon emission per unit of processing volume, the energy consumption of the current facility, and the carbon emission data of other current facilities. The processing module processes the operation status information and carbon emission status information based on the reinforcement learning model to optimize the operation and carbon emission of the water supply and drainage system, including: step 1: using the reinforcement learning model to process the operation status information at the current moment to obtain the corresponding operation action; The operation action includes maintaining the current operation mode or switching to the next operation mode; executing the operation action and obtaining the corresponding first reward value and the operation status information at the first moment; the first reward value is the negative value of the absolute value of the sum of the total carbon emission intensity of all facilities in the current water supply and drainage system, and the total carbon emission intensity includes the difference between the carbon emission intensity of water supply facilities and the carbon emission intensity of drainage facilities; the first moment is the next moment after the current moment; step 2: using the reinforcement learning model to process the carbon emission status information at the current moment to obtain the corresponding carbon emission optimization action; the carbon emission optimization action includes optimizing the operation parameters of the facility to reduce the carbon emission when the system is in a high carbon emission mode. Carbon emissions; when the system is in low carbon emission mode, maintain the current parameters and further optimize; perform carbon emission optimization actions, and obtain the corresponding second reward value and carbon emission status information at the first moment; the second reward value is obtained based on the processing efficiency, energy consumption and carbon emission intensity of the facility; step three: based on the current operating status information, the first reward value, the operating action, the carbon emission status information, the second reward value and the carbon emission optimization action, as well as the operating status information and carbon emission status information at the first moment, update the parameters of the reinforcement learning model; step four: based on the parameters of the reinforcement learning model updated in step three, repeat steps one to three until all facilities are optimized.

Images