Water conservancy hub construction site green production resource scheduling method based on reinforcement learning

Abstract

The application discloses a green production resource scheduling method for a water conservancy hub construction site based on reinforcement learning, and relates to the technical field of data processing. The method fuses construction resource state data and hydrological forecast data to construct an extended state vector, calculates a three-level hydrological risk grade of safety, early warning and danger based on the hydrological forecast data, and switches between three scheduling modes of normal, acceleration and retreat according to the three-level hydrological risk grade, to determine an optimization target priority and a resource allocation constraint condition. A reward function includes at least a green production index item composed of carbon emission intensity and construction water consumption rate, and a hydrological risk adjustment factor, the target weights are initialized and dynamically adjusted according to the optimization target priority, and the weight of the green production target is not less than a preset green bottom line value. A reinforcement learning intelligent agent trains a green production resource scheduling strategy within the action range defined by the constraint condition, outputs a scheduling scheme after deployment, and returns to re-execution when the deviation of the predicted peak flow exceeds a threshold, forming a closed loop.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method for scheduling green production resources at water conservancy project construction sites based on reinforcement learning. Background Technology

[0002] The construction of water conservancy projects requires the coordinated allocation of diverse resources, including machinery, manpower, and materials, in river valleys. The scheduling rhythm is strongly constrained by river hydrological processes—work on the foundation pit must be suspended during flood season, with the dry season being the only working window. Simultaneously, carbon emission control, water management, and river ecological protection during construction have become rigid constraints. While reinforcement learning has been applied to construction scheduling, its application to the green production resource scheduling at water conservancy project construction sites still has the following shortcomings:

[0003] First, existing reinforcement learning-based construction scheduling methods model the construction environment as a static or slowly varying Markov decision process, whose state space only includes task and resource states, failing to consider the direct constraints of natural hydrological processes on construction activities. However, the scheduling rhythm at a water conservancy project construction site is essentially determined by the hydrological cycle; the arrival of a flood can disrupt the entire construction plan, rendering the original scheduling scheme completely ineffective. Existing methods lack the ability to perceive dynamic hydrological changes and cannot proactively adjust resource allocation before the risk of flooding, leading to project delays and resource waste caused by passive work stoppages.

[0004] Second, existing reinforcement learning-based green scheduling methods are mainly geared towards manufacturing workshop scenarios, simplifying green objectives into single-dimensional indicators such as energy consumption or carbon emissions. However, the green production needs at water conservancy project construction sites are multi-dimensional, involving not only carbon emissions and energy consumption, but also construction water consumption, construction wastewater discharge, and downstream river ecological impacts. The single-dimensional green indicators of existing methods cannot comprehensively reflect the integrated impact of water conservancy project construction on the ecological environment.

[0005] Third, existing multi-objective scheduling methods, when facing emergencies, typically employ a strategy of reducing the weight of secondary objectives to ensure the primary objective, meaning that in emergency situations, green production goals are abandoned to prioritize safety and progress. This strategy has flaws in the context of water conservancy project construction—completely abandoning green management during floods could lead to the uncontrolled discharge of construction wastewater and sediment into rivers, causing secondary ecological disasters. Current technologies lack mechanisms to maintain a baseline of green production under emergency scheduling scenarios.

[0006] Fourth, existing construction scheduling methods treat construction resource allocation and diversion schemes as independent decision-making processes, handled separately by construction management and hydraulic engineering design professionals. However, in the construction of hydraulic engineering projects, the diversion scheme directly determines the workability conditions of each construction area—the cofferdam elevation and the opening and closing status of the diversion tunnel determine whether the foundation pit is dry and ready for construction. This separation of construction resource allocation and diversion control prevents the achievement of global synergistic optimization between the two.

[0007] Fifth, the hydrological conditions of water conservancy project construction are highly uncertain and scenario-specific, and the flood process characteristics of different river basins vary significantly, making it difficult to directly transfer reinforcement learning scheduling strategies trained on one project to other projects. Existing reinforcement learning scheduling methods lack generalization training mechanisms for multiple hydrological scenarios, and the robustness of the strategies cannot be guaranteed under conditions of insufficient training data.

[0008] Therefore, there is an urgent need for a method for scheduling green production resources at the construction site of water conservancy projects that can sense dynamic changes in hydrology, take into account the bottom line of green production in multi-mode hierarchical response, achieve joint optimization of construction scheduling and diversion control, and have the ability to generalize across hydrological scenarios. Summary of the Invention

[0009] To address the problems in existing construction scheduling methods, such as lack of perception of dynamic hydrological changes, single-dimensional green indicators, complete abandonment of green objectives in emergency scenarios, disconnect between construction scheduling and diversion control, and lack of generalization ability of scheduling strategies across hydrological scenarios, this invention provides a green production resource scheduling method for water conservancy project construction sites based on reinforcement learning.

[0010] The technical solution provided by this invention is as follows:

[0011] A reinforcement learning-based method for green production resource scheduling at water conservancy project construction sites includes the following steps:

[0012] S1: Obtain construction resource status data and hydrological forecast data at the construction site of the water conservancy project, integrate the construction resource status data with the hydrological forecast data, and construct an extended state vector;

[0013] S2: Calculate the hydrological risk level based on hydrological forecast data. The hydrological risk level includes the safety level, warning level, and danger level.

[0014] S3: Based on the hydrological risk level, switch between the normal scheduling mode, accelerated scheduling mode and retreat scheduling mode, and determine the optimization objective priority and resource allocation constraints corresponding to the current scheduling mode;

[0015] S4: Construct a reward function, which includes green production indicators and a hydrological risk adjustment factor. The green production indicators include at least two dimensions: carbon emission intensity and construction water consumption rate. The hydrological risk adjustment factor initializes the weight ratio between the progress target, cost target, and green production target according to the priority of the optimization target determined in step S3 based on the hydrological risk level and the current scheduling mode, and dynamically adjusts the weight ratio. The weight of the green production target is not lower than the preset green baseline value.

[0016] S5: Using the extended state vector as the input to the reinforcement learning agent, the construction resource allocation decision within the action range limited by resource allocation constraints as the action, and the output of the reward function as the feedback signal, the reinforcement learning agent is trained to obtain a green production resource scheduling strategy.

[0017] S6: Utilize the green production resource scheduling strategy, and output the construction resource scheduling scheme based on the real-time updated extended state vector and the resource allocation constraints corresponding to the current scheduling mode; during the execution of the construction resource scheduling scheme, when the deviation between the real-time acquired predicted peak flow and the predicted peak flow used at the previous decision time exceeds the preset deviation threshold, return to step S1 to reacquire data and execute steps S2, S3 and this step in sequence to output an updated construction resource scheduling scheme.

[0018] Furthermore, in step S1, the hydrological forecast data includes the predicted flow sequence, water level change trend and rainfall probability within the preset time window; the construction resource status data includes the availability and current location of various types of construction machinery, the job distribution and number of construction personnel, the inventory and arrival plan of construction materials, and the current progress of each construction area.

[0019] Furthermore, in step S1, the extended state vector also includes the operational constraint information of the current diversion scheme for each construction area; in step S5, the operational constraint information is encoded into an action mask, and when the current diversion conditions cause a specific construction area to be unoperable, the action of allocating resources to that construction area is masked.

[0020] Furthermore, in step S1, the extended state vector also includes meteorological condition data of the construction site; in step S3, when the meteorological condition data meets the extreme climate judgment conditions, the meteorologically sensitive construction activities are restricted or suspended within the current scheduling mode, and the corresponding resources are allocated to the meteorologically insensitive construction activities.

[0021] Furthermore, in step S3, the conventional scheduling mode sets the green production target as the highest priority, and the resource allocation constraints allow all types of construction activities to participate in the scheduling; the accelerated scheduling mode sets the progress target as the highest priority, and the resource allocation constraints concentrate resources on key processes and tighten process time constraints; the evacuation scheduling mode sets the safety target as the highest priority, and the resource allocation constraints limit the scope of resource allocation to evacuation actions and temporary protection actions.

[0022] Furthermore, in step S3, the construction site of the water conservancy project includes multiple construction areas, each of which has a different degree of hydrological risk exposure based on its location in the river channel. Different construction areas are allowed to implement differentiated scheduling modes according to their respective hydrological risk exposure levels. Construction areas closer to the river channel are given priority to enter the accelerated scheduling mode or the retreat scheduling mode, while construction areas farther from the river channel maintain the regular scheduling mode.

[0023] Furthermore, in step S4, the hydrological risk adjustment factor dynamically adjusts its weight ratio in the following manner: when the hydrological risk level rises from the safe level to the warning level, the weight of the progress target is increased and the weight of the cost target is reduced; when the hydrological risk level falls back from the dangerous level or the warning level to the safe level, a recovery efficiency reward item is introduced into the reward function to guide the reinforcement learning agent to prioritize the recovery of high-priority processes within the resumption window.

[0024] Furthermore, in step S4, the green production indicator also includes an ecological impact index; the ecological impact index is determined based on the amount of wastewater discharged from construction activities and the ecological flow satisfaction rate of downstream rivers; when the ecological impact index exceeds a preset ecological threshold, the reward function applies a penalty signal.

[0025] Furthermore, in step S5, the action space of the reinforcement learning agent includes a construction resource scheduling action subspace and a diversion control suggestion action subspace. The construction resource scheduling action subspace is used to determine the allocation scheme of construction machinery, manpower and materials among various construction areas. The diversion control suggestion action subspace is used to generate suggestions for cofferdam elevation adjustment and diversion tunnel opening and closing timing. A hierarchical time-scale semi-Markov decision process is used for modeling, and the decision cycle of the diversion control suggestion action subspace is longer than that of the construction resource scheduling action subspace.

[0026] Furthermore, in step S5, a set of hydrological scenarios covering typical years, dry years, and years of extreme floods is generated based on historical hydrological data; a meta-reinforcement learning method is used to train a meta-policy on multiple scenarios in the set of hydrological scenarios; in step S6, the closest historical scenario policy in the set of hydrological scenarios is matched as the initial policy based on real-time hydrological forecast data, and then fine-tuned online to adapt to the current hydrological conditions.

[0027] The beneficial effects of the technical solution provided by this invention include:

[0028] 1. Achieving dynamic hydrological perception and hierarchical adaptive response: In response to the problem pointed out in the background technology that existing construction scheduling methods lack the ability to perceive dynamic changes in hydrology, this invention introduces hydrological forecast data into the state space of reinforcement learning, and designs a switching mechanism for three scheduling modes—regular, accelerated, and retreat—based on hydrological risk levels. This enables green production resource scheduling strategies to proactively adjust resource allocation according to dynamic changes in hydrology, reducing project delays and resource waste caused by passive work stoppages.

[0029] 2. Covering multiple dimensions of green production indicators: In response to the problem that existing green scheduling methods, as pointed out in the background technology, only focus on a single energy consumption dimension, the green production indicators of this invention cover carbon emission intensity, construction water consumption rate, and ecological impact index, comprehensively evaluating the green production level of construction activities from three dimensions: carbon emission control, water resource management, and river ecological protection.

[0030] 3. Safeguarding the bottom line of green production in emergency scenarios: In response to the problem pointed out in the background technology that the existing multi-objective scheduling method completely abandons the green objectives in emergency situations, the present invention sets the weight of the green production objective in the reward function to be no less than the preset green bottom line value, so that even in emergency scheduling scenarios, basic control over carbon emissions and construction water consumption is maintained to prevent secondary ecological pollution.

[0031] 4. Achieving joint optimization of diversion control and construction scheduling: In response to the problem of separate decision-making between construction resource scheduling and diversion schemes pointed out in the background technology, this invention incorporates diversion control suggestions and construction resource scheduling into the dual action space of the same reinforcement learning agent, and adopts a hierarchical time-scale semi-Markov decision process to handle the frequency difference between the two types of decisions, thereby achieving collaborative optimization of diversion schemes and construction resource allocation.

[0032] 5. Enhance the generalization ability of strategies across hydrological scenarios: In response to the problem pointed out in the background art that existing reinforcement learning scheduling strategies lack the ability to generalize across hydrological scenarios, this invention uses a meta-reinforcement learning training method based on multiple hydrological scenario sets to enable the scheduling strategy to maintain effective decision-making through online fine-tuning when facing flood processes that have not been trained. Attached Figure Description

[0033] Figure 1 This is a flowchart illustrating the green production resource scheduling method for water conservancy project construction sites based on reinforcement learning, as provided in an embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to specific embodiments. It should be understood that the embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of this invention.

[0035] This embodiment uses a large-scale water conservancy project as an application scenario. The project includes four construction areas: a concrete gravity dam, a spillway, a water diversion tunnel, and a material transport channel. The construction period spans multiple flood and dry seasons. The construction site is located in a river valley, where flood events directly impact the safety of the foundation pit operations. Construction resource scheduling must balance four objectives—progress, cost, safety, and green production—under dynamic hydrological conditions. Details are as follows:

[0036] S1: Extended State Construction: Obtain construction resource status data and hydrological forecast data at the construction site of the water conservancy project, integrate the construction resource status data and hydrological forecast data, and construct an extended state vector.

[0037] Construction resource status data is collected through the construction site management information system. The collected data includes: the availability and current location of various construction machinery, with availability represented by three discrete values: in use, idle, and under maintenance; and current location identified by the construction area number. The data also includes the distribution and number of construction workers by trade, including steelworkers, formwork workers, concrete workers, machinery operators, and welders. Furthermore, the data includes the inventory and delivery schedule of construction materials, with inventory represented by current warehouse reserves and delivery schedules represented by daily estimated deliveries for the next seven days. Finally, the data includes the current progress of each construction area, expressed as the percentage of completed work relative to the planned work. After discrete value one-hot encoding and continuous value normalization, the data forms a dimension... The construction resource state vector, in this embodiment Take 128.

[0038] Hydrological forecast data is obtained through connection with the forecasting system of the higher-level hydrological department and consists of three components. The first is the predicted flow sequence, which is the hourly predicted flow value for the next 72 hours, totaling 72 flow values, reflecting the change process of river flow. The second is the water level change trend, which is the hourly rate of change of the predicted water level, totaling 72 rate of change values; positive values ​​indicate rising water levels, and negative values ​​indicate falling water levels. The third is the rainfall probability, which is the probability that the cumulative rainfall in the basin in the next 24 hours will exceed a preset rainfall threshold; this is a single scalar value. The three components are concatenated to form a dimension... The hydrological forecast vector, in this embodiment Take 145.

[0039] The extended state vector is formed by concatenating the construction resource state vector with the hydrological forecast vector. In this embodiment The total dimension is 273. This vector simultaneously carries information on both the construction resource dimension and the hydrological environment dimension, which is used in subsequent steps S2 for risk assessment and in step S5 for scheduling decisions by the reinforcement learning agent.

[0040] In a preferred embodiment, the extended state vector It also includes operational constraints of the current diversion plan for each construction area. This information is categorized by dimension. The binary vector representation, The component represents the number of construction areas, with each component corresponding to one construction area: a value of 1 indicates that work is possible when the cofferdam elevation is higher than the design protection elevation of the area and the diversion tunnel is in normal operation. Otherwise, a value of 0 indicates that work is not possible. This binary vector is appended to the end of the extended state vector, expanding the total dimension to [value missing]. This provides the data foundation for the action masking mechanism in step S5.

[0041] In a preferred embodiment, the extended state vector It also includes meteorological data at the construction site, specifically three components: temperature, wind speed, and precipitation. These are collected in real time by the meteorological station at the construction site, normalized and encoded, and then spliced ​​to the end of the extended state vector for use in the determination of extreme weather conditions in step S3.

[0042] In this embodiment of the invention, step S1 integrates construction resource status data and hydrological forecast data into a unified extended state vector, enabling the reinforcement learning agent's state input to simultaneously include information from both the construction resource dimension and the hydrological environment dimension. Compared to a state space that only includes task and resource states, the extended state vector allows the agent to directly acquire hydrological forecast signals during decision-making, thereby providing the ability to perceive dynamic changes in hydrology at the state representation level. After incorporating diversion scheme operational constraints and meteorological condition data into the extended state vector, the state space further encompasses the constraints of diversion conditions and meteorological conditions on construction activities, providing the necessary input data for hydrological risk assessment in step S2 and constrained action selection in step S5.

[0043] S2: Hydrological Risk Assessment: The hydrological risk level is calculated based on hydrological forecast data. The hydrological risk level includes safety level, warning level and danger level.

[0044] Based on the hydrological forecast data obtained in step S1, the hydrological risk level of the construction site can be further calculated. The hydrological risk level is divided into three levels: safety level, warning level, and danger level. The determination is based on the peak flow rate in the predicted flow sequence for the next 72 hours. With the standard flow rate of the cofferdam design The ratio relationship, and taking into account the probability of rainfall. The specific determination rules are shown in equation (1):

[0045]

[0046] in: The peak flow rate in the predicted flow sequence for the next 72 hours, in cubic meters per second, is extracted from the hydrological forecast data in step S1. The standard flow rate for the cofferdam design is expressed in cubic meters per second and is determined based on the construction diversion design scheme. For safety, a factor of 0.7 is used in this embodiment, which is determined with reference to the recommended range of flood safety margin during the construction period in the Code for Construction Organization Design of Water Conservancy and Hydropower Projects. The probability of rainfall in the basin in the next 24 hours is extracted from the hydrological forecast data in step S1. The threshold for rainfall probability warning is set to 0.6 in this embodiment. The rainfall probability hazard threshold is set to 0.85 in this embodiment. This threshold can be adjusted according to the cofferdam design standards and the hydrological characteristics of the watershed.

[0047] The determination of the safety level requires that both flow and rainfall conditions simultaneously meet the safety threshold, with the two conditions logically ANDed. The determination of the warning level and the danger level is triggered when either condition reaches its corresponding threshold, with the two conditions logically ORed. This asymmetric logic ensures that the system's response to risk signals follows the principle of "better to over-warn than under-report danger," enabling it to detect early signs of risk even when flow has not yet significantly increased but the probability of rainfall is already high.

[0048] The hydrological risk level output in step S2 will directly drive the switching of the scheduling mode in step S3.

[0049] In this embodiment of the invention, step S2 discretizes the continuous hydrological forecast data into three levels: safe, warning, and dangerous, providing clear triggering conditions for switching the scheduling mode in step S3. The safe level requires both flow and rainfall conditions to simultaneously meet a safe threshold. The warning and dangerous levels are triggered when either condition reaches its corresponding threshold. This asymmetric decision structure allows the system to enter a warning state even when the flow has not yet increased significantly but the probability of rainfall has already increased, shifting the risk response time to the precursor stage of the flood process.

[0050] S3: Scheduling mode switching: Based on the hydrological risk level, switch between the normal scheduling mode, accelerated scheduling mode and retreat scheduling mode, and determine the optimization objective priority and resource allocation constraints corresponding to the current scheduling mode.

[0051] Once the hydrological risk level is determined, the construction resource scheduling system switches between three preset scheduling modes, each corresponding to different optimization objectives, priorities, and resource allocation constraints.

[0052] When the hydrological risk level is at the safe level, the system selects the conventional scheduling mode. In this mode, green production objectives are set as the highest priority, and resource allocation constraints allow all types of construction activities to participate in the scheduling. The priority order of objectives is: green production objectives, cost objectives, and schedule objectives.

[0053] When the hydrological risk level reaches the warning level, the system switches to accelerated scheduling mode. This mode sets the schedule objective as the highest priority, and the resource allocation constraints narrow the scope of available resources to critical processes—those located on the critical path of construction and closest to the flood-affected area. The allowable completion time for each critical process is shortened to a preset percentage of the original plan; in this embodiment, this percentage is 0.7. The accelerated scheduling mode aims to concentrate resources during the limited window before the flood arrives, with the following priority order: schedule objective, safety objective, green production objective, and cost objective.

[0054] When the hydrological risk level reaches the hazardous level, the system switches to evacuation dispatch mode. This mode prioritizes safety objectives, and resource allocation constraints limit resource allocation to two categories: evacuation actions and temporary protection actions. Evacuation actions include the orderly evacuation of construction personnel and the transfer of construction machinery to safe areas. Temporary protection actions include the temporary closure and protection of the construction surface and the emergency reinforcement of unfinished works. The former refers to laying waterproof fabric and blocking construction access routes, while the latter refers to temporarily covering and protecting concrete surfaces that have been poured but have not yet reached their design strength. The priority order of objectives in the evacuation dispatch mode is: safety objectives, green production objectives, schedule objectives, and cost objectives. Green production objectives remain prioritized over schedule and cost objectives in evacuation mode. This design, combined with the green baseline value guarantee mechanism in step S4, ensures that basic control over construction wastewater discharge and carbon emissions is maintained during emergency evacuation.

[0055] The optimization objective priority output in step S3 is used to determine the initial weight values ​​when constructing the reward function in step S4, and the resource allocation constraints output are used to limit the action range of the reinforcement learning agent in step S5.

[0056] In a preferred embodiment, when the construction site comprises multiple construction areas, each area can implement a differentiated scheduling mode based on its hydrological risk exposure level. The risk exposure coefficient for each construction area... Calculate according to formula (2):

[0057]

[0058] in: For the first The risk exposure coefficient for each construction area ranges from 0 to 1, with a higher value indicating a higher degree of hydrological risk exposure. For the first The normalized distance from the center of each construction area to the river surface is obtained by dividing the actual distance by the maximum distance from each area to the river. The value ranges from 0 to 1, with the value increasing as the distance increases. For the first The normalized elevation difference between the elevation of each construction area and the top elevation of the cofferdam is obtained by dividing the actual elevation difference by the maximum elevation difference of all areas in the field. The value ranges from 0 to 1, with the larger the elevation difference, the larger the value. For the first The completeness coefficient of temporary protective facilities in a construction area is determined based on the ratio of the number of completed temporary protective measures to the number that should be completed, with a value ranging from 0 to 1. The higher the completeness, the larger the value. , and The weights are distance, elevation, and protection, respectively, and in this embodiment, they are taken as 0.4, 0.35, and 0.25, respectively, with the sum of the three equal to 1. The distance weight is taken as the highest value because the distance from the river channel is the most direct factor determining the arrival time and inundation depth of the flood. All three terms in equation (2) adopt... By subtracting the form of the corresponding normalized safety index, areas that are close to the river, have low elevation, and lack adequate protection can obtain higher exposure coefficient values, which aligns with the engineering intuition that "the larger the coefficient, the higher the risk."

[0059] When the overall hydrological risk level is at the warning level... Exceeding the preset exposure threshold The construction area enters accelerated scheduling mode or evacuation scheduling mode, while the remaining areas maintain the normal scheduling mode. In this embodiment... We set the value to 0.6. In this way, areas closer to the river and with lower elevations respond earlier, while areas farther from the river continue with regular construction.

[0060] In a preferred embodiment, when the extended state vector When meteorological data is included, step S3 also incorporates extreme climate response: when the temperature is below -10 degrees Celsius, the wind speed exceeds level 6, or the continuous precipitation exceeds the preset threshold, high-altitude operations, large-volume concrete pouring, and precision measurement operations are suspended within the current scheduling mode, and the corresponding resources are allocated to tunnel construction, material handling, and equipment maintenance.

[0061] In this embodiment of the invention, step S3 switches between three scheduling modes—regular, accelerated, and evacuation—based on the hydrological risk level. Each mode corresponds to a different domain of priority ranking of optimization objectives and resource allocation constraints. This hierarchical mechanism establishes an explicit mapping relationship between hydrological risk levels and the scheduling behavior space: under the safety level, the optimization objective prioritizes green production and resource allocation is not subject to narrowing constraints; under the warning level, the optimization objective prioritizes progress and resources are concentrated on key processes; under the danger level, the optimization objective prioritizes safety and resource allocation is limited to evacuation and temporary protection. The differentiated regional scheduling mechanism independently determines the scheduling mode based on the risk exposure coefficient of each construction area, enabling areas far from the river channel to maintain regular construction even during a site-wide warning, reducing resource idleness caused by unified work stoppages.

[0062] S4: Reward Function Construction: Construct a reward function that includes green production indicators and a hydrological risk adjustment factor. The green production indicators must include at least two dimensions: carbon emission intensity and construction water consumption rate. The hydrological risk adjustment factor dynamically adjusts the weight ratio between the progress target, cost target, and green production target based on the hydrological risk level and the current scheduling mode, ensuring that the weight of the green production target is not lower than a preset green baseline value.

[0063] The reward function is a numerical signal used by a reinforcement learning agent to evaluate the quality of scheduling decisions, and its structure directly determines the agent's behavioral orientation during training. The reward function of this invention consists of two parts: a green production indicator and a hydrological risk adjustment factor. The former quantifies the performance of construction activities in terms of carbon emissions and water consumption, while the latter initializes the weights of each objective according to the priority of the optimization objectives determined in step S3, based on the hydrological risk level output in step S2 and the scheduling mode determined in step S3, and dynamically adjusts the weight ratio between the optimization objectives, ensuring that the weight of the green production objective is not lower than a preset green baseline value.

[0064] Green production indicators should include at least two dimensions: carbon emission intensity and construction water consumption rate. Carbon emission intensity It measures the carbon emissions corresponding to a unit of construction output, at any given time. Calculate according to formula (3):

[0065]

[0066] in: For a moment Carbon emission intensity, expressed in kilograms of carbon dioxide per cubic meter. For a moment Construction output, measured in cubic meters, refers to the completed work. When there is no construction output during a certain period... Choose a very small positive number to ensure numerical stability. This represents the total number of construction machines currently in use. For the first Taiwanese construction machinery at all times The runtime within the decision-making cycle, in hours. For the first Taiwanese construction machinery at all times The average load rate, ranging from 0 to 1, where 0 indicates no-load operation and 1 indicates full-load operation. For the first The carbon emission factor of a construction machine is expressed in kilograms of carbon dioxide per hour. It can be determined by referring to the Construction Machinery Carbon Emission Factor Manual based on the type of machinery and fuel.

[0067] Equation (3) normalizes carbon emissions to unit output. Therefore, when the agent chooses to concentrate on investing in high-carbon-emission equipment to simultaneously improve construction efficiency, both the numerator and denominator increase. Equation (3) can automatically weigh this relationship.

[0068] Construction water consumption rate This reflects the extent to which construction activities occupy the site's water supply capacity, at all times. Calculate according to formula (4):

[0069]

[0070] in: For a moment The construction water consumption rate is a dimensionless ratio, ranging from 0 to 1. This represents the number of construction activities participating in the scheduling during the current time period. For the first Construction activities at all times Water consumption, expressed in cubic meters per hour. This represents the total water supply capacity at the construction site, expressed in cubic meters per hour. It is determined by the design capacity of the water supply system and is a normal value. The closer it is to 1, the greater the water supply pressure. This is especially important to pay attention to during the dry season, when the water flow in the river is limited and an excessively high water intake rate may affect the downstream ecological flow.

[0071] Carbon emission intensity and construction water consumption rate are normalized to form a green production indicator. Calculate according to formula (5):

[0072]

[0073] in: For a moment The green production indicators are non-positive. This is a baseline value for carbon emission intensity, determined based on the statistical average of historical construction data for similar water conservancy projects, and the unit is kilograms of carbon dioxide per cubic meter. This is the baseline value for construction water consumption rate, determined based on the rated consumption ratio in the water supply plan designed for the construction site, and is a dimensionless positive value. and These are the carbon emission weighting coefficient and the water consumption weighting coefficient, respectively. In this embodiment Take 0.6, We set it to 0.4. Dividing by the respective baseline value aims to eliminate differences in units and orders of magnitude between the two dimensions. Taking a negative sign increases the penalty for worse performance in the green area, consistent with the training objective of reinforcement learning to maximize cumulative rewards.

[0074] In a preferred embodiment, the green production indicator also includes an ecological impact index. The impact of construction activities on downstream river water quality and ecological flow is evaluated and calculated according to formula (6):

[0075]

[0076] in: For a moment The ecological impact index is a non-negative dimensionless value. For a moment Wastewater discharge from construction activities is measured in cubic meters per hour, including concrete washing wastewater, foundation pit drainage, and mechanical cleaning wastewater. This represents the maximum treatment capacity of the wastewater treatment facility, expressed in cubic meters per hour. For a moment The actual flow rate of the downstream river channel, measured in cubic meters per second, is obtained in real time through downstream hydrological monitoring stations. This represents the minimum ecological flow requirement for the downstream river channel, expressed in cubic meters per second, and is determined based on the calculation results of the aquatic ecosystem requirements in the environmental impact assessment report. and These are the wastewater discharge weighting coefficient and the ecological flow weighting coefficient, respectively. In this embodiment, the value is always 0.5. In the second term of equation (6) The function's purpose is: when This item is zero at the beginning, and a positive score is only generated when the actual traffic is insufficient to meet the needs of the ecosystem, and the greater the insufficiency, the higher the score.

[0077] when Exceeding the preset ecological threshold At that time, the reward function applies an ecological penalty term. As shown in equation (7):

[0078]

[0079] in: For a moment The ecological penalty item is a non-positive value. To preset the ecological threshold, this embodiment uses 0.6, which is determined based on the maximum ecological impact allowed during the construction period in the environmental impact assessment report. The ecological penalty coefficient is taken as 10 in this embodiment. Equation (7) is... No penalty is imposed if the threshold is not exceeded; once the threshold is exceeded, the penalty intensity increases linearly with the extent of the exceedance, prompting the agent to... Keep it within the threshold.

[0080] The above indicators are summarized into the total reward function. As shown in equation (8):

[0081]

[0082] in: For a moment The total reward value. The progress target reward item at time t is determined based on the ratio of the completed work volume to the planned work volume within the decision cycle at time t. The cost target reward item at time t is determined based on the ratio of actual resource consumption cost to budgeted cost within the decision-making cycle at time t. Defined by equation (5). As defined by equation (7), the value is zero when the ecological impact index is not enabled. , and The timeframes are the schedule target, the cost target, and the green production target, respectively. The weights of the three terms are such that the sum of the three terms is always equal to 1. As an independent penalty item, it is not subject to weight adjustment and is directly superimposed to ensure that overstepping ecological limits is constrained under any scheduling mode.

[0083] The three weights are dynamically adjusted by the hydrological risk adjustment factor based on the current hydrological risk level. (Schedule target weight) The adjustment rule is shown in equation (9):

[0084]

[0085] in: The basic weight for the schedule target under the safety level is set to 0.25 in this embodiment. The increment of the schedule target weight when the risk increases is 0.20 in this embodiment. For a moment The hydrological risk level is determined by step S2 using equation (1). For indicator functions, when The value is 1 when it is a warning level or a danger level, and 0 when it is a safety level.

[0086] Cost target weight The adjustment rule is shown in equation (10):

[0087]

[0088] in: In this embodiment, the basic weight for the cost target under the safety level is set to 0.35. In this embodiment, the weight of the cost target is reduced to 0.15 to account for increased risk. and The meaning is the same as that of equation (9).

[0089] Weight of green production targets By constraints Determined, that is The three basic weights under the security level , and The values ​​are 0.25, 0.35, and 0.40 respectively, and their sum equals 1. Taking the warning level as an example, after adjustment... It is 0.45. It is 0.20. The value of 0.35 reflects the strategic orientation of accelerating progress at all costs while still maintaining a high green weight under early warning conditions.

[0090] After adjustment, an inspection is required. Is it not lower than the preset green baseline value? .like Then the weight of the progress target is adjusted according to formula (11):

[0091]

[0092] in: The revised schedule target weights. To preset the green baseline value, this embodiment uses 0.15. (Denominator) equal ,because Therefore, the denominator is greater than zero.

[0093] At the same time, the weight of the cost target is adjusted according to formula (12):

[0094]

[0095] in: This represents the revised cost target weight. The meanings of the remaining symbols are the same as in equation (11). The revised green production target weight is taken as... The sum of the weights after the three corrections is still equal to 1. Equations (11) and (12) are reduced proportionally. and The reduction is proportional, keeping the relative proportions of the two objectives of schedule and cost unchanged, while only forcibly increasing the weight of the green production objective to the minimum value.

[0096] Preset green baseline value The criteria for determination include the national environmental protection standards for the limits of carbon emission intensity and wastewater discharge during the construction period, the green construction indicators stipulated in the water conservancy project construction contract, and the benchmark level of green indicators obtained from the statistical analysis of historical construction data of similar projects. This means that even in the most urgent evacuation scheduling mode, at least 15% of the weight in the reward function is still reserved for evaluating green production performance, so that the agent can still take into account wastewater discharge control and carbon emission management when making evacuation resource allocation decisions, and prevent emergency evacuation operations from causing wastewater and sediment to be discharged into the river in a disorderly manner.

[0097] In a preferred embodiment, the hydrological risk adjustment factor also includes a recovery efficiency bonus. It is activated when the hydrological risk level drops from the danger level or warning level to the safe level, and is calculated according to formula (13):

[0098]

[0099] in: For a moment The recovery efficiency bonus value is non-negative. This is a set of high-priority procedures, determined based on critical path analysis, including procedures located on the critical path that were interrupted due to flooding. High priority process At any moment The increase in the amount of work completed after the resumption of work, in cubic meters. . High priority process The total planned project volume, in cubic meters. . To restore the efficiency reward coefficient, this embodiment uses 2.0, which is higher than the implicit coefficient of the regular progress reward, so as to provide additional incentives for rapid resumption of work during the resumption window.

[0100] Activated during the resumption of work window, which is a continuous 72-hour period after the hydrological risk level drops to a safe level. After the window ends... No longer activated, the total reward function reverts to the form of equation (8). The total reward during the window period is... Equation (13) for The summation of the normalized resumption increments of each process within the set reflects both the scope and speed of resumption, prompting the agent to concentrate resources on restoring high-priority processes after the flood.

[0101] In this embodiment of the invention, the reward function constructed in step S4 includes both carbon emission intensity and construction water consumption rate in the green production indicator, enabling the reward signal to reflect the performance of construction activities in terms of carbon emissions and water consumption respectively. The hydrological risk adjustment factor sets an initial weight value according to the optimization target priority determined in step S3, and dynamically adjusts the weight ratios of progress, cost, and green production when the hydrological risk level changes. The green baseline value constraint ensures that the weight of the green production target is not lower than the preset green baseline value under any scheduling mode, so that the reward function retains the evaluation components for carbon emissions and wastewater discharge even in the retreat scheduling mode corresponding to the danger level. The recovery efficiency reward item is activated during the window period after the hydrological risk level falls back to the safe level, giving a coefficient higher than the regular progress reward to the resumption increment of high-priority processes, guiding the agent to concentrate resources on restoring critical path processes within the resumption window.

[0102] S5: Scheduling strategy training: Using the extended state vector as the input to the reinforcement learning agent, the construction resource allocation decision within the action range limited by resource allocation constraints as the action, and the output of the reward function as the feedback signal, the reinforcement learning agent is trained to obtain a green production resource scheduling strategy.

[0103] Steps S1 to S4 respectively complete the state space construction, risk assessment, mode switching, and preparation of the reward function. Step S5 then expands the state vector based on these steps. To enhance the input of the learning agent, the construction resource allocation decision within the action range defined by the resource allocation constraints determined in step S3 is used as the action, and the output of the reward function constructed in step S4 is used as the feedback signal to train the reinforcement learning agent and obtain a green production resource scheduling strategy.

[0104] Reinforcement learning agent actions Within the resource allocation constraints of the current scheduling mode, a plan is determined for the allocation of various types of construction machinery, manpower, and materials among different construction areas. The system is represented by a multidimensional discrete-continuous hybrid vector: the discrete dimension represents the allocation and combination of various resources to different construction areas, while the continuous dimension represents the proportion of input intensity. Under the conventional scheduling mode, the action scope covers all construction areas and all activity types. Under the accelerated scheduling mode, it narrows to the area where key processes are located. Under the evacuation scheduling mode, it only includes evacuation actions and temporary protection actions. The constraints on the action scope imposed by the scheduling modes cause the agent to face decision spaces of varying sizes under different hydrological risk conditions.

[0105] In a preferred embodiment, when the extended state vector When job constraint information is included, this information is encoded as an action mask, and constraints are applied to the action components of non-jobable regions at the policy network output layer. The mathematical implementation of the action mask is shown in equation (14):

[0106]

[0107] in: The output of the strategy network is directed to the construction area. The probability of allocating resources. The construction area corresponding to the output layer of the strategy network. The original logarithmic probability value. For the first The mask value for each construction area is either 0 or 1, and is determined by the operation constraint information in step S1. This represents the total number of construction areas. The construction area corresponding to the output layer of the strategy network. The original logarithmic probability value. Let be the mask value for the u-th construction area. When... hour, If the value is zero, the allocation of resources to that area is completely excluded. At that time, the probability is normalized across all workable regions according to the original preferences of the policy network. The denominator requires at least one... The workable area is always met in the actual construction of water conservancy projects. At least the material transport channel in the safe area can be operated under any diversion conditions. Equation (14) directly excludes infeasible actions from the strategy output layer. During the training process, there is no need to rely on penalty signals to guide the agent to learn to avoid infeasible solutions, thus reducing ineffective exploration.

[0108] In this embodiment, the reinforcement learning agent is trained using a proximal policy optimization algorithm. The policy network is based on... The value network uses the input and output action probability distribution as a basis. This is for estimating the input and output state values. Both networks use a three-layer fully connected structure with 256 hidden layers, ReLU activation function, and a learning rate of... The cutting parameter is 0.2, and the discount factor is... The value is 0.99. The total number of training steps is 5 million, and the parameters are updated every 2048 steps. Among the three comparisons of 128, 256 and 512 hidden layer dimensions, the training stability and convergence speed of 256 are optimal.

[0109] In a preferred embodiment, the action space of the reinforcement learning agent is expanded to include a construction resource scheduling action subspace and a diversion control suggestion action subspace. The definition of the construction resource scheduling action subspace is consistent with that described above. The diversion control suggestion action subspace is used to generate suggestions for cofferdam elevation adjustment and diversion tunnel opening and closing timing. The former is represented by discrete levels of elevation adjustment, and the latter is represented by binary decision-making of the opening and closing states.

[0110] Diversion schemes are typically adjusted on a ten-day basis, while construction resource allocation is adjusted on a daily or shift basis, resulting in a time-scale difference between the two types of decisions. Therefore, a hierarchical time-scale semi-Markov decision process is used for modeling: the decision cycle of the diversion control suggestion action subspace. The decision cycle for the construction resource scheduling action subspace is 10 days. For 1 day. Recommended action for flow control is... It remains unchanged throughout the period, serving as the background constraint for construction resource scheduling actions.

[0111] Training employs a hierarchical reinforcement learning architecture, with each upper-layer policy network... The lower-level policy network outputs a suggested flow control action once a day, and under the condition that the upper-level action is fixed, it outputs a suggested action once a day. Output one construction resource scheduling action per day. Rewards from the upper-level policy network. The sum of the discounts on the cumulative rewards of the lower levels within the decision-making cycle is taken as shown in equation (15):

[0112]

[0113] in: For the upper-layer policy network in Cumulative rewards within the period. This represents the number of lower-level decision steps contained within one upper-level cycle; in this embodiment, it is 10. The discount factor is set to 0.99. For the first The total reward value obtained in each lower-level decision step is defined by Equation (8). Equation (15) propagates the lower-level scheduling effect upwards, enabling the upper layer to consider the comprehensive impact of the diversion control suggestion on multiple subsequent construction scheduling decisions when generating the diversion control suggestion. The two-layer policy network is trained and updated alternately. This hierarchical modeling approach incorporates the slow variable of diversion control and the fast variable of construction resource scheduling into a unified optimization framework, enabling the decisions of the two to be coordinated despite differences in time scale.

[0114] In a preferred embodiment, the scheduling strategy training employs a meta-reinforcement learning method. First, a hydrological scenario set is generated based on historical hydrological data: historical hydrological records of the watershed where the construction is located and similar watersheds are collected. K-means clustering analysis is performed according to three characteristics of the flood process: peak flow, duration, and rise and fall rate. The number of clusters is set to 5, corresponding to five types: typical year, dry year, general flood year, major flood year, and extraordinary flood year. Representative process lines are selected from each category, and variants are generated by applying random perturbations with a range of ±15%. 20 process lines are generated for each category, for a total of 100 process lines.

[0115] The meta-policy is trained using a model-independent meta-learning algorithm on this scenario set. The training consists of two layers: an inner loop and an outer loop. In the inner loop, the meta-policy parameters are updated with a small number of gradients on each hydrological scenario to obtain local adaptation parameters. The update rule is shown in Equation (16):

[0116]

[0117] in: In the first The adaptive parameters are updated by the internal circulation gradient for each hydrological scenario. These are the initial parameters of the meta-strategy, i.e., the starting point of the common strategy parameters shared by all hydrological scenarios. The learning rate for the inner loop is taken as [value missing] in this embodiment. . In the first Parameters used in a hydrological scenario The policy loss obtained is defined as the negative value of the cumulative reward in this scenario. Indicates to Calculate the gradient. In this embodiment, the inner loop performs 5 steps of gradient update.

[0118] The outer loop summarizes the losses after adaptation for each scenario, and applies the initial parameters of the meta-policy. Perform a global update, with the update rules as shown in equation (17):

[0119]

[0120] in: The outer loop learning rate is taken as [value missing] in this embodiment. Lower than the inner loop learning rate This ensures the stability of the meta-parameter updates. The number of hydrological scenarios sampled in each outer loop iteration is 5 in this embodiment. For the first Use adaptive parameters in a scenario Assess the obtained strategy loss. Defined by equation (16).

[0121] Equations (16) and (17) together constitute the two-layer optimization process of meta-learning. The inner loop simulates the process of "rapidly adapting to new hydrological conditions with a small amount of data", while the optimization goal of the outer loop is not to achieve the optimum in a single scenario, but to find a set of initial parameters. This allows for good scheduling performance with only a small number of gradient updates across any hydrological scenario. In this embodiment, the outer loop is trained for 1000 rounds.

[0122] Step S5 outputs the trained green production resource scheduling strategy network parameters. This is for online deployment in step S6.

[0123] In this embodiment of the invention, step S5 trains a reinforcement learning agent using an extended state vector as input, an action range limited by resource allocation constraints as the decision space, and a reward function as the feedback signal. This ensures that the actions output by the trained policy network under a given state always satisfy the constraints of the current scheduling mode, and that the actions are balanced among the multiple objectives of progress, cost, and green production according to the weight ratio defined by the reward function. The action masking mechanism sets the action probability of non-operational areas to zero at the output layer of the policy network, eliminating infeasible solutions from the policy representation level, so that there is no need to learn to avoid infeasible actions through trial-and-error-penalty loops during training. The hierarchical time-scale semi-Markov decision process incorporates the diversion control suggestions and construction resource scheduling with different decision cycles into the dual action space of the same policy network. The upper-layer cumulative reward is passed up to the lower-layer scheduling effect through equation (15), making the two types of decisions interrelated during training. Meta-reinforcement learning uses a two-layer optimization structure: the inner loop of Equation (16) allows for rapid adaptation in a single scenario, while the outer loop of Equation (17) seeks common initial parameters across multiple scenarios. This allows the trained meta-policy parameters to obtain the adaptive parameters for a new hydrological scenario with only a small number of gradient updates.

[0124] S6: Scheduling Scheme Generation: Utilizing green production resource scheduling strategies, and based on the real-time updated extended state vector and the resource allocation constraints corresponding to the current scheduling mode, a construction resource scheduling scheme is output. During the execution of the construction resource scheduling scheme, if the deviation between the real-time acquired predicted peak flow and the predicted peak flow used at the previous decision time exceeds a preset deviation threshold, the process returns to step S1 to reacquire data and sequentially executes steps S2, S3, and this step, outputting an updated construction resource scheduling scheme.

[0125] After entering the deployment and operation phase, the following operations are performed in each decision cycle. In this embodiment, the decision cycle is once a day. Step S1: Obtain real-time data and build... Calculate the hydrological risk level using formula (1) in step S2. Determine the scheduling mode and constraints according to step S3, and then... Input the trained policy network to generate actions within the constraints. This leads to the formulation of a daily construction resource scheduling plan. The plan includes instructions for allocating various types of construction machinery to different construction areas, work schedules for construction personnel of various trades, allocation plans for construction materials, and work procedures for each construction area.

[0126] In a preferred embodiment, when step S5 uses a meta-reinforcement learning method for training, the policy initialization in the deployment phase needs to complete two steps: scenario matching and online fine-tuning. Scenario matching is based on a similarity metric between real-time hydrological forecasts and historical scenarios in a hydrological scenario set, using a distance metric. Calculate according to formula (18):

[0127]

[0128] in: For real-time hydrological forecasting and the first The distance metric between historical scenarios is a non-negative dimensionless value; the smaller the value, the higher the degree of similarity. To predict peak traffic in real time, For the first Peak flow rates for each scenario are expressed in cubic meters per second. The standard flow rate for cofferdam design is defined in equation (1) and is used for normalization. To predict flow sequences in real time and the first The dynamic time-normalized distance between flow sequences in different scenarios measures the similarity of the process line shapes. This represents the maximum value of the dynamic time-warped distance within the scenario set, used for normalization. . To predict the duration of floods in real time, For the first The duration of each flood scenario is in hours. The maximum duration of the flood in the scenario set is used for normalization. . , and These are the weighting coefficients for peak flow rate, process curve shape, and duration, respectively. In this embodiment, the values ​​are 0.4, 0.35, and 0.25, respectively. The peak flow rate has the highest weight because the peak flow rate is directly related to whether the cofferdam overflows. Equation (18) comprehensively measures similarity from three dimensions and can distinguish between different scenarios with similar peak values ​​but different process line shapes.

[0129] Select The policy parameters corresponding to the smallest scenario After being loaded into the policy network, it adapts to the current hydrological conditions through online fine-tuning. Online fine-tuning... Starting from the state, action, and reward data acquired in real time over the past 24 hours, iterative gradient updates are performed, with the update rule shown in equation (19):

[0130]

[0131] in: For the first The policy parameters after fine-tuning From 1 to initial value . To fine-tune the learning rate, this embodiment takes... The learning rate is lower than the learning rate of the meta-learning inner loop. This is to avoid the parameters deviating too far from the generalization region. To fine-tune the number of steps, 10 is chosen in this embodiment. Among the comparisons of 5, 10, and 20 steps, the optimal compromise between convergence speed and stability is achieved. For the first The data batch used in this step is taken from the real-time data of the most recent 24 hours. In order to be in The objective function of the policy evaluated above. After fine-tuning, it is... This serves as a deployment strategy parameter. In the initial stages of system deployment, if real-time interactive data is less than 24 hours old, the matched historical scenario strategy will be used directly. The scheduling decision is made, and the online fine-tuning process is initiated only after accumulating 24 hours of real-time data.

[0132] During the execution of the construction resource scheduling plan, the system continuously monitors changes in hydrological forecast data. Normalized deviation value. Calculate according to formula (20):

[0133]

[0134] in: For a moment The normalized deviation value is a non-negative dimensionless value. This represents the predicted peak flow rate at the current moment, expressed in cubic meters per second. The predicted peak flow rate used at the previous decision-making time is expressed in cubic meters per second. The standard flow rate for cofferdam design is defined in equation (1). This means that the predicted change in peak flow reaches 15% of the standard flow rate of the cofferdam design.

[0135] when Exceeding the preset deviation threshold When the system returns to step S1 to reacquire data and executes steps S2, S3, and S6 sequentially, it outputs an updated construction resource scheduling plan. In this embodiment... The value is set to 0.15, which is twice the root mean square error of the reference watershed hydrological forecasting system. This mechanism allows the scheduling system to respond instantly when significant corrections occur in the hydrological forecast, without waiting for a fixed decision cycle.

[0136] In this embodiment of the invention, step S6 sequentially executes step S1 (data acquisition and state construction), step S2 (hydrological risk assessment), step S3 (scheduling mode switching), and the inference output of the strategy network in each decision cycle during the deployment phase, forming a periodic closed loop. The peak flow deviation monitoring mechanism calculates the normalized deviation value using equation (20). When the normalized deviation value exceeds the threshold, a rescheduling process starting from step S1 is triggered outside the fixed decision cycle, so that the update frequency of the scheduling scheme is adaptively adjusted according to the change in hydrological forecast. When using meta-reinforcement learning for training, during the deployment initialization phase, the strategy parameters of historical scenarios with the smallest distance from the current real-time hydrological conditions are matched from the hydrological scenario set and loaded into the strategy network using equation (18). Then, the strategy parameters are updated with gradients using the current real-time data through online fine-tuning using equation (19), so that the strategy parameters move from the generalization starting point of the historical scenario to the local optimum of the current hydrological conditions.

[0137] The steps S1 to S6 described above constitute a complete closed-loop scheduling process. During the deployment and operation phase, steps S1 to S3 and S6 are executed cyclically in each decision cycle. The reward function structure of step S4 remains fixed after being determined in the offline phase, and the policy training in step S5 is completed in the offline phase. When deviation monitoring in step S6 triggers rescheduling, the system re-enters the loop from step S1. This closed loop enables the green production resource scheduling strategy to be continuously updated along with the evolution of hydrological processes.

[0138] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A green production resource scheduling method for water conservancy project construction sites based on reinforcement learning, characterized in that, Includes the following steps: S1: Obtain construction resource status data and hydrological forecast data at the construction site of the water conservancy project, and fuse the construction resource status data with the hydrological forecast data to construct an extended state vector; S2: Calculate the hydrological risk level based on the hydrological forecast data, wherein the hydrological risk level includes a safety level, a warning level, and a danger level; S3: Based on the hydrological risk level, switch between the normal scheduling mode, accelerated scheduling mode and retreat scheduling mode, and determine the optimization target priority and resource allocation constraints corresponding to the current scheduling mode; S4: Construct a reward function, which includes green production indicators and a hydrological risk adjustment factor; the green production indicators include at least two dimensions: carbon emission intensity and construction water consumption rate; the hydrological risk adjustment factor initializes the weight ratio between the progress target, cost target and green production target according to the priority of the optimization target determined in step S3 based on the hydrological risk level and the current scheduling mode, and dynamically adjusts the weight ratio, and the weight of the green production target is not lower than the preset green baseline value; S5: Using the extended state vector as the input to the reinforcement learning agent, taking the construction resource allocation decision within the action range limited by the resource allocation constraints as the action, and taking the output of the reward function as the feedback signal, train the reinforcement learning agent to obtain a green production resource scheduling strategy. S6: Using the green production resource scheduling strategy, output a construction resource scheduling scheme based on the real-time updated extended state vector and the resource allocation constraints corresponding to the current scheduling mode; During the execution of the construction resource scheduling scheme, when the deviation between the real-time acquired predicted peak flow and the predicted peak flow used at the previous decision time exceeds the preset deviation threshold, the process returns to step S1 to acquire data again and executes steps S2, S3 and this step in sequence, outputting an updated construction resource scheduling scheme.

2. The method according to claim 1, characterized in that, In step S1, the hydrological forecast data includes the predicted flow sequence, water level change trend and rainfall probability within a preset time window; the construction resource status data includes the availability and current location of various types of construction machinery, the job distribution and number of construction personnel, the inventory and arrival plan of construction materials, and the current process progress of each construction area.

3. The method according to claim 1, characterized in that, In step S1, the extended state vector also includes the operational constraint information of the current diversion scheme for each construction area; in step S5, the operational constraint information is encoded into an action mask, and when the current diversion conditions cause a specific construction area to be unoperable, the action of allocating resources to that construction area is masked.

4. The method according to claim 1, characterized in that, In step S1, the extended state vector also includes meteorological condition data of the construction site; in step S3, when the meteorological condition data meets the extreme climate determination conditions, the meteorologically sensitive construction activities are restricted or suspended within the current scheduling mode, and the corresponding resources are allocated to meteorologically insensitive construction activities.

5. The method according to claim 1, characterized in that, In step S3: the conventional scheduling mode sets the green production target as the highest priority, and the resource allocation constraint allows all types of construction activities to participate in the scheduling; the accelerated scheduling mode sets the progress target as the highest priority, and the resource allocation constraint concentrates resources on key processes and tightens process time constraints; the evacuation scheduling mode sets the safety target as the highest priority, and the resource allocation constraint limits the scope of resource allocation to evacuation actions and temporary protection actions.

6. The method according to claim 1, characterized in that, In step S3, the construction site of the water conservancy project includes multiple construction areas, each of which has a different degree of hydrological risk exposure based on its location in the river channel. Different construction areas are allowed to implement differentiated scheduling modes according to their respective degrees of hydrological risk exposure. Construction areas closer to the river channel are given priority to enter the accelerated scheduling mode or the retreat scheduling mode, while construction areas farther from the river channel maintain the conventional scheduling mode.

7. The method according to claim 1, characterized in that, In step S4, the hydrological risk adjustment factor dynamically adjusts the weight ratio in the following manner: when the hydrological risk level rises from the safe level to the warning level, the weight of the progress target is increased and the weight of the cost target is reduced; when the hydrological risk level falls back from the dangerous level or the warning level to the safe level, a recovery efficiency reward item is introduced into the reward function to guide the reinforcement learning agent to prioritize the recovery of high-priority processes within the resumption window.

8. The method according to claim 1, characterized in that, In step S4, the green production indicator item also includes an ecological impact index; the ecological impact index is determined based on the wastewater discharge generated by construction activities and the ecological flow satisfaction rate of downstream rivers; when the ecological impact index exceeds a preset ecological threshold, the reward function applies a penalty signal.

9. The method according to claim 1, characterized in that, In step S5, the action space of the reinforcement learning agent includes a construction resource scheduling action subspace and a diversion control suggestion action subspace; the construction resource scheduling action subspace is used to determine the allocation scheme of construction machinery, manpower and materials among various construction areas; the diversion control suggestion action subspace is used to generate suggestions for cofferdam elevation adjustment and diversion tunnel opening and closing timing. A semi-Markov decision process with a hierarchical time scale is used for modeling, and the decision cycle of the diversion control suggestion action subspace is longer than that of the construction resource scheduling action subspace.

10. The method according to claim 1, characterized in that, Step S5 includes: generating a set of hydrological scenarios covering typical years, dry years, and years of extreme floods based on historical hydrological data; training a meta-policy on multiple scenarios in the set of hydrological scenarios using a meta-reinforcement learning method; in step S6, matching the closest historical scenario policy in the set of hydrological scenarios with real-time hydrological forecast data as the initial policy, and adapting it to the current hydrological conditions through online fine-tuning.

Images