Coastal flood control adaptive strategy optimization system based on reinforcement learning

Abstract

The invention discloses a coastal flood control adaptive strategy optimization system based on reinforcement learning, and the system comprises an unmanned plane multi-dimensional data collection module which carries out the cooperative work through a multi-rotor unmanned plane group, and carries out the data collection; the deep learning data processing module is used for processing the data acquired by the unmanned aerial vehicle multi-dimensional data acquisition module, updating a three-dimensional terrain model, flood routing prediction, a flooding simulation diagram and abnormal signal real-time detection in real time; the reinforcement learning strategy generation module is used for state space design, action space expansion and reward function optimization; the unmanned aerial vehicle execution and feedback module is used for strategy execution tracking, post-disaster evaluation and model iteration; and the dynamic optimization closed loop unit comprises a daily mode subunit and an emergency mode subunit. According to the invention, the flood control strategy is sensed in real time, accurate decision is made, and dynamic optimization of the coastal flood control adaptive strategy is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of flood monitoring, in particular to a coastal flood prevention adaptive strategy optimization system based on reinforcement learning. BACKGROUND

[0002] At present, reinforcement learning technology has been applied to coastal flood prevention strategy optimization, and through a dynamic decision-making model, the design of seawalls, multi-strategy cost control and other goals are achieved. The unmanned aerial vehicle technology has been applied in the field of flood monitoring, and can complete tasks such as dam inspection and submerged area aerial photography. However, the two technologies have not formed a deep integration and cooperation system. Reinforcement learning relies on pre-set sea level rise (SLR) and storm surge observation data to drive decision-making, and the application of unmanned aerial vehicles is mostly limited to the data collection stage, and has not formed a linkage with the algorithm model of strategy optimization.

[0003] Defects of the prior art: the timeliness and accuracy of the observation data are insufficient: the sea level change and storm surge data relied on by reinforcement learning have low spatial and temporal resolution, and are mostly discrete and periodic observation results, which are difficult to capture sudden environmental changes (such as short-term storm surge intensity surge and local dam deformation), resulting in a lag in environmental state perception and a time difference in strategy adjustment.

[0004] Disconnection between strategy simulation and actual execution: reinforcement learning completes strategy optimization through numerical simulation, but does not take into account deviation factors in the actual execution process, such as construction progress lag, insufficient cooperation of residents in the evacuation area, etc. The difference between the simulation results and the real scene may lead to poor landing effect of the optimized strategy.

[0005] Weak emergency response capability for extreme events: for low-probability, high-impact extreme hydrological events, existing strategies mostly rely on pre-set schemes and lack a rapid adjustment mechanism based on real-time dynamic data, making it difficult to respond to complex and variable emergency situations. SUMMARY

[0006] In view of the deficiencies in the prior art, the present application aims to provide a coastal flood prevention adaptive strategy optimization system based on reinforcement learning, which builds a closed-loop system of "unmanned aerial vehicle real-time perception - deep learning processing - reinforcement learning decision - unmanned aerial vehicle execution feedback", solves the problems of discrete observation data, strategy update lag and simulation and reality deviation in the prior art, and realizes dynamic optimization of the adaptive strategy of coastal flood prevention. In order to achieve the above-mentioned purposes and other advantages according to the present application, a coastal flood prevention adaptive strategy optimization system based on reinforcement learning is provided, comprising: An unmanned aerial vehicle multi-dimensional data acquisition module: a multi-rotor unmanned aerial vehicle group works cooperatively to acquire data; A deep learning data processing module is configured to process data collected by the unmanned aerial vehicle multi-dimensional data collection module, update a three-dimensional terrain model, flood evolution prediction, inundation simulation map, and abnormal signal real-time detection in real time; A reinforcement learning strategy generation module is configured to design a state space, expand an action space, and optimize a reward function; An unmanned aerial vehicle execution and feedback module is configured to track policy execution, conduct post-disaster evaluation, and iterate a model; A dynamic optimization closed loop unit includes a daily mode subunit and an emergency mode subunit.

[0007] Preferably, each of the multi-rotor unmanned aerial vehicles is installed with a LiDAR, a hyperspectral camera, and a millimeter wave radar, wherein the LiDAR is configured to collect coastal terrain data with centimeter-level precision, and supplement the deficiencies of 1-foot resolution DEM data; The hyperspectral camera is configured to identify flood inundation areas and building damage levels; The millimeter wave radar is configured to penetrate rain and fog, and continuously monitor water level changes during storm surges.

[0008] Preferably, each of the multi-rotor unmanned aerial vehicles dynamically plans a route through a reinforcement learning task allocation model, specifically as follows: Daily monitoring: covering the coastal area in a grid, completing a full-area scan once a day; Extreme event response: when the SLR observation value breaks through the preset threshold, the multi-rotor unmanned aerial vehicle group switches to a high-density patrol mode, sets one monitoring point every 500 meters on the seawall section, and updates data once every 10 minutes.

[0009] Preferably, the real-time update of the three-dimensional terrain model specifically includes the following steps: Data preprocessing: normalizing and denoising the point cloud data collected by the LiDAR, and retaining key regional point sets of the dam surface and coastal terrain; Feature extraction and three-dimensional reconstruction: using an improved CNN model based on PointNet++, extracting local geometric features through hierarchical sampling and feature aggregation, and generating a three-dimensional grid model; identifying subsidence areas by calculating elevation differences through a sliding window; Dynamic update mechanism: collecting point cloud data every 2 hours, comparing with historical models through a registration algorithm, generating a deformation heat map, and updating the three-dimensional terrain model in real time.

[0010] Preferably, the flood evolution prediction specifically includes the following steps: Data input layer: integrating minute-level time series data monitored by the unmanned aerial vehicle and historical hydrological data of the same period in the past 5 years, wherein the minute-level time series data includes water level, wind speed, and SLR observation value interpolated to hourly level; Model structure: The model adopts a Transformer encoder-decoder architecture with position encoding. The encoder captures the spatio-temporal correlation of multiple variables through self-attention mechanism, and the decoder outputs the predicted flood level curve for the next 12 hours. Precision control: The model is initialized with parameters obtained through transfer learning, and the average absolute error on the validation set is less than or equal to 0.2 meters.

[0011] Preferably, the flood simulation map specifically includes the following steps: Training data construction: Based on the SLR trajectory and storm surge data under the SSP5-8.5 scenario, samples of predetermined intensity are selected; Model generation design: A conditional generative adversarial network is used, with SLR rate, storm surge peak, and coastal terrain as conditional variables. The generator outputs a 5m x 5m resolution flood range simulation map through a U-Net structure, and the discriminator distinguishes between generated maps and real historical data. Scenario expansion effect: An additional 20,000 extreme coupled scenario trajectories are generated, covering the synergistic enhancement effect of SLR and storm surge.

[0012] Preferably, the implementation of real-time anomaly signal detection specifically includes the following steps: Model training: A variational autoencoder is used to learn the normal feature distribution based on normal hydrological state data. Anomaly determination: Real-time input of monitoring data, calculation of reconstruction error, and determination of anomaly when exceeding the 99.9% confidence interval threshold. Emergency triggering: Anomaly signals are transmitted to the reinforcement learning model through an API interface, triggering the emergency decision branch, with a response delay of less than or equal to 5 minutes.

[0013] Preferably, the strategy execution tracking specifically includes: sea dike construction monitoring: unmanned aerial vehicles take pictures of the construction area every day, and use target detection algorithms to count the length of poured concrete and the density of steel reinforcement, and compare them with the sea dike construction cost model. If the actual cost exceeds the budget by 10%, send a correction signal to the reinforcement learning model. Evacuation zone management: The number of evacuated households is counted through face recognition, and a forced evacuation priority map is generated based on house identification, which serves as the basis for adjusting subsidy policies.

[0014] Preferably, the post-disaster assessment and model iteration specifically include: within 24 hours after the flood recedes, unmanned aerial vehicles take pictures of the entire area, and use a semantic segmentation model to calculate the actual flood loss, which is compared with the reinforcement learning prediction value. The loss bias value is used to correct the value function parameters of the reinforcement learning, improving the fit of the next round of strategies.

[0015] Preferably, the daily mode subunit specifically includes the integration of unmanned aerial vehicle data updates every hour, and the reinforcement learning model outputs the next day's strategy. The emergency mode subunit specifically includes triggering a minute-level update when the SLR rate exceeds the 90% quantile of the SSP5-8.5 scenario, specifically: the UAV swarm focuses on high-risk areas, and returns data once every 3 minutes; the reinforcement learning model generates an emergency strategy within 5 minutes based on the latest data; the UAV executes real-time feedback of the effect, and the reinforcement learning model immediately updates the reward value and adjusts the subsequent action.

[0016] The beneficial effects of the present application compared to the prior art: improving the timeliness and accuracy of the strategy: the combination of real-time UAV data and reinforcement learning upgrades the strategy adjustment period from a phased observation-driven to a minute-level dynamic response. For example, the water level monitoring frequency of 1 time / minute of the millimeter wave radar can quickly capture the rapid increase in short storm surge intensity, and the abnormal detection of AutoEncoder, that is, the response delay is less than or equal to 5 minutes, significantly reducing the decision loss caused by state perception lag.

[0017] Enhancing the reliability of strategy landing: The closed-loop feedback mechanism effectively corrects the deviation between simulation and reality by tracking actual execution data such as construction progress and resident transfer rate of the UAV. For example, real-time correction signals when the seawall construction cost exceeds the budget by 10% can make the reinforcement learning strategy more suitable for the actual scene and improve the landing effect.

[0018] Enhancing the extreme risk prevention and control capability: The 20000 extreme coupled scenario trajectories generated by GAN provide more comprehensive training samples for reinforcement learning, combined with the minute-level response mechanism in the emergency mode, which can significantly improve the disposal efficiency of low-probability and high-impact events. For example, for extreme scenarios of high SLR value superimposed with storm surge peak value, the system can quickly dispatch UAVs to guide evacuation and reduce life and property losses. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 The structure diagram of the reinforcement learning-based coastal flood prevention adaptive strategy optimization system according to the present application. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0021] System overall linkage logic: The system realizes dynamic optimization through the closed loop of UAV data acquisition → deep learning processing → reinforcement learning decision → UAV execution feedback, and each link interacts in real time through a data interface such as API and MQTT protocol. The core linkage relationship is as follows: The raw data collected by the unmanned aerial vehicle is input into the deep learning module after preprocessing, and the characteristic environment state is output, such as a three-dimensional terrain model and a flood prediction curve; the raw data includes LiDAR point cloud and water level time series.

[0022] The deep learning module transmits the environment state and the abnormal signal to the reinforcement learning module as the input of the strategy generation; The optimization strategy output by the reinforcement learning module is issued to the unmanned aerial vehicle group through the ground control center; the optimization strategy includes a temporary reinforcement scheme and an evacuation instruction.

[0023] When the unmanned aerial vehicle executes the strategy, the execution data such as construction progress and transfer rate are collected synchronously and fed back to the deep learning module for deviation analysis, and finally the reinforcement learning model parameters are corrected.

[0024] Reference Figure 1 A coastal flood prevention adaptive strategy optimization system based on reinforcement learning, comprising: A multi-dimensional data acquisition module of an unmanned aerial vehicle: data acquisition is performed by a multi-rotor unmanned aerial vehicle group working collaboratively; each multi-rotor unmanned aerial vehicle is installed with a LiDAR, a hyperspectral camera and a millimeter wave radar, wherein the LiDAR is used to collect coastal terrain data with centimeter-level precision, supplementing the deficiency of 1-foot resolution DEM data, and the coastal terrain data includes dam deformation and sand erosion rate; the hyperspectral camera is used to identify flood inundation areas and building damage levels, including distinguishing between standing water and wet ground, and building damage levels including roof damage and wall soaking; the millimeter wave radar is used to penetrate rain and fog, and continuously monitor water level changes during storm surges, with a sampling frequency of 1 time / minute, far exceeding the traditional observation interval of 10 years.

[0025] Each multi-rotor unmanned aerial vehicle dynamically plans a route through a reinforcement learning task allocation model, specifically as follows: Daily monitoring: the coastal area is covered by grid, and full-area scanning is completed once a day, and the coastal area refers to the sea embankment and the surrounding evacuation area and adaptation area; Extreme event response: when the SLR observation value breaks through the preset threshold, such as exceeding the 50% quantile under the SSP5-8.5 scenario, the multi-rotor unmanned aerial vehicle group switches to a high-density inspection mode, and one monitoring point is set every 500 meters along the sea embankment, and data is updated every 10 minutes.

[0026] A deep learning data processing module: used for processing the data collected by the unmanned aerial vehicle multi-dimensional data acquisition module, and updating a three-dimensional terrain model, a flood evolution prediction, a flooding simulation map and an abnormal signal in real time; the real-time updating of the three-dimensional terrain model specifically includes the following steps: Data preprocessing: The point cloud data collected by LiDAR is normalized and denoised, and the key area point set such as dam surface and coastal terrain is retained; the point density is ≥50 points / ㎡.

[0027] Feature extraction and three-dimensional reconstruction: an improved CNN model based on PointNet++ is used to extract local geometric features through hierarchical sampling and feature aggregation, and generate a three-dimensional grid model; the elevation difference is calculated through a sliding window to identify settlement areas and cracks, with an elevation difference of ≥3cm for settlement areas and a width of ≥2mm for cracks; the geometric features include normal vector and curvature change; Dynamic updating mechanism: point cloud data is collected every 2 hours, and a deformation heat map is generated by comparing with the historical model through registration algorithm, i.e. an accuracy threshold of ±3cm, and the three-dimensional terrain model is updated in real time.

[0028] Further, the flood evolution prediction specifically includes the following steps: Data input layer: integrate the minute-level time series data monitored by the unmanned aerial vehicle, i.e. water level, wind speed, SLR observation value, and interpolate it to the hour level and the historical hydrological data of the same period in the past 5 years; Model structure: adopt a Transformer encoder-decoder architecture with position encoding, the encoder captures the multi-variable spatio-temporal correlation through self-attention mechanism, and the decoder outputs the future 12-hour flood water level prediction curve; Precision control: initialize the parameters through transfer learning, i.e. the pre-training data set is the regional hydrological data in the past 20 years, and the average absolute error on the validation set is ≤0.2 meters.

[0029] Further, the submergence simulation map specifically includes the following steps: Training data construction: based on the SLR trajectory and storm surge data under the SSP5-8.5 scenario, select samples of predetermined intensity; such as SLR growth rate ≥0.2m / decade, storm surge increase ≥3m; Generate model design: adopt conditional generative adversarial network, take SLR rate, storm surge peak, and coastal terrain as conditional variables, and the generator outputs a 5m×5m resolution submergence range simulation map through U-Net structure, and the discriminator distinguishes between generated maps and real historical data, such as measured data during Hurricane Sandy; Scenario expansion effect: generate an additional 20,000 extreme coupled scenario trajectories, covering the synergistic enhancement effect of SLR and storm surge.

[0030] Further, the implementation of real-time detection of abnormal signals specifically includes the following steps: Model training: use variational autoencoder, take normal hydrological state data as sample, i.e. water level fluctuation ≤0.5m / h, dam deformation ≤1cm / 24h, learn normal feature distribution; Abnormality determination: real-time input monitoring data, calculate reconstruction error, more than 99.9% confidence interval threshold, such as dam elevation drop ≥50cm, water level rise ≥1m in 10 minutes; Emergency trigger: abnormal signal is transmitted to the reinforcement learning model through the API interface, triggering the emergency decision branch, with a response delay of ≤5 minutes.

[0031] Reinforcement learning strategy generation module: for state space design, action space expansion, and reward function optimization; state space design: based on SLR + storm surge + seawall height, new real-time features monitored by drones are added: Continuous variables: flooded area (km²), dam deformation rate (mm / hour), evacuation zone resident transfer rate (%); Discrete variables: building damage level, emergency supplies reserve, i.e. sufficient / insufficient, the level is 0-4.

[0032] Action space expansion: Short-term actions: temporary reinforcement guided by drones, such as sandbag placement location, quantity, emergency drainage pump scheduling; Long-term actions: seawall heightening pace adjustment based on real-time SLR observations, such as advancing the planned 2030 heightening to 2028, and dynamically sorting the priority of evacuation zone acquisition.

[0033] Reward function optimization: Basic reward: -(policy execution cost + flood loss); the basic reward focuses on minimizing the expected net cost; during the execution of coastal flood control strategies, two key costs need to be balanced: one is the direct cost of strategy execution itself, such as the cost of building seawalls and scheduling emergency equipment, the other is the indirect loss that may be caused by floods if the strategy does not meet expectations, such as economic losses from house flooding and farmland destruction. The basic reward is formed by taking the negative of the sum of these two costs, which is the basic income indicator of the model, i.e. when the strategy chosen by the model can reduce both execution cost and flood loss, the basic reward value will be higher, i.e. the negative degree will be reduced, otherwise the reward value will be lower, i.e. the negative degree will be increased, to guide the model to iterate towards the direction of minimizing the expected net cost; Additional reward: |actual progress - planned progress|; the purpose of the additional reward is to solve the problem of disconnection between reinforcement learning strategy simulation and actual execution, i.e. some strategies may have low cost and small loss in simulation, but actual execution may be delayed due to progress lag, resulting in discounted flood control effect. Progress lag includes delayed acquisition of evacuation zones and delayed construction of seawalls. By negatively quantifying the deviation between actual progress and planned progress, lag behavior is punished, guiding the model to generate strategies that can be implemented and timely promoted. Specific explanations are as follows: When the reinforcement learning model generates a strategy, it also formulates a plan schedule, such as completing the evacuation of 80% of residents in a high-risk area within 3 hours and completing a certain section of seawall heightening project within 1 month. However, in actual execution, it may be affected by factors such as low resident cooperation and harsh construction environment, resulting in progress lag. The additional reward is calculated by taking the absolute difference between the actual progress and the planned progress and taking the negative, forming a progress constraint reward. When the actual progress is completely consistent with the planned progress, the additional reward is 0, i.e. no punishment. When the actual progress lags or is too far ahead, causing resource waste, the absolute difference increases, and the additional reward value decreases, i.e. the negative degree increases, thereby forcing the model to fully consider the execution difficulty in strategy design and avoid idealistic decisions.

[0034] The UAV execution and feedback module is used for strategy execution tracking and post-disaster evaluation and model iteration. The strategy execution tracking specifically includes: seawall construction monitoring: the UAV takes pictures of the construction area every day, and uses a target detection algorithm to count the length of the poured concrete and the density of the steel reinforcement, and compares it with the seawall construction cost model. If the actual cost exceeds the budget by 10%, send a correction signal to the reinforcement learning model. The seawall construction cost model is a specialized model deeply coupled with the UAV multi-dimensional data acquisition module, deep learning data processing module, and reinforcement learning strategy generation module. Its construction process closely relies on the core technical logic of the system, and the specific process is as follows: 1. Basic data sources: integrate internal and external data The basic data for model construction includes two types: one is real-time scene data collected by the UAV multi-dimensional data acquisition module of the system, including the complexity of the coastal construction area, i.e. the centimeter-level terrain data determined by LiDAR, the tidal influence coefficient calculated by millimeter wave radar monitoring of water level changes, and the regional building material transportation distance determined by combining UAV aerial photography of the construction area surrounding road network data; the second is historical seawall construction project data, covering seawall engineering budget, actual construction cost, concrete unit consumption cost, steel reinforcement laying density corresponding material cost, i.e. labor and equipment cost proportion data of different scales and different geographical environments in the past 5 years, i.e. the same source management as the historical hydrological data stored in the deep learning data processing module of the system.

[0035] 2. Model construction algorithm: combine deep learning and statistical analysis Use a two-step method of statistical regression + deep learning calibration to build the model: First, through multivariate linear regression analysis, establish the basic mathematical relationship between "concrete length-concrete material cost" and "steel reinforcement laying density-steel material cost", where the independent variable is the length of the poured concrete and the density of the steel reinforcement counted by the UAV, and the dependent variable is the corresponding material cost. The regression coefficients are determined by fitting historical project data, such as unit length of concrete pouring cost and unit area of steel reinforcement laying cost.

[0036] Second, the time series analysis capability of the deep learning data processing module of the system is used to dynamically calibrate the basic model: input the construction environment parameters collected by the unmanned aerial vehicle in real time into the CNN model, the construction environment parameters include the construction efficiency change caused by the high and low tide level, the influence of the terrain slope on the equipment loss; learn the correlation between the environment parameters and the labor cost, equipment cost, generate an environment correction coefficient, and integrate the coefficient into the basic model, and finally obtain a seawall construction cost model that can adapt to the construction scene in real time. For example, when the unmanned aerial vehicle monitors that the construction area tide is 1m higher than the historical average, the model outputs a labor cost correction coefficient of 1.2 through deep learning, so that the cost calculation result is more in line with the actual construction situation.

[0037] 3. Model parameter iteration: relying on system feedback closed loop update The core parameters of the model will be continuously iterated through the post-disaster evaluation data of the unmanned aerial vehicle execution and feedback module: after the flood recedes, the actual construction loss data of the unmanned aerial vehicle aerial photography will be fed back to the deep learning module, and by comparing the deviation between the model predicted cost and the actual cost, the regression coefficient and the correction coefficient of the model are corrected to ensure that the model accuracy continues to improve with the system running. The core parameters include material cost unit price, environment correction coefficient weight; the actual construction loss data includes concrete actual consumption and budget deviation.

[0038] The associated logic of the seawall construction cost model and the technical features of the present application is as follows: In the present application, unmanned aerial vehicle photography → target detection statistics → cost model comparison → reinforcement learning correction to form a complete strategy execution tracking link, the association of each technical feature is completely realized relying on the closed loop architecture of the system, as follows: 1. Data acquisition and feature extraction: linkage of unmanned aerial vehicle and target detection algorithm According to the unmanned aerial vehicle of the present application in the daily monitoring mode, the coastal area is covered by grid, and the whole area is scanned once a day. After the construction area image is photographed every day, the image data is transmitted to the deep learning data processing module; the module calls the target detection algorithm, that is, the feature extraction network shared with the CNN model processing LiDAR point cloud, based on the feature extraction logic of the improved CNN model of PointNet++, the outline of the poured concrete area and the distribution area of the steel bar laying are identified from the image, through the mapping of pixel coordinates and actual geographic coordinates, that is, relying on the centimeter-level terrain data collected by LiDAR to realize coordinate calibration, the poured concrete length and steel bar laying density are calculated, the poured concrete length is the outline perimeter or linear extension length. The steel bar laying density is the actual density converted from the pixel ratio of steel bar per unit area.

[0039] 2. Cost calculation and comparison: linkage of target detection results and cost model After inputting the above statistical data into the seawall construction cost model, the model calculates the current construction cost according to the following logic: Material cost = length of poured concrete × unit length of concrete cost × environmental correction factor + steel reinforcement laying area × unit area of steel reinforcement cost × environmental correction factor; the steel reinforcement laying area is calculated by the steel reinforcement laying density and the construction area; Total cost = material cost + (material cost × labor cost proportion + material cost × equipment cost proportion), wherein the labor cost proportion and the equipment cost proportion are determined by the model based on historical data and real-time environmental parameters.

[0040] After the calculation is completed, the model compares the current total cost estimate with the seawall construction budget cost preset by the system, i.e., the reward function logic of the base reward: -(strategy execution cost + flood loss) set by the reinforcement learning strategy generation module according to the expected net cost minimization goal.

[0041] 3. Feedback correction: linkage of cost comparison results and reinforcement learning model If the comparison finds that the actual cost exceeds the budget by 10%, the seawall construction cost model sends a correction signal to the reinforcement learning strategy generation module through an API interface; after receiving the signal, the reinforcement learning model adjusts the construction strategy parameters in the action space, for example: if the overspending is due to excessive use of concrete, the concrete pouring schedule is optimized, i.e., reducing the daily pouring amount, adjusting the pouring process, or adjusting the seawall heightening pace, i.e., long-term action: adjusting the seawall heightening pace based on real-time SLR observation to ensure that the subsequent construction cost returns to the budget range, realizing the closed loop of strategy execution tracking → cost deviation feedback → strategy optimization.

[0042] Evacuation zone management: through face recognition to count the number of evacuated households, generate a forced evacuation priority map based on house identification, and use it as the basis for adjusting subsidy policies.

[0043] (I) Specific implementation of face recognition to count the number of evacuated households In the present invention, face recognition to count the number of evacuated households relies entirely on the hardware and algorithm capabilities of the unmanned aerial vehicle execution and feedback module, without the need for additional deployment of independent devices. The specific process is as follows: 1. Data acquisition: unmanned aerial vehicle equipped with face recognition function to realize air-space-ground cooperation According to the unmanned aerial vehicle has been equipped with hyperspectral camera, on this basis increases the lightweight face recognition algorithm module, with the hyperspectral camera sharing image acquisition channel, in the daily monitoring or emergency mode, that is, the high-density inspection under the threshold of SLR rate, the low-altitude aerial photography of the evacuation area house, that is, the flight height is controlled in 5-10m, to ensure that the face is clear and identifiable; for high-rise building windows and tree sheltering and other unmanned aerial vehicle aerial photography blind area, through the blind area positioning map of unmanned aerial vehicle aerial photography, deploy temporary cameras at key nodes on the ground, and the camera parameters match the unmanned aerial vehicle image acquisition parameters, to ensure the consistency of the face features, the key nodes on the ground include building entrances, forming an acquisition mode of unmanned aerial vehicle aerial photography as the main and ground supplement as the auxiliary.

[0044] 2. Data processing: household number statistics in conjunction with housing identification database The system pre-establishes a "household identification-resident information association database": each building is assigned a unique digital code, such as "HT-XX-XX", the code contains the coordinate information of the building location, which is consistent with the terrain data coordinates collected by LiDAR, and the number of residents in the building, the resident face features, and whether the information of the vulnerable groups, such as the elderly, children, and disabled people, is stored with the code. The data is stored in the database of the deep learning data processing module, and is managed synchronously with the historical hydrological data.

[0045] When face recognition, the face information collected by unmanned aerial vehicle or ground camera is transmitted to the deep learning data processing module and compared with the resident face features in the database: if the faces of all registered residents in a house are not recognized or the proportion of recognized residents is less than 10%, it is determined that the house has been evacuated; if the proportion of recognized residents is higher than 90%, it is determined that it has not been evacuated; by counting the number of evacuated houses, the number of evacuated households is obtained.

[0046] (II) Generation logic of forced evacuation priority map The generation of forced evacuation priority map takes the real-time data of unmanned aerial vehicle multi-dimensional data acquisition module and the prediction data of deep learning data processing module as the core basis to ensure the scientificity and timeliness of priority division, the specific steps are as follows: 1. Risk level assessment: combined with flood prediction and housing status data First, obtain two types of key data output by the deep learning data processing module: one is the flood evolution prediction curve, that is, the future 12-hour flood level prediction curve, to determine the predicted flooding time and depth of each house area; the second is the flooding simulation map, that is, the 5m x 5m resolution flooding range simulation map, to determine whether the house is in the flooding core area.

[0047] Secondly, combined with the number of evacuated households and housing identification information: for the number of households with high evacuation rate, the number of vulnerable groups is large, and the housing is marked as high attention housing; for the housing structure is fragile, that is, the building damage degree identified by the hyperspectral camera, such as wall soaking and roof damage, is marked as high-risk housing.

[0048] Finally, the risk value of each house is calculated by the formula: risk value = flood emergency degree x house vulnerability x non-evacuation coefficient: the flood emergency degree is determined by the predicted flood time, such as 1.0 for 3 hours, 0.6 for 6 hours, the house vulnerability is determined by the building damage degree, 1.0 for severe damage, and 0.3 for perfect, and the non-evacuation coefficient is determined by the proportion of non-evacuation households, 1.0 for 100% non-evacuation, and 0.5 for 50% non-evacuation.

[0049] 2. Map generation: visualization based on GIS technology The above risk value is divided into three priorities: risk value ≥ 0.8 is the first priority, i.e. evacuation needs to be completed within 1 hour, 0.5 ≤ risk value < 0.8 is the second priority, i.e. evacuation needs to be completed within 3 hours, and risk value < 0.5 is the third priority, i.e. evacuation needs to be completed within 6 hours; using geographic information system technology, the house code is associated with the corresponding priority, and different priority houses are marked with red for the first, yellow for the second and blue for the third on the regional map of unmanned aerial vehicle aerial photography, to generate a forced evacuation priority map, and real-time synchronization to the reinforcement learning strategy generation module and the ground command system.

[0050] (Three) associated logic with adjustment of subsidy policy The forced evacuation priority map is the core input of the reinforcement learning strategy generation module to formulate the subsidy policy, and its associated logic fully fits the design concept of dynamic optimization closed loop of the system, i.e. dynamic optimization closed loop unit, as follows: 1. Subsidy standard formulation: differential policy based on priority division The reinforcement learning model sets differential subsidy rules according to the priority of the map: for the first priority area residents, the subsidy amount includes emergency transfer transportation subsidy, which is 50% higher than the second priority, temporary resettlement subsidy, which is 3 times the daily standard per capita, and additional fast evacuation bonus, which can be received within 1 hour; for the second priority area residents, the subsidy amount is calculated according to the regular emergency transfer standard; for the third priority area residents, the subsidy amount is calculated according to the basic evacuation subsidy standard.

[0051] 2. Adjustment of subsidy policy: iteration of evacuation progress relying on unmanned aerial vehicle feedback The evacuation progress of each priority area is counted by the UAV and feedback module every 30 minutes, the number of evacuated households is updated through face recognition, and the progress data is fed back to the reinforcement learning model: if the evacuation progress of a priority area is less than 50%, that is, it does not meet the expected value, the model will immediately adjust the subsidy policy, such as increasing the amount of quick evacuation bonus or increasing the material subsidy, such as providing emergency food and medicine; if the evacuation progress of a region is ahead of schedule, such as completing the evacuation of a secondary priority area within 2 hours, the model will allocate the remaining subsidy quota of this region to the evacuation lagging region, realizing the dynamic optimization of subsidy resources.

[0052] The above adjustment logic is completely consistent with the minute-level update feature of the emergency mode subunit of the system, that is, when the SLR rate exceeds the 90th percentile of the SSP5-8.5 scenario, an emergency strategy is generated within 5 minutes, ensuring that the subsidy policy can quickly respond to evacuation needs and improve overall evacuation efficiency.

[0053] Further, the post-disaster assessment and model iteration specifically includes that within 24 hours after the flood recedes, the UAV takes aerial photos of the entire region, uses a semantic segmentation model to calculate the actual flood loss, and compares it with the predicted value of the reinforcement learning; The loss deviation value is used to correct the value function parameters of the reinforcement learning, and the fitting degree of the next round of strategy is improved, such as increasing the evacuation weight for areas prone to water accumulation.

[0054] The dynamic optimization closed loop unit includes a daily mode subunit and an emergency mode subunit. The daily mode subunit specifically includes updating the state every hour by fusing UAV data, and the reinforcement learning model outputs the next day's strategy; The emergency mode subunit specifically includes triggering minute-level updates when the SLR rate exceeds the 90th percentile of the SSP5-8.5 scenario, specifically: the UAV group focuses on high-risk areas and transmits data back every 3 minutes; the reinforcement learning model generates an emergency strategy within 5 minutes based on the latest data; the UAV executes real-time feedback, and the model immediately updates the reward value and adjusts the subsequent action.

[0055] Embodiment 1: 1. UAV data collection and deep learning module linkage Data transmission and trigger conditions: The UAV packages the raw data at a preset frequency, that is, in JSON format, including timestamp, device ID, and sensor data, the preset frequency is 1 time / hour in daily mode and 1 time / 3 minutes in emergency mode, and the data is transmitted to the deep learning server through 5G. When the data volume reaches the threshold or an abnormal signal is triggered, the threshold is LiDAR point cloud ≥ 1 million points, the deep learning module starts the real-time processing process.

[0056] Three-dimensional terrain model update, that is, CNN and UAV linkage: Let the LiDAR point cloud collected by the UAV at time t be Pt, and the historical model be M t-1 Calculate the deformation difference through the registration algorithm: ; Where M t(x,y) is the elevation value of the three-dimensional model at time t, and when , it is marked as a deformation area and the heat map is updated. This heat map is synchronized in real time to the state space of the reinforcement learning module through shared memory. Flood prediction and UAV sampling frequency linkage: The Transformer model outputs the predicted water level value for the next 12 hours When the prediction error , z(t) is the current measured water level, triggering the UAV sampling frequency to increase: ; For example, the original frequency of 1 time / minute can be increased to 2 times / minute until the prediction error falls within the threshold.

[0057] 2. Linkage between deep learning and reinforcement learning modules Abnormal signal triggers emergency decision-making, i.e., the linkage between AutoEncoder and reinforcement learning: When AutoEncoder detects abnormal patterns in the environment state, including sudden changes in dam seepage and rapid increases in water level, it transmits the compressed and encoded abnormal feature vector to the reinforcement learning module. Based on Markov decision process and in combination with the preset risk threshold, the module maps the abnormal features to the emergency strategy trigger condition. The module builds an anomaly detection mechanism based on variational autoencoder, accurately identifies abnormal patterns in the environment state by calculating the reconstruction error of real-time data. Specifically: Variational autoencoder calculates the reconstruction error of real-time data: ; Where x i is the input feature, such as water level and dam deformation, is the reconstructed feature, and D is the feature dimension. When the reconstruction error > θ, θ is the 99.9% confidence interval threshold, an abnormal signal Alert = 1 is generated, and the reinforcement learning module immediately switches to the emergency state space.

[0058] Extreme scenario expansion, namely the integration of generative adversarial networks and reinforcement learning: Generative adversarial networks generate historically rare extreme scenarios through adversarial training. Conditional generative adversarial networks, based on these simulations and the current environmental state, output extreme scenario simulation results that meet real-world constraints. The reinforcement learning module uses a policy gradient algorithm to adjust policy network parameters based on these simulation results and the current actual environmental state. This allows for preemptive planning for extreme scenarios, improving the system's decision-making robustness in complex scenarios. Extreme scenario trajectories generated by conditional generative adversarial networks , 100 high-risk trajectories are selected through Monte Carlo sampling and input into the reinforcement learning training pool to expand the strategy search space. Assume that the trajectory risk value is: ; Select The trajectory of is taken as the key training sample.

[0059] 3. Linking reinforcement learning with the drone execution module to generate linkage strategies and guide drone actions: Short-term actions output by the reinforcement learning module a t = (number of sandbags, stacking coordinates), converted into drone navigation instructions through coordinate conversion algorithm: GPS (a t ) = T(local coordinate(a t )) Where T is the conversion function from UTM coordinates to WGS84. When the drone is executing, the target detection algorithm is used to count the actual number of stackings nactual and calculate the execution deviation rate: ; When it is greater than 10%, it is fed back to the reward function of reinforcement learning: ; Where λ = 0.5 is the penalty coefficient, which drives the model to optimize the feasibility of subsequent actions.

[0060] Linkage between emergency evacuation and drone swarm dispatch: When the reinforcement learning decision is "evacuate 100 households in area A", the drone swarm uses facial recognition to calculate the transfer rate ρ = Number of evacuated households / 100, updated every 5 minutes. Assume that the evacuation completion threshold is ρ* = 90%. When ρ < ρ*, the model triggers additional actions: a 附加 = Increase the number of drone guidance*(ρ* - ρ).

[0061] 4. Post-disaster assessment and reinforcement learning parameter correction using feedback loops and model iterations: After the flood recedes, the UAV semantic segmentation result gets the actual loss L 实际 The bias of the reinforcement learning prediction loss L 预测 is: ; The learning rate for correcting the value function V(s) is: ; Where β=0.1, so that the model accelerates learning in states with large loss bias.

[0062] The number of devices and processing scale described herein are used to simplify the description of the present application, and it is obvious to those skilled in the art that the application, modification and change of the present application. Although the embodiments of the present application have been disclosed as above, it is not limited to the application listed in the specification and the embodiments, and it can be fully applied to various fields suitable for the present application, and other modifications can be easily realized by those skilled in the art. Therefore, the present application is not limited to specific details and figures shown and described herein, and the general concept defined by the claims and the equivalent scope.​

Claims

1. A coastal flood control adaptive strategy optimization system based on reinforcement learning, characterized by: include: The multi-dimensional data acquisition module of UAVs collects data through the collaborative work of multi-rotor UAVs; A deep learning data processing module is used to process data collected by the drone's multi-dimensional data acquisition module, and to update the 3D terrain model, flood evolution prediction, inundation simulation map, and abnormal signal real-time detection in real time; Reinforcement learning policy generation module for state space design, action space expansion, and reward function optimization; UAV execution and feedback module, used for strategy execution tracking, post-disaster assessment, and model iteration; A dynamic optimization closed-loop unit, wherein the dynamic optimization closed-loop unit includes a daily mode subunit and an emergency mode subunit.

2. A coastal flood control adaptive strategy optimization system based on reinforcement learning according to claim 1, characterized in that: Each multi-rotor drone is equipped with a LiDAR, a hyperspectral camera, and a millimeter-wave radar, wherein the LiDAR is used to collect coastal terrain data with centimeter-level accuracy to supplement the deficiency of 1-foot resolution DEM data; The hyperspectral camera is used to identify flooded areas and the extent of building damage; The millimeter-wave radar is used to penetrate rain and fog and continuously monitor water level changes during storm surges.

3. A coastal flood control adaptive strategy optimization system based on reinforcement learning as claimed in claim 2, characterized in that: Each multi-rotor drone dynamically plans its route through a reinforcement learning task allocation model, as follows: Daily monitoring: Cover the coastal area in a grid format, completing one full-area scan daily; Response to extreme events: When the SLR observation value exceeds the preset threshold, the multi-rotor drone swarm switches to "high-density inspection mode", setting up a monitoring point every 500 meters of the seawall section and updating the data every 10 minutes.

4. The coastal flood control adaptive strategy optimization system based on reinforcement learning according to claim 1, characterized in that: The real-time updating of the three-dimensional terrain model specifically includes the following steps: Data preprocessing: normalize and denoise the point cloud data collected by LiDAR, retaining point sets in key areas such as the dam surface and coastal terrain; Feature extraction and 3D reconstruction: Using an improved CNN model based on PointNet++, local geometric features are extracted through layered sampling and feature aggregation to generate a 3D mesh model. Elevation differences are calculated using a sliding window to identify subsidence areas. Dynamic update mechanism: Point cloud data is collected every 2 hours, compared with the historical model through the registration algorithm, and a deformation heat map is generated to update the 3D terrain model in real time.

5. The coastal flood control adaptive strategy optimization system based on reinforcement learning according to claim 4, characterized in that: The flood evolution prediction specifically includes the following steps: Data input layer: Integrate minute-level time series data monitored by drones and hydrological data from the same period of the past five years; Model structure: It uses a Transformer encoder-decoder architecture with positional encoding. The encoder captures multivariate spatiotemporal correlations through a self-attention mechanism, and the decoder outputs a 12-hour flood level forecast curve. Accuracy control: Initialize the parameters through transfer learning and achieve MAE ≤ 0.2 meters on the validation set.

6. A coastal flood control adaptive strategy optimization system based on reinforcement learning according to claim 5, characterized in that: The flooding simulation diagram specifically includes the following steps: Training data construction: Based on the SLR trajectory and storm surge data under the SSP5-8.5 scenario, the predetermined intensity samples were screened; Generative model design: A conditional generative adversarial network (GAN) is used, with SLR rate, storm surge peak, and coastal topography as conditional variables. The generator uses a U-Net structure to output a 5m×5m resolution simulated inundation range map, and the discriminator distinguishes the generated map from real historical data. Scenario extension effect: Generate an additional 20,000 extreme coupled scenario trajectories to cover the synergistic enhancement effects of SLR and storm surge.

7. A coastal flood control adaptive strategy optimization system based on reinforcement learning according to claim 6, characterized in that: Real-time detection of abnormal signals specifically includes the following steps: Model training: Using a variational autoencoder, we use normal hydrological data as samples to learn the normal feature distribution. Abnormality determination: Real-time monitoring data is input, reconstruction error is calculated, and an abnormality is determined when it exceeds the 99.9% confidence interval threshold; Emergency triggering: Abnormal signals are transmitted to the reinforcement learning model through the API interface, triggering the emergency decision branch, with a response delay of ≤5 minutes.

8. The coastal flood control adaptive strategy optimization system based on reinforcement learning according to claim 1, characterized in that: The strategy execution tracking specifically includes: Seawall construction monitoring: Drones capture the construction area daily, using object detection algorithms to calculate the length of poured concrete and the density of rebar. These statistics are then compared with the "seawall construction cost model." If the actual cost exceeds the budget by 10%, a correction signal is sent to the reinforcement learning model. Evacuation area management: Use facial recognition to count the number of evacuated households, and combine house identification to generate a "forced evacuation priority map" as a basis for adjusting subsidy policies.

9. The coastal flood control adaptive strategy optimization system based on reinforcement learning according to claim 8, characterized in that: The post-disaster assessment and model iteration specifically involves taking drone aerial photos of the entire area within 24 hours after the flood recedes, using a semantic segmentation model to calculate the actual flood damage, and comparing it with the reinforcement learning predictions; Use the loss deviation value to correct the value function parameters of reinforcement learning to improve the fit of the next round of strategy.

10. The coastal flood control adaptive strategy optimization system based on reinforcement learning according to claim 1, characterized in that: The daily mode subunit specifically includes integrating drone data to update status every hour, and the reinforcement learning model outputs the next day's strategy; The emergency mode sub-unit specifically includes triggering "minute-level updates" when the SLR rate exceeds the 90th percentile of the SSP5-8.5 scenario. Specifically, the drone swarm focuses on high-risk areas and transmits data back once every 3 minutes; the reinforcement learning model generates an emergency strategy within 5 minutes based on the latest data; the drone provides real-time feedback on the effect after execution, and the model immediately updates the reward value and adjusts subsequent actions.

Images