An intelligent rescue force deployment method based on reinforcement learning

Abstract

The present invention relates to the field of earthquake disaster assessment and emergency rescue management technology, and in particular to an intelligent rescue force deployment method based on reinforcement learning. The present invention inputs the image to be evaluated into a damaged classification model, outputs the damaged category and calculates the actual damaged area, and combines multiple factors to predict the number of casualties by dynamically adjusting coefficients and weights; establishes a multi-rescue team and multi-disaster point rescue planning model based on reinforcement learning, calculates the rescue needs of different disaster points in combination with the weights of factors such as different altitudes of the disaster points, constructs a data training rescue planning model based on the rescue needs and team capacity, and solves the rescue force deployment results under different examples through the pre-trained rescue planning model; the damaged classification model of the present invention adopts the EAFE‑YOLOv5‑Seg network model, which greatly improves the accuracy of the post-earthquake building damage assessment results, and further improves the accuracy of casualty assessment and the accuracy of rescue force deployment.

Description

Technical Field

[0001] The present invention relates to the technical field of earthquake disaster assessment and emergency rescue management, and in particular to an intelligent rescue force deployment method based on reinforcement learning. Background Art

[0002] Major natural disasters are an important factor leading to casualties. my country is located in the southeast of the Eurasian Plate. Due to the interaction and influence of the Indian Plate and the Pacific Plate, earthquakes are very active, with the characteristics of high intensity, high frequency, shallow epicenter, wide distribution, and severe disasters.

[0003] After natural disasters such as earthquakes occur, it is crucial to quickly and accurately obtain the damage to buildings after the earthquake and to quickly respond and effectively deploy rescue teams. Reasonable deployment of rescue teams can quickly organize a strong rescue force, operate efficiently, quickly rescue trapped people, and reduce casualties. At the same time, deployment can also ensure the rational use of rescue resources, avoid waste of resources and repeated investment, and provide the necessary materials and manpower support for rescue operations. Rapidly planning the deployment and allocation of rescue teams based on the damage assessment of buildings after an earthquake and the needs of emergency rescue is an important reference for government departments and emergency management departments to initiate emergency response and deployment of rescue forces. Accurate assessment results help to maximize rescue operations and are an important scientific issue that needs to be studied urgently in my country's post-earthquake emergency rescue at this stage. Summary of the invention

[0004] Therefore, the technical problem to be solved by the present invention is to overcome the problem in the prior art that the post-earthquake rescue team cannot be deployed quickly and accurately.

[0005] In order to solve the above technical problems, the present invention provides an intelligent rescue force deployment method based on reinforcement learning, comprising:

[0006] Obtain post-earthquake UAV orthophoto data and perform image feature point stitching to obtain images of the post-earthquake area;

[0007] Inputting the post-earthquake regional image into a pre-trained building collapse damage classification model to predict the damage classification of each pixel in the post-earthquake regional image, the building collapse damage classification model is a building edge aggregation feature enhanced neural network EAFE-YOLOv5-Seg model;

[0008] Calculate the proportion of building areas of different damaged categories to the total building area according to the damage classification of each pixel point in the post-earthquake regional image, and predict the number of casualties;

[0009] Establishing a mathematical model for rescue planning of multiple rescue teams and multiple disaster-stricken locations, establishing a pointer network model based on the mathematical model, and training the pointer network model based on a reinforcement learning environment with the goal of minimizing rescue costs;

[0010] The rescue force requirements of different disaster-stricken points are calculated based on the number of casualties, and the pre-trained pointer network model is input to obtain the post-earthquake rescue force deployment decision.

[0011] Preferably, the step of inputting the post-earthquake regional image into a pre-trained building collapse damage classification model to predict the damage classification of each pixel in the post-earthquake regional image comprises:

[0012] The post-earthquake regional image is input into the backbone part of the EAFE-YOLOv5-Seg network structure for feature extraction to obtain a multi-layer feature map, wherein a feature enhancement module is added after some C3 modules and SPPF modules in the backbone part, and the feature enhancement module uses two branches to extract the global information and local spatial information of the input respectively, and the outputs of the two branches are weighted added and fused with the original input to obtain an output after the building edge aggregation feature is enhanced;

[0013] The scale sequence feature fusion module is used to process some feature maps of different levels output by the FPN structure into the same size and upsample them. Then, they are input into the 3D convolution layer after residual connection to obtain the scale fusion feature map.

[0014] Multiple triple feature encoding modules are used to capture the building information in three feature maps of different sizes in the spatial dimension, and the captured triple feature information is integrated into the corresponding feature branches through the PANet structure;

[0015] Inputting the captured triple feature information into a channel attention network, and inputting the output of the channel attention network and the scale fusion feature map into a position attention network to obtain a target feature map;

[0016] The Head part is used to perform damage classification prediction according to the target feature map to obtain the proportion of building areas of different damage categories to the total building area.

[0017] Preferably, the calculating the proportion of building areas of different damaged categories to the total building area according to the damage classification of each pixel point in the post-earthquake regional image, and predicting the number of casualties includes:

[0018] Calculate the ratio of the sum of the pixel areas of all buildings to the total pixel area of ​​the region, and combine it with the actual area of ​​the region to calculate the actual area of ​​buildings of different damage categories;

[0019] Calculate indoor casualty density based on the tidal correction coefficient, indoor occupant density, the proportion of building areas of different damaged categories to the total building area, the damage probability of historical buildings of different damaged categories, and the casualty probability of different damaged categories;

[0020] The number of casualties is calculated based on the actual areas of the buildings of different damage categories and the indoor casualty density.

[0021] Preferably, the calculation formula for calculating the indoor casualty density based on the tidal correction coefficient, the indoor occupant density, the proportion of the building area of ​​different damaged categories to the total building area, the damage probability of historical buildings of different damaged categories and the casualty probability of different damaged categories is:

[0022]

[0023] in, represents the indoor casualty density, represents the tidal correction factor, Indicates indoor occupancy density represents the total number of building structure damage categories, Indicates the damaged category, Indicates the proportion of the building area of ​​this damaged category to the total building area. represents the damage probability of the historical building in this damage category, Represents the probability of casualties for this damage category.

[0024] Preferably, the calculation formula for the damage probability of the historical buildings of different damage categories is:

[0025]

[0026] in, Indicates the type of building damage. Indicates the building structure category. Indicates the total number of building structure categories in the area, Indicates the proportion of the building area occupied by the building structure. Indicated in building structure The category and intensity level are In the case of The probability value of .

[0027] Preferably, the step of establishing a mathematical model of rescue planning for multiple rescue teams and multiple disaster-stricken locations, and establishing a pointer network model based on the mathematical model includes:

[0028] A mathematical model for rescue planning with multiple rescue teams and multiple disaster sites was established, and model constraints were established;

[0029] A Transformer pointer network model is established based on the rescue planning mathematical model, wherein the encoder is used to map the location information of the disaster-stricken points and the rescue teams and the rescue force requirements of different disaster points into a high-dimensional vector space, and the decoder is used to generate the path planning of the rescue team through a greedy strategy or a sampling strategy according to the output of the encoder and the current environmental state.

[0030] Preferably, the training of the pointer network model with the goal of minimizing rescue costs includes:

[0031] Specify different disaster-affected point sizes and training team sizes, and use the number of rescuers required at different disaster-affected points as the rescue demand at the point to generate training data;

[0032] With the goal of minimizing the rescue cost, a penalty term is added to the loss function to ensure that all disaster-affected points are rescued. Based on the reinforcement learning actor network parameters and the critic network model, the pointer model is trained using the back propagation algorithm to update the model parameters and optimize the model.

[0033] Preferably, the calculation formula of the rescue cost is: + , where N is the number of rescue teams, M is the number of disaster sites, is the distance cost from rescue team i to disaster site j, is a decision variable, representing the rescue team Distributed to disaster sites The number of rescuers, is the number of teams used, and α is the penalty factor for using too many teams.

[0034] Preferably, the calculating the rescue force requirements of different disaster-affected points according to the number of casualties includes:

[0035] The number of rescuers required at different disaster-stricken points is calculated based on the number of casualties, population density, earthquake intensity, altitude, the capacity of each rescue team and the weight coefficient of each parameter.

[0036] Preferably, the calculation formula for calculating the number of rescuers required at different disaster-stricken points based on the number of casualties, population density, earthquake intensity, altitude, the capacity of each rescue team and the weight coefficient of each parameter is:

[0037]

[0038] Where D represents the number of casualties, P represents the population density, I represents the earthquake intensity, and E represents the altitude. represents the weight of casualties, represents the weight of population density, represents the weight of the earthquake intensity level, represents the weight of the rescue team's capability, represents the weight of the altitude of the disaster-stricken area; A represents the capacity of each rescue team.

[0039] The above technical solution of the present invention has the following advantages compared with the prior art:

[0040] The rescue force deployment method described in the present invention inputs the image to be evaluated into a damage classification model, outputs the damage category and calculates the actual damaged area, and combines factors such as intensity, population density, adaptive mobile flow tidal information and other multivariate factors to calculate the predicted number of casualties by dynamically adjusting coefficients and weights; then, a multi-rescue team and multi-disaster point rescue planning model based on reinforcement learning is established, and the number of rescue needs at different disaster points is calculated by combining weights of factors such as different altitudes of the disaster points, and a data training rescue planning model is constructed according to the rescue needs and team capacity, and the rescue force deployment results under different examples are solved by the pre-trained rescue planning model; the damage classification model of the present invention adopts an improved EAFE-YOLOv5-Seg network model, which greatly improves the accuracy of the post-earthquake building damage assessment results, and further improves the accuracy of casualty assessment and the accuracy of rescue force deployment. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0042] Figure 1 It is a flow chart of the implementation of a rescue force deployment method provided by the present invention;

[0043] Figure 2 It is a flow chart of a method for deploying rescue forces provided by an embodiment of the present invention;

[0044] Figure 3 A schematic diagram of the structure of the building edge aggregation feature enhancement neural network (EAFE-YOLOv5-Seg) provided in an embodiment of the present disclosure;

[0045] Figure 4 A schematic diagram of the structure of a feature enhancement module provided in an embodiment of the present disclosure;

[0046] Figure 5 A schematic diagram of the structure of a building edge aggregation module provided in an embodiment of the present disclosure;

[0047] Figure 6 A schematic diagram of the structure of a multi-factor weighted model provided in an embodiment of the present disclosure;

[0048] Figure 7A diagram of a Transfromer pointer network structure provided by an embodiment of the present disclosure;

[0049] Figure 8 A flow chart of reinforcement learning training provided by an embodiment of the present disclosure;

[0050] Fig. 9 A schematic diagram of the rescue deployment of the rescue team provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0051] The core of the present invention is to provide an intelligent rescue force deployment method based on reinforcement learning, which effectively improves the accuracy of rescue force deployment.

[0052] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0053] Please refer to Figure 1 and Figure 2 , Figure 1 A flowchart of a rescue force deployment method provided by the present invention is provided. Figure 2 The following is a flowchart of a method for deploying rescue forces provided by an embodiment of the present invention; the specific operation steps are as follows:

[0054] S101: Obtain post-earthquake UAV orthophoto data and perform image feature point stitching to obtain post-earthquake area images;

[0055] S102: Inputting the post-earthquake regional image into a pre-trained building collapse damage classification model to predict the damage classification of each pixel in the post-earthquake regional image, wherein the building collapse damage classification model is a building edge aggregation feature enhanced neural network EAFE-YOLOv5-Seg model;

[0056] S103: calculating the proportion of building areas of different damaged categories to the total building area according to the damage classification of each pixel point in the post-earthquake regional image, and predicting the number of casualties;

[0057] S104: establishing a mathematical model for rescue planning of multiple rescue teams and multiple disaster-stricken locations, establishing a pointer network model based on the mathematical model, and training the pointer network model based on a reinforcement learning environment with the goal of minimizing rescue costs;

[0058] S105: Calculate the rescue force requirements of different disaster-stricken points according to the number of casualties, and input the pre-trained pointer network model to obtain a post-earthquake rescue force deployment decision.

[0059] Based on the above embodiments, this embodiment describes step S101 in detail:

[0060] The continuous orthophotos taken by the drone are converted into image format and cropped to the same size. Image feature points are extracted through professional software (such as DJI Terra), multiple images are stitched together, and the position, flight altitude and shooting focal length information in the image are processed to obtain the regional image and the actual area of ​​the region;

[0061] Due to the advantages of drone data such as high resolution, mobility and low operating cost, it has become the choice for assessing building damage in a small area. This small-scale high-resolution image can effectively meet the needs of detailed earthquake assessment and rescue.

[0062] Based on the above embodiments, this embodiment describes step S102 in detail:

[0063] The training process of the building collapse damage classification model is as follows:

[0064] Constructing a training set: obtaining a set of post-earthquake drone orthophoto images taken after the earthquake, and annotating them respectively to obtain a training set; specifically, converting the drone image orthophoto data into an image format such as JPG format to obtain an unlabeled orthophoto. Since the sizes of all the unlabeled orthophotos obtained are different, in order to avoid the influence of different sizes on the accuracy of post-earthquake assessment, it is necessary to crop all the unlabeled orthophotos to obtain unlabeled orthophotos of the same size, wherein the cropped size of each unlabeled orthophoto is, for example, 1024*1024. Since the unlabeled post-earthquake drone orthophoto includes at least one building, each building must be labeled with a corresponding type of label based on the post-earthquake damage. An unlabeled post-earthquake drone orthophoto plus the labeled labels corresponding to all buildings in the image constitute a post-earthquake drone orthophoto with a building damage category label (hereinafter referred to as a post-earthquake orthophoto). In the disclosed embodiment, the building damage category label of each post-earthquake orthophoto includes at least one type of label in the basically intact class, slightly damaged class, severely damaged class, and collapsed class.

[0065] Using the training set to train the building collapse damage classification model until the loss function converges;

[0066] The three evaluation indicators of precision (P), recall (R), and mean average precision (mAP50) are used to evaluate the accuracy of the trained building damage classification model. The following equation is used to calculate the precision, recall, and mAP scores. The higher the parameters, the better the segmentation results:

[0067]

[0068]

[0069]

[0070]

[0071] Among them, all results are under the condition of iou=0.5, TP represents the actual positive sample with positive prediction, which means FP represents the number of predicted boxes with positive predictions, which means The number of predicted boxes FN represents the actual positive sampling with negative prediction. AP represents the average precision of segmentation; the segmentation performance of the model increases with the increase of AP score. C represents the number of categories, P represents precision, and R represents recall.

[0072] like Figure 3 As shown in the figure, the building edge aggregation feature enhancement neural network EAFE-YOLOv5-Seg is an image segmentation model based on YOLOv5. YOLOv5 is a target detection algorithm based on deep learning. It uses a lightweight network structure and an efficient inference algorithm to greatly improve the detection speed while maintaining accuracy. EAFE-YOLOv5-Seg improves and expands YOLOv5 and applies it to image segmentation tasks. The network of EAFE-YOLOv5-Seg mainly includes three main parts: Backbone (backbone network), Neck (feature fusion), and Head (prediction). The Backbone part is mainly composed of three modules: Focus, C3, and SPPF (spatial pyramid pooling), and its main function is feature extraction. The Neck part is an FPN+PAN structure, which extracts feature maps in a top-down and bottom-up manner. The Head part predicts the result of the feature map passed in by the Neck, and outputs the location box and damage category mask map of the building in each image. In our network structure, we added our feature enhancement module (EF) to the Backbone and improved the Neck part using our architectural edge aggregation module.

[0073] Inputting the post-earthquake regional image into a pre-trained building collapse damage classification model to predict the damage classification of each pixel in the post-earthquake regional image includes:

[0074] The post-earthquake regional image is input into the backbone part of the EAFE-YOLOv5-Seg network structure for feature extraction to obtain a multi-layer feature map, wherein a feature enhancement module is added after some C3 modules and SPPF modules in the backbone part, and the feature enhancement module uses two branches to extract the global information and local spatial information of the input respectively, and the outputs of the two branches are weighted added and fused with the original input to obtain an output after the building edge aggregation feature is enhanced;

[0075] The scale sequence feature fusion module is used to process some feature maps of different levels output by the FPN structure into the same size and upsample them. Then, they are input into the 3D convolution layer after residual connection to obtain the scale fusion feature map.

[0076] Multiple triple feature encoding modules are used to capture the building information in three feature maps of different sizes in the spatial dimension, and the captured triple feature information is integrated into the corresponding feature branches through the PANet structure;

[0077] Inputting the captured triple feature information into a channel attention network, and inputting the output of the channel attention network and the scale fusion feature map into a position attention network to obtain a target feature map;

[0078] The Head part is used to perform damage classification prediction according to the target feature map to obtain the proportion of building areas of different damage categories to the total building area.

[0079] Based on the above embodiments, distinguishing different degrees of damage is a challenging task because they have similar features. The damage features of buildings are mainly manifested on the roof, but the area of ​​damage is different. Therefore, it is necessary to aggregate similar features to enhance the distinction between different degrees of building damage. In addition, the texture and spatial information around the building can provide necessary auxiliary information for the evaluation. Exploring the relationship between global features and contextual features in the image helps the model to more accurately classify the degree of damage to the building. Therefore, a feature enhancement module is added to address these issues. This module can effectively adjust the model's weight ratio for building outlines, roof and wall features, achieve different degrees of attention to input data at different locations in the local perception field of view, and at the same time discover the correlation and dependency between different channels in the data, adjust the attention weights between channels, and the model can more effectively utilize the information between channels, thereby improving the overall performance and performance of the model.

[0080] like Figure 4 As shown, in the embodiment of the present disclosure, the feature enhancement module (EF) process is as follows. Assume that the input data is X with a shape of ( ), first perform a local average pooling operation on the input data X, Represents the output pooling result, Represents the size of the feature map after pooling, obtained after local average pooling ( )Eigenvector;

[0081]

[0082] On this basis, two branches are used to extract different feature information. The first branch Includes global information, while the second branch Contains local spatial information.

[0083] The processing is as follows. The input is subjected to global average pooling again, and the output shape is ( ), then Reshape to ( ), and then processed by Conv1d. After that, Reshape is performed, and the output shape is ( ) The result is passed through the sigmoid activation function and then de-averaged pooled to make it consistent with the shape of S2, and the output , the shape is ( )

[0084]

[0085]

[0086] The processing is as follows, for input Reshape is ( ), and then processed by Conv1d. After that, Reshape is performed, and the output shape is ( ), and finally through the Sigmoid activation function, the output

[0087]

[0088]

[0089] The outputs of the two branches are then weighted and added together, and then the inverse average pooling is performed to convert the feature map size to the original size ( ), and finally multiply it by the original input to get the final output ,in is the weighting coefficient, the default value is 0.5

[0090]

[0091] In the above formula, is local average pooling, is the global average pooling, It is anti-pooling, which can be achieved through adaptive pooling. is to change the shape of the feature map, is the Sigmoid activation function, It is a one-dimensional convolution, the convolution kernel size With channel dimension This indicates that when capturing local cross-channel interaction information, only each channel and its The relationship between adjacent channels. The choice of is determined by the following formula:

[0092]

[0093] is the number of channels, is the size of the convolution kernel, and are all hyperparameters, The default value is 2, The default value is 1. If If it is an even number, add 1.

[0094] Based on the above embodiments, Figure 5 As shown in Figure 1, the building edge aggregation module combines the spatial and multi-scale features of instance segmentation. The module consists of two main component networks that can provide complementary feature information for small building segmentation: the scale sequence feature fusion module (MSF), which combines global or high-level feature information from different image scales; the triple feature encoding module (TSF), which can capture the local fine details of smaller buildings. The effective combination of local and global feature information in the image can produce a more accurate segmentation map.

[0095] First, the MSF module fuses the high-level feature information of the feature maps P3, P4 and P5 at different levels in the Feature Pyramid Network (FPN) output from the EAFE-YOLOv5-Seg backbone network; this three-layer structure captures the information of various sizes and shapes of buildings at different spatial scales. In MSF, the P3, P4 and P5 feature maps are processed to the same size, upsampled, and then concatenated together as the input data of the 3D convolution to effectively combine feature information of different scales. Specifically:

[0096] Since the high-resolution feature map level P3 contains most of the information that is crucial for small object detection and segmentation, the MSF module is designed based on the P3 level. The implementation details of the MSF module are as follows:

[0097] use Two-dimensional convolution changes the number of channels of P4 and P5 layers to be consistent with P3, and uses the nearest neighbor interpolation method to adjust the size of their feature maps to be consistent with the P3 layer; the unsqueeze method is used to increase the dimension of each feature layer, changing it from a 3D tensor [height, width, channel] to a 4D tensor [depth, height, width, and channel]; the 4D feature maps are then concatenated along the depth dimension to form a 3D feature map for subsequent convolution. Finally, 3D convolution, 3D batch normalization, and SiLU activation function are used to complete scale sequence feature extraction.

[0098] The TSF module is designed to enhance the network's ability to detect small buildings in dense buildings. It captures detailed information of small buildings by combining three feature maps of different sizes in the spatial dimension. Then, the information of the TSF module is integrated into each feature branch through the PANet structure. One TSE module is used to integrate P3, P4, and P5 features, and the other is used to integrate P2, P3, and P4 features, which are then combined with the multi-scale information of the MSF module into the P3 branch. Specifically:

[0099] In order to identify densely overlapping small objects, the shape or appearance changes at different scales can be referenced and compared by enlarging the image. Since different feature layers of the backbone network have different sizes, the traditional FPN fusion mechanism only upsamples the feature maps of small sizes and then splits or adds them to the features of the previous layer, ignoring the rich detailed information of the larger size feature layer. With the TSF module, it can segment large, medium, and small features, add large-size feature maps, and perform feature amplification to improve detailed feature information.

[0100] Before feature encoding, the number of feature channels is first adjusted to be consistent with the medium-sized feature channels. After the convolution module processes the large-size feature map (large), its number of channels is adjusted to 1C, and then a hybrid structure of maximum pooling + average pooling is used for downsampling, which helps to maintain the high-resolution features and effectiveness and diversity of the image. For the small-size feature map (small), the convolution module is also used to adjust the number of channels, and then the nearest neighbor interpolation method is used for upsampling. This helps to maintain the richness of local features of low-resolution images and prevent the loss of feature information of small targets. Finally, the three feature maps of large, medium, and small with the same size are convolved once and then spliced, as shown below:

[0101]

[0102] in Represents the feature map output by the TSF module. Represent large-size, medium-size, and small-size feature maps, respectively. Depend on The connection is generated. With The resolution is the same as the number of channels and is three times that of Fm.

[0103] The channel and location attention mechanism (CLA) is further introduced in the P3 branch to utilize high-level multi-scale features and detailed features. The channel and location attention mechanisms in CLA can capture information channels and refine the spatial positioning of small buildings, respectively, thereby improving their detection and segmentation accuracy. Specifically:

[0104] In order to extract representative feature information contained in different channels, CLA is used to integrate detailed feature information and multi-scale feature information. It consists of a channel attention network ( ) and the position attention network ( )composition.

[0105] The specific implementation process is as follows: For the input data Global average pooling, then reshape the output and pass it through Conv1d, and finally reshape it to the original input shape to get the output , This process is used to effectively capture the connections between different channels. Finally, the input data and Add together to get ,composition The input data.

[0106] Different from the channel attention mechanism, the position attention mechanism first divides the input feature map into two parts according to width and height, and then processes them separately. and ) and finally merged to generate the output. More precisely, the input feature map information is gathered in two different directions, namely in the horizontal direction ( ) and vertical ( ) direction, which can effectively retain the spatial structure-related feature information of the feature map, which can be calculated by the following formula:

[0107]

[0108]

[0109] We use the width and height of the input feature map to represent and To express, and They represent the input feature maps respectively. Location When generating the position attention coordinates, the horizontal and vertical features are first connected and then subjected to a two-dimensional convolution operation:

[0110]

[0111] In the above formula Conv2d representation Convolution, Concat means connection.

[0112] Then the attention features obtained above are segmented to obtain the feature map associated with the position, as shown below:

[0113]

[0114]

[0115] in and Represent the feature maps in width and height respectively. Expand its shape and dimension to The three are consistent, and then multiplied to get the final output result

[0116] The final output of CLA is defined as:

[0117]

[0118] like Figure 6 As shown, based on the above embodiments, this embodiment describes step S103 in detail:

[0119] The acquired post-earthquake UAV orthophoto test data is input into the trained building damage classification model to calculate the damage category of different building instances in the image. The classification of each pixel point in the image can also be calculated according to needs, providing diversified evaluation data and providing data support for different algorithms after the earthquake; we calculate the ratio of the sum of the pixel areas of all buildings to the total pixel area of ​​the area, and combine it with the actual area of ​​the area to calculate the actual area of ​​buildings with different damage categories ;

[0120] The indoor casualty density is calculated based on the tidal correction coefficient, indoor occupant density, the proportion of the building area of ​​the different damaged categories to the total building area, the damage probability of the historical buildings of different damaged categories, and the casualty probability of different damaged categories:

[0121]

[0122] in, represents the indoor casualty density, represents the tidal correction factor, Indicates indoor occupancy density represents the total number of building structure damage categories, Indicates the damaged category, Indicates the proportion of the building area of ​​this damaged category to the total building area. represents the damage probability of the historical building in this damage category, Represents the probability of casualties for this damage category.

[0123] In the above formula, due to the "tidal characteristics" of population mobility, the population density in the same area varies in different time periods. For earthquakes occurring between 1 and 6 a.m., Select 2, the correction factor from 6 am to 9 am is 1, from 9 am to 20 pm is 5 / 9, and from 20 pm to 1 am is 5 / 3.

[0124] represents the weighting coefficient, and A weighted combination is performed to combine the drone image results identified by the deep learning model with the field survey results, further improving the accuracy of classification of different types of building damage

[0125] The indoor population density represented by The average value can be obtained by using the population density statistics of the region. It represents the proportion of building areas with different degrees of damage to the total building area.

[0126] Represents the level of damage The probability of indoor casualties is shown in Table 1.

[0127] Table 1 Relationship between earthquake casualty probability and damage level

[0128]

[0129] It represents the probability of building structure damage based on the building structure earthquake damage matrix, which is given by earthquake engineering and structural engineering experts from the Institute of Engineering Mechanics of the China Earthquake Administration based on on-site building surveys, earthquake damage experience and statistical results. Its format is shown in Table 2:

[0130] Table 2 Seismic damage matrix of reinforced concrete structures (%)

[0131]

[0132] In actual application, it is necessary to refer to the earthquake damage matrix of different building structures. The value of needs to be weighted and calculated in turn for the regional building structure components. , the calculation formula is as follows:

[0133]

[0134] in, Indicates the type of building damage. Indicates the building structure category. Indicates the total number of building structure categories in the area, Indicates the proportion of the building area occupied by the building structure. Indicated in building structure The category and intensity level are In the case of The probability value of .

[0135] The number of casualties D is calculated based on the actual area of ​​the buildings of different damaged categories and the indoor casualty density:

[0136]

[0137] Based on the above embodiments, this embodiment describes step S104 in detail:

[0138] Establishing the mathematical model of rescue planning for multiple rescue teams and multiple disaster-stricken locations MRMAVRP: It involves allocating multiple rescue teams to different disaster-stricken locations while meeting the needs of all disaster-stricken locations and determining the optimal path for each rescue team. The assumptions of the rescue model are:

[0139] In the region, the attributes related to the rescue team include the coordinate points of the team (the coordinate generation range is between 0-1), the capacity of the team is in the range L, the disaster-affected points include the coordinate points (same as the team coordinate system, also between 0-1) and the rescue demand range K of each disaster-affected point;

[0140] The same rescue team can reach multiple disaster-stricken areas. When the capacity of a team meets the rescue needs of multiple disaster-stricken areas, it can go to multiple disaster-stricken areas.

[0141] You only need to focus on the distance between the team and the disaster site, without considering the situation where the team and each disaster site need to return to the team;

[0142] Assuming that the same team is transferred to multiple disaster-stricken areas, the distance cost is , the sum of the distances from the team to each disaster site, without considering the maximum value of a single distance ,in , because the team and each disaster-stricken point set out at the same time, the cost of each distance is the cumulative result, so the sum of the distances to each disaster-stricken point is calculated;

[0143] In order to use fewer teams as quickly as possible and complete the allocation of teams to the disaster-stricken points with the shortest distance, a penalty factor for using a team once and the sum of the distances between the team and the disaster-stricken point are given, and the negative value of the sum of the two is used as the optimization target of the reward. Therefore, the final optimization target is:

[0144]

[0145] in The distances and is the number of teams used, α is the penalty factor for using too many teams, and the penalty factor α in the disclosed method is temporarily set to 10, that is, α=10.

[0146] According to the goal of optimizing the minimum distance, the relevant parameters of the rescue planning problem model based on multiple rescue teams and disaster sites are defined as follows:

[0147] Table 3. Related parameters of the rescue planning model based on multiple rescue teams and disaster-stricken locations

[0148]

[0149] The model objective function is as follows:

[0150] +

[0151] Here Z is the total distance cost we wish to minimize, is the distance cost from rescue team i to disaster site j, is a decision variable, representing the rescue team Distributed to disaster sites The number of rescuers.

[0152] Establish model constraints:

[0153] 1. Every disaster site needs to be rescued:

[0154] 2. The number of people assigned to each rescue team cannot exceed its capacity:

[0155] 3. The rescue needs of each disaster site are met:

[0156] 4. The arrival time of the rescue team is affected by its speed:

[0157] 5. The number of rescuers allocated to the rescue team is non-negative:

[0158] Among them, constraint 1 means that rescue needs must be provided for each disaster-stricken point, constraint 2 means that the number of rescue points allocated by each rescue team must not exceed the total number of people (capacity) in its own team, constraint 3 means that the rescue needs of each disaster-stricken point must be met, constraint 4 means that the arrival time of the rescue team is affected by its speed, and constraint 5 means that the number of people rescued each time by the rescue team must be a non-negative number.

[0159] A Transformer pointer network model is established based on the rescue planning mathematical model, in which the encoder is used to map the location information of the disaster site and the rescue team and the rescue force requirements of different disaster sites into a high-dimensional vector space, and the decoder is used to generate the path planning of the rescue team through a greedy strategy or a sampling strategy according to the output of the encoder and the current environmental state:

[0160] It is implemented using the PyTorch deep learning framework. A rescue team deployment environment class Env is defined, which includes parameters such as the location of the rescue team and the disaster site, rescue needs, and team resources. By constructing a distance matrix and a mask, the distance between rescue teams and the constraints of optional deployment can be effectively represented. At the same time, a rescue team path update mechanism is implemented to update the team's path information and driving distance according to the selected deployment. Based on the Transformer pointer network, it is used to implement path planning and deployment of the rescue team to maximize rescue efficiency. The model consists of two parts: an encoder and a decoder:

[0161] like Figure 7 As shown, construct the pointer network model:

[0162] First, the encoder part:

[0163] The encoder's task is to map the location information of the disaster site and rescue team as well as the demand in the disaster area into a high-dimensional vector space. The encoder uses a graph attention encoder to process these location information and demand for subsequent processing.

[0164] Embedding layer: The encoder uses one-dimensional convolution to embed the position information and demand quantity and map them to a K-dimensional vector space.

[0165] The purpose of this is to convert the position information and demand into a form that the model can understand and provide effective input for the subsequent decoder.

[0166] For the decoder part: The task of the decoder is to generate the path planning of the rescue team based on the output of the encoder and the current input.

[0167] In the rescue mission, the decoder uses a long short-term memory network (LSTM) as a basic unit to process sequence information, including the movement path of the rescue team and rescue allocation.

[0168] Initialize weight matrix: The decoder also uses a linear layer to initialize the weight matrix to map the input data to the embedding space.

[0169] Dynamically calculate attention weights: The decoder dynamically calculates the current environment state and the embedding vector output by the encoder to generate attention weights. The MultiHeadAttention model is used here to calculate the attention weights.

[0170] Rescue planning allocation logic: The decoder selects the next deployment based on the attention weights obtained by dynamic calculation and the current environment status through path planning logic. Greedy and random sampling strategies are used here to meet different scenario requirements.

[0171] State update and information passing: The decoder updates the current environment state according to the selected deployment and passes the state information to the next time step, continuing path planning until the end condition is reached.

[0172] ·Network initialization: In the initialization training network parameter phase, the given training parameters are used to configure the training process. First, the parameters required for training are obtained, including random seeds, the number of affected nodes, the number of rescue teams, and the location information of the affected points and rescue teams. Then, an instance of the Config class is created and the parsed parameters are passed to the class so that these parameters can be used throughout the training process. Next, the configuration is saved as a .pkl file for easy use during the training process or subsequent testing. In the phase of inputting the generated data into the model, the method of generating data can be used, and the data generation function can generate training data that meets the rescue scale of the rescue team. These data include the capacity and coordinates of the rescue team required for the rescue mission, the demand and coordinates of the disaster site, and the location of the rescue team initialization nodes. Then, the generated data is input into the training model for training. During the training process, the model will continuously adjust the network parameters using the back propagation algorithm based on the input data and the given training parameters to minimize the loss function, thereby realizing the training of the model. In one embodiment, the coordinate system range of the team is (0,1), the coordinate range of the disaster site is also (0,1), the capacity range of the rescue model team is set to (10,500), and the rescue demand range K of the disaster site is set to (1,100).

[0173] like Figure 8As shown in the figure, in order to ensure that all disaster-stricken points are rescued and repeated training is performed based on the training data set, an objective function is first defined to minimize the rescue cost, and a penalty term is added to the loss function to ensure that all disaster-stricken points are rescued. Then, during the training process, the back propagation algorithm is used to update the model parameters to optimize the model, while monitoring the changes in the loss function and the rescue of the disaster-stricken points by the model during the training process. If the model performs poorly on the training data set, adjust the hyperparameters and repeat the training process. Finally, the performance of the model is evaluated using the test data set to ensure that it can effectively rescue all disaster-stricken points and to see whether the needs of all disaster-stricken points are met after the training is completed.

[0174] Keep the best pre-trained weight files based on the data after training at different scales, and solve new scale problems on new test cases:

[0175] The decoder input includes the output from the encoder, namely the node embedding representation and the graph embedding representation, as well as the current state of the environment, such as the location of the rescue team, the nodes visited, etc. The decoder is passed to the decoder through the parameters encoder_output and x, and then the decoder performs the following operations:

[0176] Calculate static information: First, the decoder calculates static information using node embedding representation and graph embedding representation. In the code, the graph embedding representation is mapped to a query vector of the same dimension through a linear transformation, and the node embedding representation is linearly transformed to obtain the key and value. This static information is reused in subsequent dynamic calculations.

[0177] Calculating dynamic information: In each step of dynamic calculation, the decoder calculates the probability distribution of the next deployment based on the current environment state and static information through the attention mechanism. Specifically, the decoder first calculates the query vector based on the current state and static information, and uses the multi-head attention mechanism to obtain context information. Then, the context information is mapped to the same dimension through linear transformation, and the logarithmic distribution of the deployment probability is calculated using the dot product attention mechanism.

[0178] Deployment decision: Based on the probability distribution of deployment, the decoder uses different strategies to make deployment decisions. In the disclosed embodiment, two options are provided: greedy strategy and random sampling strategy. The greedy strategy makes decisions by selecting the deployment with the highest probability, while the random sampling strategy performs random sampling based on the probability distribution. The decoder updates the current state based on the selected deployment and repeats the calculation until the termination condition is met.

[0179] Finally, the decoder outputs the path plan and corresponding cost required to complete the task, and returns the path nodes, costs, and possible deployment probability distribution information of the rescue team.

[0180] like Fig. 9 As shown, Fig. 9 A schematic diagram of the rescue deployment of the rescue team provided in an embodiment of the present disclosure.

[0181] Based on the above embodiments, this embodiment describes step S105 in detail:

[0182] The calculation of rescue force requirements at different disaster-affected locations based on the number of casualties includes:

[0183] The number of rescuers required at different disaster sites is calculated based on the number of casualties, population density, earthquake intensity, altitude, the capabilities of each rescue team and the weight coefficient of each parameter:

[0184]

[0185] Among them, D represents the number of casualties, P represents the population density, I represents the earthquake intensity, and E represents the altitude. represents the weight of casualties, represents the weight of population density, represents the weight of the earthquake intensity level, represents the weight of the rescue team's capability, represents the weight of the altitude of the disaster-stricken area; A represents the capacity of each rescue team, and F represents the final rescue demand.

[0186] Based on the above embodiments, this embodiment provides a problem example. The coordinate range of the rescue team and the disaster site is randomly distributed in [0, 1]. The capacity range of the rescue team is set to (10, 500). The rescue team's rescue capacity and the disaster site are randomly sampled according to the data. 500 samples are generated with the number of teams being 10 and the number of disaster sites being 5. New samples are tested under the pre-training scale of 10-5 to predict the results. Table 4 shows the prediction input data and structure of the test sample team 10-disaster site 5:

[0187] Table 4 Prediction input data and structure of test sample team 10-disaster site 5

[0188]

[0189] Among them, 0-9 are the nodes of the rescue team, 10-14 are the nodes of the disaster site, and the two groups of assigned paths mean that all rescue needs can be met by calling two teams.

[0190] The embodiment of the present invention further provides a post-earthquake rescue force deployment device; the specific device may include:

[0191] Image acquisition module, used to obtain post-earthquake UAV orthophoto data and perform image feature point stitching to obtain post-earthquake area images;

[0192] A damage classification module is used to input the post-earthquake area image into a pre-trained building collapse damage classification model to predict the damage classification of each pixel in the post-earthquake area image, wherein the building collapse damage classification model is a building edge aggregation feature enhanced neural network EAFE-YOLOv5-Seg model;

[0193] A casualty prediction module, used to calculate the proportion of building areas of different damaged categories to the total building area according to the damage classification of each pixel point in the post-earthquake regional image, and predict the number of casualties;

[0194] A deployment model building module, used to establish a mathematical model of rescue planning for multiple rescue teams and multiple disaster-stricken locations, establish a pointer network model based on the mathematical model, and train the pointer network model based on a reinforcement learning environment with the goal of minimizing rescue costs;

[0195] The decision output module is used to calculate the rescue force requirements of different disaster-stricken points according to the number of casualties, and input the pre-trained pointer network model to obtain the post-earthquake rescue force deployment decision.

[0196] The post-earthquake rescue force deployment device of this embodiment is used to implement the aforementioned rescue force deployment method. Therefore, the specific implementation method of the post-earthquake rescue force deployment device can be seen in the implementation example part of the aforementioned rescue force deployment method. For example, the image acquisition module, the damage classification module, the casualty prediction module, the deployment model construction module, and the decision output module are respectively used to implement steps S101, S102, S103, S104 and S105 in the aforementioned rescue force deployment method. Therefore, its specific implementation method can refer to the description of the corresponding various parts of the implementation example, which will not be repeated here.

[0197] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. An intelligent rescue force deployment method based on reinforcement learning, characterized in that: include: Obtain post-earthquake UAV orthophoto data and perform image feature point stitching to obtain images of the post-earthquake area; Inputting the post-earthquake regional image into a pre-trained building collapse damage classification model to predict the damage classification of each pixel in the post-earthquake regional image, the building collapse damage classification model is a building edge aggregation feature enhanced neural network EAFE-YOLOv5-Seg model; Calculate the proportion of building areas of different damaged categories to the total building area according to the damage classification of each pixel point in the post-earthquake regional image, and predict the number of casualties; Establishing a mathematical model for rescue planning of multiple rescue teams and multiple disaster-stricken locations, establishing a pointer network model based on the mathematical model, and training the pointer network model based on a reinforcement learning environment with the goal of minimizing rescue costs; The rescue force requirements of different disaster-stricken points are calculated according to the number of casualties, and the pre-trained pointer network model is input to obtain the post-earthquake rescue force deployment decision; wherein, Inputting the post-earthquake regional image into a pre-trained building collapse damage classification model to predict the damage classification of each pixel in the post-earthquake regional image includes: The post-earthquake regional image is input into the backbone part of the EAFE-YOLOv5-Seg network structure for feature extraction to obtain a multi-layer feature map, wherein a feature enhancement module is added after some C3 modules and SPPF modules in the backbone part, and the feature enhancement module uses two branches to extract the global information and local spatial information of the input respectively, and the outputs of the two branches are weighted added and fused with the original input to obtain an output after the building edge aggregation feature is enhanced; The scale sequence feature fusion module is used to process some feature maps of different levels output by the FPN structure into the same size and upsample them. Then, they are input into the 3D convolution layer after residual connection to obtain the scale fusion feature map. Multiple triple feature encoding modules are used to capture the building information in three feature maps of different sizes in the spatial dimension, and the captured triple feature information is integrated into the corresponding feature branches through the PANet structure; Inputting the captured triple feature information into a channel attention network, and inputting the output of the channel attention network and the scale fusion feature map into a position attention network to obtain a target feature map; The Head part is used to perform damage classification prediction according to the target feature map to obtain the proportion of building areas of different damage categories to the total building area.

2. The intelligent rescue force deployment method according to claim 1, characterized in that: The calculating the proportion of the building area of ​​different damaged categories to the total building area according to the damage classification of each pixel point in the post-earthquake regional image, and predicting the number of casualties includes: Calculate the ratio of the sum of the pixel areas of all buildings to the total pixel area of ​​the region, and combine it with the actual area of ​​the region to calculate the actual area of ​​buildings of different damage categories; Calculate indoor casualty density based on the tidal correction coefficient, indoor occupant density, the proportion of building areas of different damaged categories to the total building area, the damage probability of historical buildings of different damaged categories, and the casualty probability of different damaged categories; The number of casualties is calculated based on the actual areas of the buildings of different damage categories and the indoor casualty density.

3. The intelligent rescue force deployment method according to claim 2, characterized in that: The calculation formula for calculating indoor casualty density based on the tidal correction coefficient, indoor occupant density, the proportion of the building area of ​​different damaged categories to the total building area, the damage probability of historical buildings of different damaged categories, and the casualty probability of different damaged categories is: in, represents the indoor casualty density, represents the tidal correction factor, Indicates indoor occupancy density represents the total number of building structure damage categories, Indicates the damaged category, Indicates the proportion of the building area of ​​this damaged category to the total building area. represents the damage probability of the historical building in this damage category, represents the probability of casualties in this damage category, Represents the weighting coefficient.

4. The intelligent rescue force deployment method according to claim 2, characterized in that: The calculation formula for the damage probability of the historical buildings of different damage categories is: in, Indicates the type of building damage. Indicates the building structure category. Indicates the total number of building structure categories in the area, Indicates the proportion of the building area occupied by the building structure. Indicated in building structure The category and intensity level are In the case of The probability value of .

5. The intelligent rescue force deployment method according to claim 1, characterized in that: The method of establishing a mathematical model for rescue planning of multiple rescue teams and multiple disaster-stricken locations, and establishing a pointer network model based on the mathematical model includes: A mathematical model for rescue planning with multiple rescue teams and multiple disaster sites was established, and model constraints were established; A Transformer pointer network model is established based on the rescue planning mathematical model, wherein the encoder is used to map the location information of the disaster-stricken points and the rescue teams and the rescue force requirements of different disaster-stricken points into a high-dimensional vector space, and the decoder is used to generate the path planning of the rescue team through a greedy strategy or a sampling strategy according to the output of the encoder and the current environmental state.

6. The intelligent rescue force deployment method according to claim 1, characterized in that: The training of the pointer network model with the goal of minimizing the rescue cost includes: Specify different disaster-affected site sizes and training team sizes, and use the number of rescuers required at different disaster-affected sites as the rescue demand of the disaster-affected sites to generate training data; With the goal of minimizing the rescue cost, a penalty term is added to the loss function to ensure that all disaster-affected points are rescued. Based on the reinforcement learning actor network parameters and the critic network model, the pointer model is trained using the back propagation algorithm to update the model parameters and optimize the model.

7. The intelligent rescue force deployment method according to claim 6, characterized in that: The calculation formula of the rescue cost is: + , where N is the number of rescue teams, M is the number of disaster sites, is the distance cost from rescue team i to disaster site j, is a decision variable, representing the rescue team Distributed to disaster sites The number of rescuers, is the number of teams used, and α is the penalty factor for using too many teams.

8. The intelligent rescue force deployment method according to claim 1, characterized in that: The calculation of rescue force requirements at different disaster-affected locations based on the number of casualties includes: The number of rescuers required at different disaster-stricken points is calculated based on the number of casualties, population density, earthquake intensity, altitude, the capacity of each rescue team and the weight coefficient of each parameter.

9. The intelligent rescue force deployment method according to claim 1, characterized in that: The calculation formula for calculating the number of rescuers required at different disaster sites based on the number of casualties, population density, earthquake intensity, altitude, the capacity of each rescue team and the weight coefficient of each parameter is: Where D represents the number of casualties, P represents the population density, I represents the earthquake intensity, and E represents the altitude. represents the weight of casualties, represents the weight of population density, represents the weight of the earthquake intensity level, represents the weight of the rescue team's capability, represents the weight of the altitude of the disaster-stricken area; A represents the capacity of each rescue team.

Images