Remote sensing image building damage assessment method based on multi-task learning, storage medium and equipment
Through the remote sensing image building damage assessment method based on multi-task learning, combined with building positioning and damage degree classification tasks, the problem of insufficient accuracy and timeliness of building damage assessment in the existing technology is solved, and efficient and automated evaluation results are achieved.
Patent Information
- Application Number
- CN202510255801.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-05
- Publication Date
- 2025-06-20
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing building damage assessment methods are difficult to ensure accuracy and timeliness, and rely on manual judgment and are less efficient.
The remote sensing image building damage assessment method based on multi-task learning is adopted, and network structures such as encoders and decoders are evaluated through the feature extraction of remote sensing image before and after building damage, combined with building positioning and damage degree classification tasks.
Improves the accuracy and automation of building damage assessments, ensures timeliness and reliability of assessments, and provides decision support in rapid post-disaster responses.
Smart Images
Figure CN120182819A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of building damage assessment, and particularly relates to a method for building damage assessment from remote sensing images, a storage medium, and a device. Background Art
[0002] With the continuous advancement of urbanization, buildings have become an important part of urban infrastructure. In the event of natural disasters, emergencies, military conflicts, etc., buildings are often damaged to varying degrees, resulting in casualties, property losses, and interruptions to social and economic activities. Effectively assessing the degree of building damage and the scope of damage is of great significance for post-disaster rescue, restoration and reconstruction, and the formulation of relevant decisions.
[0003] Currently, building damage assessment mainly relies on manual inspections, on-site assessments, and traditional visual monitoring methods. Although these methods can provide a basis for damage assessment to a certain extent, due to the need to rely on manual judgment during the assessment process, the assessment results are highly subjective and the efficiency is low. In addition, general deep learning methods are difficult to ensure the accuracy and timeliness of the assessment. Therefore, there is an urgent need for a new method for building damage assessment that can overcome the deficiencies of the existing technology, improve the accuracy, automation level, and efficiency of the assessment, and provide more reliable and timely decision-making support, especially in the case of rapid post-disaster response. Summary of the Invention
[0004] The present invention aims to solve the problem that the existing building damage assessment methods are difficult to ensure accuracy and timeliness.
[0005] A method for building damage assessment from remote sensing images based on multi-task learning, for a building to be damaged and assessed, obtaining pre-damage remote sensing images of the building and post-damage remote sensing images of the building, and using a building damage assessment model for damage assessment. The building damage assessment model includes an encoder corresponding to the pre-damage remote sensing image of the building, an encoder corresponding to the post-damage remote sensing image of the building, a decoder and a classifier corresponding to the building positioning task, and a damage positioning module, a decoder, and a classifier corresponding to the damage degree classification task; the process of using the building damage assessment model for damage assessment includes:
[0006] The pre-damage remote sensing image of the building and the post-damage remote sensing image of the building are respectively sent to the encoder for feature extraction to extract feature maps of different scales before and after damage where i = 1, 2 represents the features before and after damage, j = 1, 2,..., J represents different scale feature maps, and J represents the last J layers of the encoder, that is, the feature map is the feature map obtained from the last J layers of the encoder;
[0007] Building positioning: Features at different scales before damage are continuously upsampled by the decoder. The decoder at each layer upsamples the input and aggregates feature information at different scales through skip connections. The structure of the decoder is shown in the formula:
[0008]
[0009] where D j represents the output of the j-th layer decoder, [·] represents the concatenation operation in the channel dimension, Up(·) represents the upsampling layer of bilinear interpolation, and Conv represents the convolutional unit;
[0010] The output D1 of the last layer of the decoder is fed into the building positioning task classifier to obtain the predicted probability map P loc , and then the building positioning result is obtained;
[0011] Damage degree classification: The feature maps before and after building damage are processed as follows to obtain the building damage feature H j :
[0012]
[0013] where DLM represents the damage localization module, and its processing process includes: First, according to to obtain and Then pass through the convolutional layer with a convolution kernel of 1 and add them element by element to obtain the damage feature H j '; and perform subtraction and absolute value operation to obtain the difference feature before and after damage. This difference feature is concatenated with the feature H j ' in the channel dimension and passed through the convolutional layer with a convolution kernel of 1 to obtain the final damage feature H j ;
[0014] Damage features H at different scales j are continuously upsampled by the decoder with the same structure as the building positioning structure but without shared parameters and aggregate information at different scales. Finally, the predicted probability map P cls is obtained through the damage degree classification classifier, and then the damage degree classification result is obtained.
[0015] Furthermore, the encoder for processing the remote sensing image before building damage has the same structure as the encoder for processing the remote sensing image after building damage, but without shared parameters.
[0016] Preferably, J is taken as 3.
[0017] Furthermore, the encoder is ResNet18.
[0018] Furthermore, the convolutional unit Conv in the decoder includes a convolutional layer with a convolution kernel of 1, an activation function ReLU(·), and a regularization layer.
[0019] Furthermore, the decoder for building location includes two layers of pixel-level convolutional layers and a Softmax(·) layer.
[0020] Furthermore, the building damage assessment model is obtained through the following training process:
[0021] First, obtain the building damage assessment dataset and divide it into a training set and a validation set; the building damage assessment dataset includes pre-damage remote sensing images of buildings, damage label maps Y corresponding to post-damage remote sensing images of buildings hs , and the damage label map is for damage degree classification;
[0022] For the images in the training set, process them to obtain the predicted probability map P of the building location task classifier loc and the predicted probability map P of the damage degree classification classifier cls ;
[0023] Optimize the building damage assessment model based on the loss function L of remote sensing image building damage assessment based on multi-task learning to obtain the optimized building damage assessment model; the loss function L is as follows:
[0024] L = L CE (P loc , Y loc ) + L CE (P cls , Y hs ) + λL task
[0025]
[0026] where MSE(·) represents the mean square error, r loc represents the category of each pixel in R loc , p cls and p loc represent the probabilities of each category at each pixel position in P cls and P loc ; represents the operation of gradient clipping, that is is the representation of p loc after gradient clipping; p cls (x) and p loc (x) represent the probability values of the category x, maxhs(·) represents the category with the largest probability among the damage categories after removing the background category 0 in the probability prediction map p cls of the damage classification task; Y loc is with Yhs The binary label modified based on Y hs sets the non-background class to 1 and the background class to 0; L CE (P loc , Y loc ) and L CE (P hs , Y hs ) are the cross-entropy loss functions of the predicted results and their corresponding labels for the final building damage location task and building damage classification task, and λ is the hyperparameter for adjusting the L task weight;
[0027] During the training process, the performance of the building damage assessment model is verified through the validation set, and the final building damage assessment model is determined based on the validation set.
[0028] Furthermore, during the process of determining the final building damage assessment model based on the validation set, the model weight with the highest mIoU in the validation set is used as the finally selected model weight;
[0029] where the calculation method of mIoU is as follows:
[0030]
[0031] Among them, in IoU j is the intersection over union index for class j, TP is the true positive: for class j, TP means the model correctly predicts the samples belonging to class j as class j; FP is the false positive: for class j, FP means the model wrongly predicts the samples not belonging to class i as class j; FN is the false negative: for class j, FN means the model wrongly predicts the samples belonging to class j as other classes that are not class j.
[0032] A computer storage medium, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement the described remote sensing image building damage assessment method based on multi-task learning.
[0033] A remote sensing image building damage assessment device based on multi-task learning, the device includes a processor and a memory, and at least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to implement the described remote sensing image building damage assessment method based on multi-task learning.
[0034] Beneficial effects:
[0035] The present invention extracts the features of the damaged area before and after damage, and then extracts representative damage features through the damage feature enhancement part. Similar to the building positioning task, multi-scale damage features are sent to the decoder to aggregate multi-scale information and gradually up-sample, and finally sent to the classifier to obtain the prediction result of the damage classification task. This network structure can effectively extract features to analyze the features before and after damage, thereby effectively ensuring the final segmentation accuracy. At the same time, the network structure of the present invention is simple, the processing process is simple and efficient, and the timeliness of processing can be guaranteed. Through the example effect, it can be seen that the method of the present invention has achieved excellent results in building damage assessment. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 It is a flowchart of a remote sensing image building damage assessment method based on multi-task learning in a specific embodiment.
[0037] Figure 2 It is a schematic diagram of the model structure in a remote sensing image building damage assessment method based on multi-task learning in a specific embodiment.
[0038] Figure 3 It is a schematic diagram of the structure of the damage positioning module.
[0039] Figure 4 It is a diagram of the segmentation assessment result of building damage assessment. SPECIFIC EMBODIMENTS
[0040] The present invention first obtains the remote sensing images before and after building damage and their corresponding building damage labels to construct a data set, which is divided into a training set, a validation set and a test set, and data augmentation is performed by horizontal flipping, vertical flipping and rotation before training. During the training process, the images before and after building damage extract corresponding multi-scale features through the backbone network ResNet18 with pre-trained weights. Then, in the building positioning task, only the features before building damage are used to locate the boundary of the building. In the decoder, up-sampling is continuously performed and sent to the classifier to obtain the building positioning prediction result. In the damage classification task, it is necessary to combine the features before and after damage for evaluation. Specifically, the features of the damaged area are extracted for the features before and after damage, and then representative damage features are extracted through the damage feature enhancement part. Similar to the building positioning task, multi-scale damage features are sent to the decoder to aggregate multi-scale information and gradually up-sample, and finally sent to the classifier to obtain the prediction result of the damage classification task. In order to improve the positive incentive effect between tasks, a task interaction module is adopted to constrain the prediction probabilities of the two tasks. The final result takes the dot product of the result of the building positioning task and the prediction result of the damage classification task, so as to obtain a damage assessment result with clear boundaries. The damage estimation model is optimized according to the obtained model result loss function; the method of the present invention has achieved excellent results in building damage assessment. The present invention will be described in detail below in conjunction with specific embodiments.
[0041] Embodiment 1: Combined with Figures 1 to 3 to illustrate this embodiment,
[0042] A method for remote sensing image building damage assessment based on multi-task learning described in the specific embodiment includes:
[0043] Step 1: Construct a building damage assessment data set, which includes remote sensing images before and after building damage, and a damage label map Y corresponding to the damaged building hs , where the values in Y hs are 0, 1, 2, 3, 4 respectively; 0 represents the background class, 1 represents that the building is not damaged, 2 represents that the building is slightly damaged, 3 represents that the building is moderately damaged, and 4 represents that the building is completely damaged. The number of damage categories is artificially defined and is related to the data set used.
[0044] It should be noted that: the remote sensing images before and after damage can be different modal data, such as optical remote sensing data and SAR remote sensing data. If it is SAR remote sensing data, the data needs to be copied three times to obtain an input data with 3 channels.
[0045] Divide the data set into a training set, a validation set and a test set;
[0046] Step 2: Perform horizontal flipping, vertical flipping and rotation data augmentation operations on the remote sensing images before and after building damage in the training set;
[0047] For each group of remote sensing images before and after building damage in the training set and their corresponding damage labels, randomly perform horizontal flipping, vertical flipping, clockwise rotation by 90 degrees, and counterclockwise rotation by 90 degrees with a probability of 50% to perform data augmentation, so as to increase the diversity of the data.
[0048] Step 3: Send the training data into the building damage assessment model, decompose the building damage assessment task into a building location task and a damage classification task through a multi-task learning strategy, and obtain a classification result map R of the damaged image cls and a building location prediction result map R loc , where the building location task is used to extract the building area in the image and belongs to a two-class dense prediction task, and the damage classification task is to directly classify the damage situation of the building and belongs to a multi-class dense prediction task;
[0049] In this embodiment, the remote sensing images before and after building damage first pass through ResNet18 pre-trained on the open-source data set ImageNet to extract feature maps of different scales before and after damage Indicates the features before and after damage. j = 1, 2, 3 represent feature maps of different scales, where the feature maps are taken from the last three layers of ResNet18. The images before and after damage are respectively passed through a ResNet18 to extract features of different scales. The backbone networks before and after damage do not share weights to adapt to different-modal images before and after damage.
[0050] After obtaining the features of different scales before and after damage, for the building localization task, since it is difficult to accurately define the building boundary in the remote sensing image after building damage, in this task, only the features before building damage are relied on for building localization. Specifically, the features of different scales before damage are continuously upsampled by the decoder. Each layer of the decoder upsamples the input and aggregates the feature information of different scales through skip connections. The structure of the decoder is shown in the formula:
[0051]
[0052] Among them, D j represents the output of the j-th layer decoder. [·] represents the concatenation operation in the channel dimension. Up(·) represents the upsampling layer of bilinear interpolation. Conv represents the convolutional layer with a kernel size of 1, the activation function ReLU(·), and the superposition of the regularization layer.
[0053] The output D1 of the last layer of the decoder is sent to the classifier to obtain the final predicted probability map P loc .
[0054] For the damage classification task, since the damage situation needs to be judged by comprehensively considering the building before and after damage, it is necessary to process the features of the building before and after damage to obtain the building damage features, as shown in the formula:
[0055]
[0056] Among them, DLM represents the damage localization module. In this module, in order to accurately locate the damaged building and extract the damage features, the features of the building before and after damage reduce the time dependence by exchanging the concatenation order, so that the model pays more attention to the damage changes of the building, as shown in the formula:
[0057]
[0058] The features of the two combination orders are first aggregated through the convolutional layer with a kernel size of 1, and then added element-wise to obtain the damage feature H j ′; for the features before and after damage perform the absolute value operation of subtraction to obtain the difference feature before and after damage. This difference feature and the previous damage feature H j'Continue to splice in the channel dimension and aggregate through a convolutional layer with a convolution kernel of 1 to achieve further enhancement, obtaining the final damage feature H j The final damage features H at different scales j Continuously upsample through the decoder and aggregate information at different scales, and finally obtain the predicted probability map P through the classifier cls .
[0059] In the remote sensing image building damage assessment method based on multi-task learning, the decoders used in the building location task and the damage classification task adopt the same structure but do not share weights. The classifier structures of the two tasks are similar and are both composed of two layers of pixel-level convolutional layers. A Softmax(·) activation function is connected to the last layer to convert the final feature map into the corresponding probability. Among them, the number of channels of the output result of the damage location task is 2, corresponding to 1 and 0, which are used to represent the building and the background. The number of channels of the output result of the damage classification task is 5, corresponding to 0-4. 0 represents the background class, 1 represents that the building has no damage, 2 represents that the building has minor damage, 3 represents that the building has moderate damage, and 4 represents that the building is completely damaged.
[0060] In multi-task learning, two tasks with strong correlation can play a mutually promoting role. Therefore, in order to strengthen the connection between the two tasks, the predicted probability results P of the classification task and the location task cls and P loc are further mutually constrained in the task interaction module. Specifically, since the building location task is a binary classification task, which is simpler than the five-classification task in the damage classification task, it is more convincing in terms of detection accuracy. In the present invention, P in this module loc is used to constrain P cls , that is, to make the credibility of P cls approach that of P loc . First, the class with the largest probability in P loc is used as the predicted result R of building location loc , as shown in the formula:
[0061]
[0062] where MSE(·) represents the mean square error, and r loc represents the category (0 or 1) of each pixel in R loc , and p cls and p loc represent the probabilities of each category at each pixel position in P cls and P loc ; represents the operation of gradient truncation, that is is p locThe representation after gradient truncation. During model optimization, the model does not optimize with respect to in the loss function. p cls (x) and p loc (x) represent the probability values for class x, and maxhs(·) represents the class with the highest probability among all damage classes after removing the background class 0 in the probability prediction map p cls in the damage classification task (i.e., only taking 1, 2, 3, 4).
[0063] Finally, the loss function of the remote sensing image building damage assessment method based on multi-task learning is as shown in the formula:
[0064] L = L CE (P loc , Y loc ) + L CE (P cls , Y hs ) + λL task
[0065] Taking L = L CE (P loc , Y loc ) + L CE (P cls , Y hs ) + λL task as the loss function to optimize the building damage assessment model, and obtaining the optimized building damage assessment model; where Y loc is a binary label modified based on Y hs (i.e., setting the non-background classes of Y hs to 1 and the background class to 0), L CE (P loc , Y loc ) and L CE (P hs , Y hs ) are the cross-entropy loss functions of the prediction results of the final building damage localization task and building damage classification task and their corresponding labels, and λ is a hyperparameter for adjusting the weight of L task ; it should be noted that: in the damage detection task of the present invention, L CE (P hs , Y hs ) in the loss function adds the loss function L CE (P loc , Y loc) There are two reasons for doing this: First, since the encoder part is shared by the two tasks, optimizing the model through the two loss functions enables the encoder part of the model to not only learn the building recognition ability but also acquire the ability to classify the damage situation of the building. Second, when the correlation between the two tasks is strong, the two tasks can promote each other. Based on this multi-task learning strategy, the present invention introduces L task Strengthens the connection between the damage location task and the damage classification task, enabling the two tasks to inspire each other and further optimizing the model.
[0066] During the training process, the performance of the building damage assessment model is verified through the pre-divided validation set. After each training ends, the validation set data is fed into the model to obtain the prediction probability P of the validation set cls , and take P cls The category with the highest probability in is used as the final prediction result R cls , and then the model weight with the highest mean intersection over union (mIoU) in the validation set is used as the finally selected model weight. Therefore, before optimizing the model, the data needs to be divided into a training set, a validation set, and a test set. The training set is used to train the model, the validation set verifies the model trained by each round of the training set, and the result of the time with the highest mIoU is taken as the final result. Then, the test set is tested. The calculation method of mIoU is as shown in the formula:
[0067]
[0068] Among them, in IoU j , j being 0 represents the background class, 1 represents that the building has no damage, 2 represents that the building has minor damage, 3 represents that the building has moderate damage, and 4 represents that the building has complete damage.
[0069] TP is the true positive: For category j, TP means that the model correctly predicts the sample belonging to category j as category j;
[0070] FP is the false positive: For category j, FP means that the model wrongly predicts the sample not belonging to category ì as category j;
[0071] FN is the false negative: For category j, FN means that the model wrongly predicts the sample belonging to category j as other categories that are not category j.
[0072] Step Four: Input the remotely sensed image of the building damage to be measured into the selected optimal building damage assessment model, output the building damage prediction probability P corresponding to the remotely sensed image of the building damage to be measured cls , and take the category with the highest prediction probability as the final prediction result to obtain R clsNote that the damage location task is only used to assist in optimizing the model during the training phase. During the testing phase, the final prediction results can be directly obtained from the encoder part and the decoder part of the damage classification task for the image data stream.
[0073] The segmentation evaluation results of the present invention are as Figure 4 shown, where the first row is the pre-damage image (optical), the second row is the post-damage image (SAR), the third row is the label image, and the fourth row is the segmentation image of the present invention. It can be seen that the method of this embodiment can achieve excellent results in building damage assessment. Specific Embodiment 2:
[0075] This embodiment is a computer storage medium, and at least one instruction is stored in the storage medium. The at least one instruction is loaded and executed by a processor to implement the method for remote sensing image building damage assessment based on multi-task learning described above.
[0076] It should be understood that the instruction includes a computer program product, software, or computerized method corresponding to any method described in the present invention; the instruction can be used to program a computer system or other electronic devices. The computer storage medium can include a readable medium on which the instruction is stored, which can include but is not limited to a magnetic storage medium and an optical storage medium; the magneto-optical storage medium includes a read-only memory ROM, a random access memory RAM, an erasable programmable memory (e.g., EPROM and EEPROM), and a flash memory layer, or other types of media suitable for storing electronic instructions. Specific Embodiment 3:
[0078] This embodiment is a device for remote sensing image building damage assessment based on multi-task learning. The device includes a processor and a memory. It should be understood that the device includes any device including a processor and a memory described in the present invention. The device may further include other units and modules for display, interaction, processing, control, etc. through signals or instructions, and other functions.
[0079] At least one instruction is stored in the memory, and the at least one instruction is loaded and executed by the processor to implement the method for remote sensing image building damage assessment based on multi-task learning described above.
[0080] Those skilled in the art should understand that at least one stored instruction is a computer program product corresponding to a method or system. Therefore, the present application can be implemented in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be implemented in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present application can be implemented using various computer languages. For example, object-oriented programming languages such as Java and interpreted scripting languages such as JavaScript.
[0081] The present application is described with reference to the flowcharts and / or block diagrams of methods, systems, and computer program products according to the embodiments of the present application, and can also be used for corresponding devices. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.
[0082] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.
[0083] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.
[0084] Although the preferred embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they know the basic creative concepts. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0085] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these modifications and variations.
[0086] The above numerical examples of the present invention are only for illustrating in detail the calculation model and calculation process of the present invention, rather than limiting the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or alterations can be made on the basis of the above description. It is impossible to list all the implementation manners here. Any obvious changes or alterations derived from the technical solutions of the present invention still fall within the protection scope of the present invention.
Claims
1. A remote sensing image building damage assessment method based on multi-task learning, characterized in that: For the building to be assessed for damage, remote sensing images before and after the building damage are obtained, and damage assessment is performed using a building damage assessment model. The process of performing damage assessment using the building damage assessment model includes: Remote sensing images before and after building damage are sent to encoders for feature extraction to extract feature maps of different scales before and after damage. Where i=1,2 represents the features before and after damage, j=1,2,…,J represents feature maps of different scales, and J represents the last J layers of the encoder, that is, the feature map is the feature map obtained by the last J layers of the encoder; Building positioning: Features of different scales before damage are continuously upsampled through the decoder. The decoder of each layer upsamples the input and aggregates feature information of different scales through skip connections. The structure of the decoder is shown in the formula: Among them, D j represents the output of the j-th layer decoder, [·] represents the concatenation operation in the channel dimension, Up(·) represents the upsampling layer of bilinear interpolation, and Conv represents the convolution unit; The last layer output D1 of the decoder is sent to the building positioning task classifier to obtain the predicted probability map P loc , and then get the building positioning result; Damage degree classification: The characteristic images before and after the building damage are processed as follows to obtain the building damage feature H j : Among them, DLM stands for damage location module, and its processing process includes: first, according to get and Then The damage feature H is obtained by passing through the convolution layer with the convolution kernel of 1 and then adding them element by element. j ';and Subtract and take the absolute value operation to obtain the difference feature before and after the damage. The difference feature and feature H j ′ is concatenated in the channel dimension and passed through a convolution layer with a convolution kernel of 1 to obtain the final damage feature H j ; Damage characteristics of different scales H j The decoder with the same structure as the building positioning and no shared parameters continuously upsamples and aggregates information of different scales, and finally obtains the prediction probability map P through the damage degree classification classifier. cls , and then obtain the damage degree classification result.
2. According to the multi-task learning-based remote sensing image building damage assessment method of claim 1, it is characterized in that: The encoder for processing the remote sensing image before the building is damaged has the same structure as the encoder for processing the remote sensing image after the building is damaged, and the parameters are not shared.
3. The method for building damage assessment based on remote sensing images based on multi-task learning according to claim 2 is characterized in that: The J is 3.
4. The method for building damage assessment based on remote sensing images based on multi-task learning according to claim 3 is characterized in that: The encoder is ResNet18.
5. The method for building damage assessment based on remote sensing images based on multi-task learning according to claim 1, characterized in that: The convolution unit Conv in the decoder includes a convolution layer with a convolution kernel of 1, an activation function ReLU(·), and a regularization layer.
6. The method for building damage assessment based on remote sensing images based on multi-task learning according to claim 1, characterized in that: The decoder for building localization includes two pixel-level convolutional layers and a Softmax(·) layer.
7. A remote sensing image building damage assessment method based on multi-task learning according to any one of claims 1 to 6, characterized in that: The building damage assessment model is obtained through the following training process: First, a building damage assessment dataset is obtained and divided into a training set and a validation set; the building damage assessment dataset includes a remote sensing image before building damage and a damage label map Y corresponding to the remote sensing image after building damage hs ,The damage label diagram is the classification of damage degree; For the images in the training set, the predicted probability map P of the building positioning task classifier is obtained by processing loc and the predicted probability map P of the damage classification classifier cls ; The loss function L of remote sensing image building damage assessment based on multi-task learning is used to optimize the building damage assessment model to obtain the optimized building damage assessment model; the loss function L is as follows: L=L CE (P loc ,Y loc )+L CE (P cls ,Y hs )+λL task Where MSE(·) represents the mean square error, r loc Represents R loc The category of each pixel in cls and p loc Indicates P cls and P loc The probability of each category at each pixel position in ; represents the operation of gradient truncation, that is Yes loc Representation after gradient clipping; p cls (x) and p loc (x) represents the probability value of category x, and maxhs(·) represents the probability prediction graph p in the damage classification task. cls After removing the background category 0, the category with the highest probability among all damage categories; Y loc Y hs The binary label modified based on hs The non-background class is set to 1, and the background class is set to 0; L CE (P loc ,Y loc ) and L CE (P hs ,Y hs ) is the cross entropy loss function between the prediction results of the final building damage location task and the building damage classification task and their corresponding labels, and λ is the adjustment L task Hyperparameters of weights; During the training process, the performance of the building damage assessment model is verified through the validation set, and the final building damage assessment model is determined based on the validation set.
8. The method for building damage assessment based on remote sensing images based on multi-task learning according to claim 7 is characterized in that: In the process of determining the final building damage assessment model based on the validation set, the model weight with the highest mIoU in the validation set is used as the final selected model weight; The calculation method of mIoU is: Among them, in IoU j is the intersection-over-union ratio indicator for category j, TP is a true positive example: for category j, TP means that the model correctly predicts samples belonging to category j as category j; FP is a false positive example: for category j, FP means that the model incorrectly predicts samples that do not belong to category i as category j; FN is a false negative example: for category j, FN means that the model incorrectly predicts samples belonging to category j as other categories that are not category j.
9. A computer storage medium, characterized in that: The storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the remote sensing image building damage assessment method based on multi-task learning as described in any one of claims 1 to 8.
10. A remote sensing image building damage assessment device based on multi-task learning, characterized in that: The device includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement a remote sensing image building damage assessment method based on multi-task learning as described in any one of claims 1 to 8.