Mask R-CNN-based high-frequency detail enhancement remote sensing image feature extraction improvement method

By adding high-frequency detail enhancement module and multi-scale feature fusion module to the Mask R-CNN feature extractor, combined with context information extraction, the shortcomings of traditional algorithms in high-frequency details and multi-scale feature extraction are solved, and the satellite is significantly improved Image recognition accuracy and robustness.

CN118982677APending Publication Date: 2024-11-19河南省种业发展中心
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411078566.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

The traditional Mask R-CNN algorithm has recognition accuracy and performance bottlenecks when processing satellite images, and cannot effectively capture high-frequency details and small-scale features. It has poor recognition effect in complex backgrounds, which is prone to overfitting or insufficient generalization capabilities. question.

Method used

Adding high-frequency detail enhancement module (HFDEM) in the feature extractor ResNet of Mask R-CNN, and adding multi-scale feature fusion module (MSFFM) in the feature pyramid network FPN to enhance the extraction of high-frequency details and multi-scale features and introduce a context information extraction module to improve feature fusion.

Benefits of technology

It significantly improves the richness and detection accuracy of feature representation, improves the recognition ability of small targets and the robustness of complex backgrounds, and enhances the detection accuracy and segmentation quality of small targets in high-resolution satellite images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118982677A_ABST
    Figure CN118982677A_ABST
Patent Text Reader

Abstract

The invention discloses a Mask R-CNN-based high-frequency detail enhancement remote sensing image feature extraction improved method, which comprises the steps of collecting and labeling an original high-resolution satellite image containing a recognition target, and preprocessing and enhancing the labeled original high-resolution satellite image to obtain a recognition target data set; dividing the recognition target data set into a training set and a verification set; constructing an improved high-frequency multi-scale remote sensing image processing model; training and verifying the improved high-frequency multi-scale remote sensing image processing model by using the training set and the verification set to obtain a trained and improved high-frequency multi-scale remote sensing image processing model; and inputting a to-be-recognized satellite image into the trained and improved high-frequency multi-scale remote sensing image processing model to obtain a recognition target extraction result of the to-be-recognized satellite image. According to the method, the problem that a traditional Mask R-CNN algorithm loses high-frequency details in the multi-layer convolution and pooling process is solved, detection is more accurate, and noise robustness is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of high-frequency detail enhanced remote sensing image feature extraction, and in particular to an improved method for high-frequency detail enhanced remote sensing image feature extraction based on Mask R-CNN. Background Art

[0002] Satellite images have unique high-frequency details and multi-scale characteristics, and feature extractors need to be improved to better capture this information. The traditional Mask R-CNN algorithm encounters recognition accuracy and performance bottlenecks when processing these images. Therefore, the existing algorithm needs to be optimized and improved to improve its accuracy and efficiency in satellite imagery farm identification tasks.

[0003] Many existing models, such as the basic ResNet, may have limited capabilities in processing complex objects and details and cannot effectively capture high-frequency details and small-scale features. The basic FPN may not fully integrate the features of each layer when processing multi-scale features, resulting in poor recognition of objects of different scales.

[0004] The invention with application number 202310873327.9 discloses a remote sensing image landslide recognition method based on a deep learning method, the method comprising: collecting satellite remote sensing image data with landslide areas, and obtaining a public remote sensing image landslide data set; preprocessing the collected satellite remote sensing image data with landslide areas, cropping and classifying them to obtain a classified and reorganized data set; then performing data enhancement and landslide sample annotation; then constructing an improved landslide recognition model based on Mask R-CNN, and training and verifying it; finally, using the improved landslide recognition model based on Mask R-CNN to perform landslide recognition on the image to be recognized, and obtaining a recognition result. The improved landslide recognition model based on Mask R-CNN changes the feature extraction network ResNet in the Mask R-CNN model to a feature extraction network ResNeSt, changes the feature fusion structure FPN in the Mask R-CNN model to a feature fusion structure RFP, and changes the RPN structure in the Mask R-CNN model to the detection head of the single-stage target detection model PISA-RetinaNet. The improved model of the above invention ignores the global context information and only focuses on local features, resulting in poor recognition effect in complex backgrounds; and the training process is not fully optimized and adjusted for specific tasks, which is prone to problems of overfitting or insufficient generalization ability. Summary of the invention

[0005] Aiming at the technical problems of recognition accuracy and performance bottlenecks when the traditional Mask R-CNN algorithm processes satellite images, the present invention proposes an improved method for high-frequency detail enhanced remote sensing image feature extraction based on Mask R-CNN. The present invention improves the feature extractors ResNet and FPN of Mask R-CNN, adds a high-frequency detail enhancement module (HFDEM) in the residual network ResNet, and enhances its ability to extract high-frequency details and multi-scale features. The feature pyramid network FPN performs feature fusion between feature maps of different scales, and adds a multi-scale feature fusion module (MSFFM), which can improve the feature pyramid network FPN to better adapt to the multi-scale characteristics of satellite images.

[0006] In order to achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] A high-frequency detail enhanced remote sensing image feature extraction method based on Mask R-CNN includes the following steps:

[0008] S1. Collect and annotate original high-resolution satellite images containing identification targets, preprocess and enhance the annotated original high-resolution satellite images to obtain identification target datasets, and divide the identification target datasets into training sets and validation sets;

[0009] S2. Construct an improved high-frequency multi-scale remote sensing image processing model. The improved high-frequency multi-scale remote sensing image processing model is to add a high-frequency detail enhancement module HFDEM to the residual network ResNet of the Mask R-CNN model, add a multi-scale feature fusion module MSFFM to the feature pyramid network FPN, and introduce a context information extraction module after the multi-scale feature fusion module MSFFM;

[0010] S3, using the training set and the validation set to train and validate the improved high-frequency multi-scale remote sensing image processing model, to obtain a trained improved high-frequency multi-scale remote sensing image processing model;

[0011] S4. Input the satellite image to be identified into the trained improved high-frequency multi-scale remote sensing image processing model to obtain the identification target extraction result of the satellite image to be identified.

[0012] The preprocessing includes cutting the original high-resolution satellite image into smaller sub-images, standardizing the image pixel values ​​of the sub-images, adjusting the colors and enhancing the contrast of the sub-images under different lighting conditions, and applying bilateral filtering of a filter to remove noise in the sub-images.

[0013] The method for constructing the improved high-frequency multi-scale remote sensing image processing model is as follows: first, a high-frequency detail enhancement module HFDEM is added to the residual network ResNet; second, a multi-scale feature fusion module MSFFM is added to the feature pyramid network FPN; then, a context information extraction module is introduced after the multi-scale feature fusion module MSFFM; finally, a loss function and an optimizer are defined. When defining the loss function and the optimizer, two loss functions, classification loss and regression loss, are used. A cross entropy loss function is defined for classification tasks, and a smooth L1 loss function is defined for regression tasks. An optimizer is selected and a learning rate scheduling is performed.

[0014] The enhancement method is: randomly rotating the sub-image by 0° to 360°, randomly translating the sub-image, adjusting the brightness and contrast of the sub-image, adding Gaussian noise to the sub-image, and finally performing pseudo-color processing to change the hue, saturation and brightness of the sub-image.

[0015] The specific steps of the high frequency detail enhancement module HFDEM processing the feature map output by the residual network ResNet are as follows:

[0016] ⅰ. Take the feature map of a layer in the residual network ResNet as input x;

[0017] ii. Perform Gaussian blur processing on the input feature map x to obtain a blurred image GaussianBlur(x)=G(x), where G is the Gaussian blur operation;

[0018] iii. Subtract the blurred image from the input feature map x to calculate the high-frequency detail map HF(x) = xG(x);

[0019] iv. Fusing the high-frequency detail map with the input feature map and the output of the standard residual block to obtain the output feature map y = F(x, {W i})+x+HF(x),F(x,{W i}) represents the feature map after being processed by the standard ResNet residual block;

[0020] ⅴ. Perform ReLU activation on the output feature map to obtain the final output feature map y = ReLU (y).

[0021] The process of obtaining the output feature map of the feature pyramid network FPN after adding the multi-scale feature fusion module MSFFM is as follows: through 1×1 convolution, the number of channels of the feature maps of each scale is adjusted to be consistent. The feature maps after adjusting the channels are P′3, P′4, P′5, and P′6. The formula is: P′ i =Conv 1×1 (P i), i = 3, 4, 5, 6, the multi-scale feature maps after adjusting the channel are spliced ​​and fused in the channel dimension to obtain the fused feature map P fused =Concat(P′3,P′4,P′5,P′6), use 3×3 convolution to fused feature map P fused Perform feature extraction and noise reduction to obtain the final output fusion feature map P output =Conv 3×3 (P fused ).

[0022] The specific steps of the context information extraction module processing the fusion feature map output by the multi-scale feature fusion module MSFFM are as follows: inputting the fusion feature map P output by the multi-scale feature fusion module MSFFM output , extract the fusion feature map P through the global pooling operation output The global context information C global =GlobalAveragePooling(P output ), extract local context information C through convolution operation local =Conv 3×3 (P output ), the global context information C global and local context information C local Fusion is performed to obtain the enhanced feature map C fused =Concat(C global ,C local ), use 1×1 convolution to enhance the feature map C fused Processing is performed to obtain the final context-enhanced feature map C output =Conv 1×1 (C fused ). GlobalAveragePooling,Conv 3×3 、Concat、Conv 1×1 They are global pooling operation, 3×3 convolution operation, connection operation, and 1×1 convolution operation.

[0023] The specific process of training the improved high-frequency multi-scale remote sensing image processing model is as follows: the data of the training set is standardized and enhanced; in the forward propagation stage, the data of the input training set is passed through the model structure of the improved high-frequency multi-scale remote sensing image processing model, and the classification and regression losses are calculated respectively using the cross entropy loss function and the smooth L1 loss function; in the back propagation stage, the gradient is cleared and then the gradient is calculated, and the parameters of the improved high-frequency multi-scale remote sensing image processing model are updated, and the learning rate is dynamically adjusted.

[0024] The method for verifying the improved high-frequency multi-scale remote sensing image processing model is as follows: switching the improved high-frequency multi-scale remote sensing image processing model to evaluation mode and disabling gradient calculation, inputting the verification set data into the improved high-frequency multi-scale remote sensing image processing model for forward propagation, calculating the loss using the classification loss function and the regression loss function, and recording the accumulated loss and the number of accurately predicted samples, accumulating the loss and the number of correct predictions for each batch, and finally calculating the average loss and the overall accuracy of the verification set to complete the verification.

[0025] The specific process of step S4 inputting the satellite image to be identified into the trained improved high-frequency multi-scale remote sensing image processing model to obtain the identification target extraction result of the satellite image to be identified is as follows: loading the weights of the trained improved high-frequency multi-scale remote sensing image processing model and switching to the evaluation mode, performing the same preprocessing as step S1 on the input satellite image to be identified, and then inputting the improved high-frequency multi-scale remote sensing image processing model for forward propagation to obtain the prediction result, interpreting the identified identification target according to the classification and regression results output by the improved high-frequency multi-scale remote sensing image processing model, and displaying the detected bounding box through a visualization tool.

[0026] Beneficial effects of the present invention:

[0027] 1. By adding a high-frequency detail enhancement module HFDEM to the ResNet residual network, the present invention can extract, retain and enhance high-frequency details, significantly improve the richness of feature representation, help to better identify small and complex targets, and solve the problem that the traditional Mask R-CNN algorithm may lose high-frequency details in the process of multi-layer convolution and pooling. For small targets in high-resolution satellite images, HFDEM can provide higher detection accuracy and segmentation quality, making detection more accurate and improving monitoring accuracy.

[0028] 2. The present invention is noise robust and can filter out low-frequency background noise by enhancing high-frequency details, highlighting useful information, thereby improving the robustness of the model to complex backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 This is a flow chart of the improved method for extracting high-frequency detail enhanced remote sensing image features based on Mask R-CNN of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] like Figure 1 As shown, a method for extracting features from high-frequency detail-enhanced remote sensing images based on Mask R-CNN is provided. This embodiment takes the identification of farms as an example and includes the following steps:

[0033] S1. Collect and annotate original high-resolution satellite images containing the identified target (a farm in this embodiment), preprocess and enhance the annotated original high-resolution satellite images to obtain an identified target dataset, and divide the identified target dataset into a training set and a validation set.

[0034] The identified target data set includes annotated label data and satellite image data obtained by preprocessing and enhancing the original high-resolution satellite image containing the identified target.

[0035] Original high-resolution satellite images containing farms are collected from public remote sensing data sources. The farm areas are manually annotated using the VGG Image Annotator tool to generate annotation files required for target detection and segmentation in JSON format.

[0036] The preprocessing includes using the cv2.resize function of OpenCV to crop the original high-resolution satellite images into smaller sub-images to adapt to the model input size, defining the normalize_image(image) method, inputting the cropped sub-image, normalizing the image pixel values ​​of the sub-image so that its distribution conforms to the standard normal distribution (mean is 0, standard deviation is 1), and using the function cv2.convertScaleAbs(image,alpha=1.2,beta=0) to adjust the color and enhance the contrast of the sub-image for different lighting conditions. Use the function cv2.medianBlur(image,3) to apply the bilateral filtering of the filter to remove the noise in the sub-image and improve the image quality of the sub-image.

[0037] The enhancement method is as follows: introducing the Albumentations library, using the RandomRotate90 function to randomly rotate the sub-image image by a certain angle (between 0° and 360°), using the ShiftScaleRotate function to randomly translate the sub-image image, using the RandomBrightnessContrast function to adjust the brightness and contrast of the sub-image image, using the IAAAdditiveGaussianNoise function to add Gaussian noise to the sub-image image, and finally performing pseudo-color processing, using the HueSaturationValue function to change the hue, saturation and brightness of the sub-image image, thereby improving data quality and diversity and laying a good foundation for subsequent feature extraction and model training.

[0038] S2. Construct an improved high-frequency multi-scale remote sensing image processing model. The improved high-frequency multi-scale remote sensing image processing model is to add a high-frequency detail enhancement module HFDEM to the residual network ResNet of the Mask R-CNN model, add a multi-scale feature fusion module MSFFM to the feature pyramid network FPN, and introduce a context information extraction module after the multi-scale feature fusion module MSF FM.

[0039] Firstly, the high-frequency detail enhancement module HFDEM is added to the standard ResNet through the function ImprovedResNet(original_resnet) to enhance the high-frequency detail information. Secondly, the multi-scale feature fusion module MSFFM is added to FPN through the function ImprovedFPN(original_fpn) to better fuse the feature maps of different scales. Then, the context information extraction module is introduced through the function ContextModule() to extract global and local context information and fuse it into the feature map, thereby improving the recognition ability of the improved high-frequency multi-scale remote sensing image processing model for targets in complex scenes. The high-frequency detail enhancement module (HFDEM), the multi-scale feature fusion module (MSFFM) and the context information extraction module are tightly integrated through specific convolution, pooling and fusion operations to achieve efficient feature extraction and multi-scale information processing of the input image.

[0040] Finally, define the loss function and optimizer. When defining the loss function and optimizer, two loss functions, classification loss and regression loss, are used. The appropriate optimization strategy Adam (adaptive moment estimation) strategy is selected. The Adam optimizer combines adaptive learning rate and momentum, which can converge quickly and effectively handle sparse gradients. The cross entropy loss function (criterion_cls) is defined by the function nn.CrossEntropyLoss() for classification tasks, and the smooth L1 loss function (criterion_reg) is defined by the function nn.SmoothL1Loss() for regression tasks. In order to optimize the model parameters, torch.optim.Adam (model.parameters(), lr = 0.001) is used as the optimizer, and torch.optim.lr_scheduler.StepLR (optimizer, step_size = 7, gamma = 0.1) is used for learning rate scheduling to ensure dynamic adjustment of the learning rate during training, thereby improving the training effect and stability of the model.

[0041] Furthermore, adding a high-frequency detail enhancement module HFDEM on the basis of the residual network ResNet can improve the high-frequency detail information in the feature extraction stage, enhance the high-frequency details in the image, and enhance the detection ability of the improved high-frequency multi-scale remote sensing image processing model for details and small targets, thereby improving the performance of the improved high-frequency multi-scale remote sensing image processing model in detecting small targets such as farms or targets with complex details in remote sensing images.

[0042] The specific steps of the high frequency detail enhancement module HFDEM processing the feature map output by the residual network ResNet are as follows:

[0043] ⅰ. Take the feature map of a layer in the residual network ResNet as input x.

[0044] ⅱ. Perform Gaussian blur processing on the input feature map x to obtain the blurred image GaussianBlur(x). The formula is:

[0045] GaussianBlur(x)=G(x) (1)

[0046] Where G is the Gaussian blur operation.

[0047] ⅲ. Subtract the blurred image from the input feature map to calculate the high-frequency detail map HF(x).

[0048] The formula is:

[0049] HF(x)=xG(x) (2)

[0050] iv. Fuse the high-frequency detail map with the input feature map and the output of the standard residual block to obtain the output feature map:

[0051] y=F(x,{W i})+x+HF(x) (3)

[0052] F(x,{W i}) represents the feature map after being processed by the standard ResNet residual block.

[0053] ⅴ. Perform ReLU activation on the output feature map to obtain the final output feature map:

[0054] y=ReLU(y) (4)

[0055] Furthermore, a multi-scale feature fusion module MSFFM is added to the feature pyramid network FPN. By fusing feature maps of different scales, the detection ability of the improved high-frequency multi-scale remote sensing image processing model for targets of different scales can be enhanced, thereby improving the overall detection effect.

[0056] The feature pyramid network FPN with the multi-scale feature fusion module MSFFM is added to adjust the number of channels of the feature map of each scale. Through 1×1 convolution, the number of channels of the feature map of each scale is adjusted to be consistent. The feature maps after adjusting the channels are P′3, P′4, P′5, and P′6. The formula is:

[0057] P′ i =Conv 1×1 (P i ),i=3,4,5,6 (5)

[0058] The multi-scale feature maps after adjusting the channel are spliced ​​and fused in the channel dimension to obtain the fused feature map P fused , the formula is:

[0059] P fused =Concat(P′3,P′4,P′5,P′6) (6)

[0060] Use 3×3 convolution to fused feature map P fused Perform feature extraction and noise reduction to obtain the final output fusion feature map P output , the formula is:

[0061] P output =Conv 3×3 (P fused ) (7)

[0062] By fusing feature maps of different scales, MSFFM makes full use of the information of feature maps of each scale and improves the model's ability to detect targets of different scales. Compared with traditional methods that only use single-scale feature maps for detection, MSFFM can better handle targets of different sizes and shapes. 1×1 convolution is used to adjust the channels of feature maps of each scale to ensure effective fusion in the channel dimension. Compared with the traditional simple addition operation, this fusion strategy can better retain and integrate multi-scale information and improve the expression ability of the fused feature map. After the feature map is fused, it is processed using 3×3 convolution to further extract effective features and perform noise reduction, so that the final output feature map has a stronger representation ability, which is helpful for subsequent target detection tasks.

[0063] Furthermore, the context information extraction module is closely related to the multi-scale feature fusion module MSFFM added in the feature pyramid network FPN. The context information extraction module enhances the expression ability of the feature map by capturing the extensive context information in the input image of the improved high-frequency multi-scale remote sensing image processing model, and improves the understanding and recognition ability of the improved high-frequency multi-scale remote sensing image processing model for identifying targets. The specific steps of the context information extraction module processing the fusion feature map output by MSFFM are as follows:

[0064] Input feature map, the input feature map comes from the fused feature map P output by the multi-scale feature fusion module MSFFM output .

[0065] Extract the fused feature map P through the global pooling operation output The global context information C global , the formula is:

[0066] C global =GlobalAveragePooling(P output ) (8)

[0067] Extract local context information C through convolution operation local , enhance the ability to express details and local features, the formula is:

[0068] C local =Conv 3×3 (P output ) (9)

[0069] The global context information C global and local context information C local Fusion is performed to obtain the enhanced feature map C fused , the formula is:

[0070] C fused =Concat(C global ,C local ) (10)

[0071] Use 1×1 convolution to enhance the feature map C fused Processing is performed to obtain the final context-enhanced feature map C output , the formula is:

[0072] C output =Conv 1×1 (C fused ) (11)

[0073] GlobalAveragePooling, Conv 3×3 、Concat、Conv 1×1 They are global pooling operation, 3×3 convolution operation, connection operation, and 1×1 convolution operation.

[0074] S3. Use the training set and the validation set to train and validate the improved high-frequency multi-scale remote sensing image processing model to obtain a trained improved high-frequency multi-scale remote sensing image processing model.

[0075] The training steps of the improved high-frequency multi-scale remote sensing image processing model include data preprocessing, forward propagation, loss calculation, back propagation and parameter updating. The specific process of training the improved high-frequency multi-scale remote sensing image processing model is as follows: the training set data is standardized and enhanced through the function data_transforms.Compose([...]) to improve the generalization ability of the improved high-frequency multi-scale remote sensing image processing model. In the forward propagation stage, the input training set data is passed through the model structure of the improved high-frequency multi-scale remote sensing image processing model, including ImprovedResNet (origin al_resnet) and ImprovedFPN (original_fpn), to enhance the high-frequency details and multi-scale feature fusion capabilities.

[0076] The cross entropy loss function criterion_cls = nn.CrossEntropyLoss() and the smooth L1 loss function criterion_reg = nn.SmoothL1Loss() are used to calculate the classification and regression losses respectively.

[0077] In the back-propagation phase, the gradient is cleared through optimizer.zero_grad(), loss.backward() is called to calculate the gradient, and optimizer.step() is used to update the parameters of the improved high-frequency multi-scale remote sensing image processing model. At the same time, the learning rate is dynamically adjusted through scheduler.step(). When the loss function value tends to be stable, the training is completed.

[0078] During the verification process of the improved high-frequency multi-scale remote sensing image processing model, the improved high-frequency multi-scale remote sensing image processing model is switched to the evaluation mode model.eval(), and the gradient calculation with torch.no_grad() is disabled to save memory and computing time. The validation set data is input into the improved high-frequency multi-scale remote sensing image processing model for forward propagation, and the classification loss function criterion_cls and the regression loss function criterion_reg are used to calculate the loss, and the accumulated loss and the number of accurately predicted samples are recorded. By accumulating the loss and the number of correct predictions for each batch, the average loss and the overall accuracy of the validation set are finally calculated, so as to evaluate the performance of the improved high-frequency multi-scale remote sensing image processing model and prevent overfitting.

[0079] S4. Input the satellite image to be identified into the trained improved high-frequency multi-scale remote sensing image processing model to obtain the farm extraction result of the satellite image to be identified.

[0080] Load the trained weights of the improved high-frequency multi-scale remote sensing image processing model and switch to the evaluation mode. Perform the same preprocessing as step S1 on the input satellite image to be identified to match the input requirements of the improved high-frequency multi-scale remote sensing image processing model. Input the preprocessed satellite image to be identified into the improved high-frequency multi-scale remote sensing image processing model for forward propagation to obtain the prediction result. Finally, interpret the identified farms according to the classification and regression results output by the improved high-frequency multi-scale remote sensing image processing model, and display the detected bounding box through visualization tools such as matplotlib, so as to effectively identify and locate the farms in the satellite image to be identified.

[0081] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An improved method for high-frequency detail enhanced remote sensing image feature extraction based on Mask R-CNN, It is characterized in that The following steps are involved: S1. Collect and annotate original high-resolution satellite images containing identification targets, preprocess and enhance the annotated original high-resolution satellite images to obtain identification target datasets, and divide the identification target datasets into training sets and validation sets; S2. Construct an improved high-frequency multi-scale remote sensing image processing model. The improved high-frequency multi-scale remote sensing image processing model is to add a high-frequency detail enhancement module HFDEM to the residual network ResNet of the Mask R-CNN model, add a multi-scale feature fusion module MSFFM to the feature pyramid network FPN, and introduce a context information extraction module after the multi-scale feature fusion module MSFFM; S3, using the training set and the validation set to train and validate the improved high-frequency multi-scale remote sensing image processing model, to obtain a trained improved high-frequency multi-scale remote sensing image processing model; S4. Input the satellite image to be identified into the trained improved high-frequency multi-scale remote sensing image processing model to obtain the identification target extraction result of the satellite image to be identified.

2. The improved method for extracting high-frequency detail enhanced remote sensing image features based on Mask R-CNN according to claim 1, It is characterized in that The preprocessing includes cutting the original high-resolution satellite image into smaller sub-images, standardizing the image pixel values ​​of the sub-images, adjusting the colors and enhancing the contrast of the sub-images under different lighting conditions, and applying bilateral filtering of a filter to remove noise in the sub-images.

3. The improved method for extracting high-frequency detail enhanced remote sensing image features based on Mask R-CNN according to claim 2, It is characterized in that The method for constructing the improved high-frequency multi-scale remote sensing image processing model is as follows: first, a high-frequency detail enhancement module HFDEM is added to the residual network ResNet; second, a multi-scale feature fusion module MSFFM is added to the feature pyramid network FPN; then, a context information extraction module is introduced after the multi-scale feature fusion module MSFFM; finally, a loss function and an optimizer are defined. When defining the loss function and the optimizer, two loss functions, classification loss and regression loss, are used. A cross entropy loss function is defined for classification tasks, and a smooth L1 loss function is defined for regression tasks. An optimizer is selected and a learning rate scheduling is performed.

4. The improved method for extracting high-frequency detail enhanced remote sensing image features based on Mask R-CNN according to claim 2 or 3, It is characterized in that The enhancement method is: randomly rotating the sub-image by 0° to 360°, randomly translating the sub-image, adjusting the brightness and contrast of the sub-image, adding Gaussian noise to the sub-image, and finally performing pseudo-color processing to change the hue, saturation and brightness of the sub-image.

5. The improved method for extracting high-frequency detail enhanced remote sensing image features based on Mask R-CNN according to claim 4, It is characterized in that The specific steps of the high-frequency detail enhancement module HFDEM processing the feature map output by the residual network ResNet are as follows: ⅰ. Take the feature map of a layer in the residual network ResNet as input x; ii. Perform Gaussian blur processing on the input feature map x to obtain a blurred image GaussianBlur(x)=G(x), where G is the Gaussian blur operation; iii. Subtract the blurred image from the input feature map x to calculate the high-frequency detail map HF(x) = xG(x); iv. Fusing the high-frequency detail map with the input feature map and the output of the standard residual block to obtain the output feature map y = F(x, {W i })+x+HF(x),F(x,{W i }) represents the feature map after being processed by the standard ResNet residual block; ⅴ. Perform ReLU activation on the output feature map to obtain the final output feature map y = ReLU (y).

6. The improved method for extracting high-frequency detail enhanced remote sensing image features based on Mask R-CNN according to claim 5, It is characterized in that The process of obtaining the output feature map by adding the feature pyramid network FPN with the multi-scale feature fusion module MSFFM is as follows: the number of channels of the feature maps of each scale is adjusted to be consistent through 1×1 convolution, and the feature map after the adjustment of the channels is P 3 ′,P 4 ′,P 5 ′,P 6 ′, the formula is: P i ′=Conv 1×1 (P i ), i = 3, 4, 5, 6, the multi-scale feature maps after adjusting the channel are spliced ​​and fused in the channel dimension to obtain the fused feature map P fused =Concat(P 3 ′,P 4 ′,P 5 ′,P 6 ′), use 3×3 convolution to extract and denoise the fused feature map Pfused, and get the final output fused feature map P output =Conv 3×3 (P fused ).

7. The improved method for extracting high-frequency detail enhanced remote sensing image features based on Mask R-CNN according to claim 6, It is characterized in that The specific steps of the context information extraction module processing the fusion feature map output by the multi-scale feature fusion module MSFFM are as follows: inputting the fusion feature map P output by the multi-scale feature fusion module MSFFM output , extract the fusion feature map P through the global pooling operation output The global context information C global =GlobalAveragePooling(P output ), extract local context information C through convolution operation local =Conv 3×3 (P output ), the global context information C global and local context information C local Fusion is performed to obtain the enhanced feature map C fused =Concat(C global ,C local ), use 1×1 convolution to enhance the feature map C fused Processing is performed to obtain the final context-enhanced feature map C output =Conv 1×1 (C fused ), where GlobalAveragePooling, Conv 3×3 、Concat、Conv 1×1 They are global pooling operation, 3×3 convolution operation, connection operation, and 1×1 convolution operation.

8. The improved method for extracting high-frequency detail enhanced remote sensing image features based on Mask R-CNN according to claim 7, It is characterized in that The specific process of training the improved high-frequency multi-scale remote sensing image processing model is as follows: the data of the training set is standardized and enhanced, and in the forward propagation stage, the data of the input training set is passed through the model structure of the improved high-frequency multi-scale remote sensing image processing model, and the classification and regression losses are calculated respectively using the cross entropy loss function and the smooth L1 loss function; In the back-propagation stage, the gradient is calculated after the gradient is cleared, and the parameters of the improved high-frequency multi-scale remote sensing image processing model are updated, and the learning rate is dynamically adjusted.

9. The improved method for extracting high-frequency detail enhanced remote sensing image features based on Mask R-CNN according to claim 8, It is characterized in that The method for verifying the improved high-frequency multi-scale remote sensing image processing model is as follows: switching the improved high-frequency multi-scale remote sensing image processing model to evaluation mode and disabling gradient calculation, inputting the verification set data into the improved high-frequency multi-scale remote sensing image processing model for forward propagation, calculating the loss using the classification loss function and the regression loss function, and recording the accumulated loss and the number of accurately predicted samples, accumulating the loss and the number of correct predictions for each batch, and finally calculating the average loss and the overall accuracy of the verification set to complete the verification.

10. The improved method for extracting high-frequency detail enhanced remote sensing image features based on Mask R-CNN according to claim 1 or 9, It is characterized in that The specific process of step S4 inputting the satellite image to be identified into the trained improved high-frequency multi-scale remote sensing image processing model to obtain the identification target extraction result of the satellite image to be identified is as follows: loading the weights of the trained improved high-frequency multi-scale remote sensing image processing model and switching to the evaluation mode, performing the same preprocessing as step S1 on the input satellite image to be identified, and then inputting the improved high-frequency multi-scale remote sensing image processing model for forward propagation to obtain the prediction result, interpreting the identified identification target according to the classification and regression results output by the improved high-frequency multi-scale remote sensing image processing model, and displaying the detected bounding box through a visualization tool.

Citation Information

Cited By

  • Building extraction and roof identification method and device based on RGB satellite picture

    CN119832446A

  • High-resolution satellite remote sensing image power transmission tower extraction method

    CN119992366A

  • Mask R-CNN-based remote sensing image small-size aircraft target detection method

    CN121353739A