Remote sensing image target detection method and device, computer equipment and storage medium

Abstract

The invention discloses a remote sensing image target detection method and device, computer equipment and a storage medium, relates to the field of computer vision, and can perform image recognition on remote sensing images with randomly arranged targets and dense positions and accurately detect the positions of the targets in the remote sensing images. The method comprises the following steps: acquiring a to-be-detected remote sensing image, and inputting the to-be-detected remote sensing image into a target feature extraction network; mapping the to-be-detected remote sensing image into a high-frequency output feature map and a low-frequency output feature map of different levels based on a target feature extraction layer of a target feature extraction network, and performing feature fusion by adopting a feature pyramid network to obtain a first target feature map; inputting the first target feature map into a pre-constructed target detection frame generation module, so that the target detection frame generation module outputs a second target feature map; and inputting the second target feature map into a preset classifier for category classification and detection frame regression to obtain a target detection result.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a remote sensing image target detection method, device, computer equipment and storage medium. Background Art

[0002] With the continuous development of computer vision, the Faster R-CNN (Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks) target detection algorithm has emerged. The Faster R-CNN target detection algorithm can be used to detect targets in remote sensing images.

[0003] In the related art, the ZF network (Zeiler and Fergus Network) and the VGG16 network are often used as feature extraction networks of the Faster R-CNN target detection algorithm to extract features from the input image. The RPN network generates candidate regions from the input feature map, and then performs target detection on each candidate region, and corrects the border of the region where the target exists to obtain a more accurate target frame. The RoiHead module is consistent with the target detection network in the Fast R-CNN network, and realizes the target category classification and precise adjustment of the bounding box for each candidate box generated by the RPN module.

[0004] In the process of implementing this application, the applicant found that the related technology has at least the following problems:

[0005] The high resolution of remote sensing images can capture more detailed information, but the target to be detected will become smaller. At the same time, the background of remote sensing images may contain a variety of objects and complex landform features, such as trees, water bodies, mountains, etc. These complex backgrounds may interfere with the detection and segmentation of targets, resulting in low detection accuracy of remote sensing images using the Faster R-CNN target detection algorithm. Summary of the invention

[0006] In view of this, the present application provides a remote sensing image target detection method, device, computer equipment and storage medium, the main purpose of which is to solve the problem that the background of the current remote sensing image may contain a variety of objects and complex landform features, such as trees, water bodies, mountains, etc. These complex backgrounds may interfere with the detection and segmentation of the target, resulting in a low detection accuracy of the target detection in the remote sensing image using the Faster R-CNN target detection algorithm.

[0007] According to the first aspect of the present application, a remote sensing image target detection method is provided, the method comprising:

[0008] Acquire a remote sensing image to be detected, and input the remote sensing image to be detected into a target feature extraction network, wherein the target feature extraction network is constructed based on a specified residual network, and the specified residual network is a ResNet101 network;

[0009] Based on the target feature extraction layer of the target feature extraction network, the remote sensing image to be detected is mapped into high-frequency output feature maps and low-frequency output feature maps at different levels, and a feature pyramid network is used to perform feature fusion to obtain a first target feature map;

[0010] Inputting the first target feature map into a pre-built target detection frame generation module, so that the target detection frame generation module outputs a second target feature map, wherein the second target feature map is marked with a plurality of target detection frames;

[0011] The second target feature map is input into a preset classifier for category classification and detection frame regression to obtain a target detection result.

[0012] Optionally, the target feature extraction layer based on the target feature extraction network maps the remote sensing image to be detected into high-frequency output feature maps and low-frequency output feature maps at different levels, and uses a feature pyramid network to perform feature fusion to obtain a first target feature map, including:

[0013] For the first convolution layer in the target feature extraction layer, the remote sensing image to be detected is mapped to a remote sensing feature map to be detected through the first convolution layer, and according to a preset division factor, the remote sensing feature map to be detected is mapped to a high-frequency input feature map and a low-frequency input feature map, and an octave convolution calculation is performed on the high-frequency input feature map and the low-frequency input feature map to obtain a high-frequency output feature map and a low-frequency output feature map corresponding to the first convolution layer, wherein the resolution of the low-frequency input feature map is one-fourth of the high-frequency input feature map, and the length and width of the low-frequency input feature map are respectively one-half of the high-frequency input feature map, wherein the first convolution layer is a top convolution layer;

[0014] For the second convolution layer in the target feature extraction layer, the high-frequency output feature map and the low-frequency data feature map output by the first convolution layer are received through the second convolution layer, and the high-frequency output feature map and the low-frequency output feature map corresponding to the first convolution layer are performed with octave convolution calculation to obtain the high-frequency output feature map and the low-frequency output feature map corresponding to the second convolution layer, and the second convolution layer is the next layer of the first convolution layer;

[0015] For the third convolution layer in the target feature extraction layer, an octave convolution calculation is performed on the high-frequency output feature map and the low-frequency output feature map output by the second convolution layer through the third convolution layer, until the last convolution layer, to obtain the high-frequency output feature map and the low-frequency output feature map corresponding to each convolution layer;

[0016] A pyramid feature network is used to perform feature fusion on the high-frequency output feature map and the low-frequency output feature map corresponding to each convolutional layer to obtain the first target feature map.

[0017] Optionally, receiving the high-frequency output feature map and the low-frequency data feature map output by the first convolution layer through the second convolution layer, performing octave convolution calculation on the high-frequency output feature map and the low-frequency output feature map corresponding to the first convolution layer, and obtaining the high-frequency output feature map and the low-frequency output feature map corresponding to the second convolution layer, includes:

[0018] Determine a high-frequency output feature map and a low-frequency output feature map output by the first convolutional layer, perform frequency internal convolution calculations on the high-frequency output feature map and the low-frequency output feature map, respectively, to obtain a high-frequency internal convolution result and a low-frequency internal convolution result;

[0019] Performing a pooling downsampling operation on the high-frequency output feature map to obtain a frequency-adjusted high-frequency output feature map, and performing an inter-frequency convolution calculation based on the frequency-adjusted high-frequency output feature map and the low-frequency output feature map to obtain a low-frequency external convolution result;

[0020] Performing a pooling upsampling operation on the low-frequency output feature map to obtain a frequency-adjusted low-frequency output feature map, and performing an inter-frequency convolution calculation based on the frequency-adjusted low-frequency output feature map and the high-frequency output feature map to obtain a high-frequency external convolution result;

[0021] Performing a point addition operation on the high-frequency internal convolution result and the high-frequency external convolution result to obtain a high-frequency output feature map corresponding to the current convolution layer;

[0022] Perform a point addition operation on the low-frequency internal convolution result and the low-frequency external convolution result to obtain a low-frequency output feature map corresponding to the current convolution layer.

[0023] Optionally, the adopting a pyramid feature network to perform feature fusion on the high-frequency output feature map and the low-frequency output feature map corresponding to each convolutional layer to obtain the first target feature map includes:

[0024] Obtaining a high-frequency output feature map and a low-frequency output feature map corresponding to each convolutional layer, aligning the resolutions of the high-frequency output feature map and the low-frequency output feature map to perform feature fusion, and obtaining a feature map to be spliced;

[0025] Starting from the top convolutional layer, a splicing operation is performed layer by layer along the channel direction to perform feature fusion on the to-be-splice feature map corresponding to each convolutional layer to obtain the first target feature map.

[0026] Optionally, the preset division factor is 0.5.

[0027] Optionally, inputting the first target feature map into a pre-built target detection frame generation module so that the target detection frame generation module outputs a second target feature map includes:

[0028] The target detection frame generation module slides on the first target feature map through a rotation region generation network to generate multiple rotation anchor frames for each position;

[0029] Selecting a rotation anchor frame whose confidence is higher than a preset threshold from all the rotation anchor frames as a candidate detection frame, and marking the candidate detection frame on the first target feature map to obtain the second target feature map;

[0030] The rotation anchor frame is represented by a quintuple, including the coordinates of the center point of the rotation anchor frame, the width and height of the rotation anchor frame, and the rotation angle of the rotation anchor frame.

[0031] Optionally, the step of inputting the second target feature map into a preset classifier for category classification and detection frame regression to obtain a target detection result includes:

[0032] The preset classifier classifies each of the candidate detection frames in the second target feature map, and generates a category classification probability according to the classification result, wherein the category classification probability is used to indicate the probability that the candidate detection frame contains the target;

[0033] The candidate detection frames are screened according to the category classification probability to obtain the detection frame to be regressed;

[0034] Perform bounding box regression on each of the detection boxes to be regressed, correct the anchor box parameters of each of the detection boxes to be regressed, and obtain the target detection result.

[0035] Optionally, construct a target detection box generation module, including:

[0036] Constructing an initial detection frame generation module to obtain multiple sample feature maps, wherein the initial detection frame generation module slides on each of the sample feature maps through a rotation region generation network to generate a predicted rotation anchor frame for each of the sample feature maps;

[0037] For each of the sample feature maps, calculating the intersection-over-union ratio between the predicted rotation anchor frame corresponding to the sample feature map and the real detection frame corresponding to the sample feature map, and dividing the sample feature map into a positive sample or a negative sample according to the intersection-over-union ratio result;

[0038] For each of the sample feature maps, classify and regress the predicted rotated anchor frame corresponding to the sample feature map to obtain a predicted category and a predicted detection frame, and use a predefined loss function to calculate the classification loss and the regression loss;

[0039] By using a back propagation algorithm, the parameters of the target detection frame generation module are updated according to the classification loss and the regression loss to obtain the target detection frame generation module.

[0040] According to a second aspect of the present application, a remote sensing image target detection device is provided, the device comprising:

[0041] An acquisition module is used to acquire a remote sensing image to be detected, and input the remote sensing image to be detected into a target feature extraction network, wherein the target feature extraction network is constructed based on a specified residual network, and the specified residual network is a ResNet101 network;

[0042] A first generating module is used to map the remote sensing image to be detected into high-frequency output feature maps and low-frequency output feature maps at different levels based on the target feature extraction layer of the target feature extraction network, and perform feature fusion using a feature pyramid network to obtain a first target feature map;

[0043] A second generating module is used to input the first target feature map into a pre-built target detection frame generating module, so that the target detection frame generating module outputs a second target feature map, wherein the second target feature map is marked with a plurality of target detection frames;

[0044] The detection module is used to input the second target feature map into a preset classifier for category classification and detection frame regression to obtain a target detection result.

[0045] Optionally, the first generation module is used for the first convolution layer in the target feature extraction layer, mapping the remote sensing image to be detected into a remote sensing feature map to be detected through the first convolution layer, and mapping the remote sensing feature map to be detected into a high-frequency input feature map and a low-frequency input feature map according to a preset division factor, and performing octave convolution calculation on the high-frequency input feature map and the low-frequency input feature map to obtain a high-frequency output feature map and a low-frequency output feature map corresponding to the first convolution layer, wherein the resolution of the low-frequency input feature map is one-fourth of the high-frequency input feature map, and the length and width of the low-frequency input feature map are respectively one-half of the high-frequency input feature map, wherein the first convolution layer is the top convolution layer; for the second convolution layer in the target feature extraction layer, through the second The convolution layer receives the high-frequency output feature map and the low-frequency data feature map output by the first convolution layer, performs octave convolution calculation on the high-frequency output feature map and the low-frequency output feature map corresponding to the first convolution layer, and obtains the high-frequency output feature map and the low-frequency output feature map corresponding to the second convolution layer, and the second convolution layer is the next layer of the first convolution layer; for the third convolution layer in the target feature extraction layer, the high-frequency output feature map and the low-frequency output feature map output by the second convolution layer are performed with the third convolution layer. Until the last convolution layer, the high-frequency output feature map and the low-frequency output feature map corresponding to each convolution layer are obtained; the pyramid feature network is used to perform feature fusion on the high-frequency output feature map and the low-frequency output feature map corresponding to each convolution layer to obtain the first target feature map.

[0046] Optionally, the first generation module is used to determine the high-frequency output feature map and the low-frequency output feature map output by the first convolution layer, perform frequency internal convolution calculation on the high-frequency output feature map and the low-frequency output feature map respectively, and obtain a high-frequency internal convolution result and a low-frequency internal convolution result; perform a pooling downsampling operation on the high-frequency output feature map to obtain a frequency-adjusted high-frequency output feature map, perform an inter-frequency convolution calculation based on the frequency-adjusted high-frequency output feature map and the low-frequency output feature map, and obtain a low-frequency external convolution result; perform a pooling upsampling operation on the low-frequency output feature map to obtain a frequency-adjusted low-frequency output feature map, perform an inter-frequency convolution calculation based on the frequency-adjusted low-frequency output feature map and the high-frequency output feature map, and obtain a high-frequency external convolution result; perform a point addition operation on the high-frequency internal convolution result and the high-frequency external convolution result to obtain a high-frequency output feature map corresponding to the current convolution layer; perform a point addition operation on the low-frequency internal convolution result and the low-frequency external convolution result to obtain a low-frequency output feature map corresponding to the current convolution layer.

[0047] Optionally, the first generation module is used to obtain a high-frequency output feature map and a low-frequency output feature map corresponding to each convolutional layer, align the resolutions of the high-frequency output feature map and the low-frequency output feature map to perform feature fusion, and obtain a feature map to be spliced; starting from the top convolutional layer, a splicing operation is performed layer by layer along the channel direction to perform feature fusion on the feature map to be spliced ​​corresponding to each convolutional layer to obtain the first target feature map.

[0048] Optionally, the preset division factor is 0.5.

[0049] Optionally, a second generation module is used for the target detection frame generation module to slide on the first target feature map through a rotation region generation network to generate multiple rotation anchor frames for each position; a rotation anchor frame with a confidence level higher than a preset threshold is selected from all the rotation anchor frames as a candidate detection frame, and the candidate detection frame is marked on the first target feature map to obtain the second target feature map; wherein the rotation anchor frame is represented by a quintuple, including the coordinates of the center point of the rotation anchor frame, the width and height of the rotation anchor frame, and the rotation angle of the rotation anchor frame.

[0050] Optionally, the detection module is used to classify each of the candidate detection boxes in the second target feature map through the preset classifier, and generate a category classification probability based on the classification result, wherein the category classification probability is used to indicate the probability that the candidate detection box contains the target; screen the candidate detection boxes according to the category classification probability to obtain the detection boxes to be regressed; perform bounding box regression on each of the detection boxes to be regressed, correct the anchor box parameters of each detection box to be regressed, and obtain the target detection result.

[0051] Optionally, the detection module is used to construct an initial detection frame generation module, obtain multiple sample feature maps, and the initial detection frame generation module slides on each of the sample feature maps through a rotation region generation network to generate a predicted rotation anchor frame for each of the sample feature maps; for each of the sample feature maps, the intersection and union ratio between the predicted rotation anchor frame corresponding to the sample feature map and the real detection frame corresponding to the sample feature map is calculated, and the sample feature map is divided into a positive sample or a negative sample according to the intersection and union ratio result; for each of the sample feature maps, the predicted rotation anchor frame corresponding to the sample feature map is classified and regressed to obtain a predicted category and a predicted detection frame, and a predefined loss function is used to calculate the classification loss and the regression loss; through the back propagation algorithm, the parameters of the target detection frame generation module are updated according to the classification loss and the regression loss to obtain the target detection frame generation module.

[0052] According to a third aspect of the present application, a computer device is provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0053] According to a fourth aspect of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.

[0054] By means of the above technical scheme, the present application provides a remote sensing image target detection method, device, computer equipment and storage medium. The present application first obtains the remote sensing image to be detected, and inputs the remote sensing image to be detected into a target feature extraction network, wherein the target feature extraction network is constructed based on the specified residual network ResNet101 network. Furthermore, the target feature extraction layer based on the target feature extraction network maps the remote sensing image to be detected into high-frequency output feature maps and low-frequency output feature maps of different levels, and uses a feature pyramid network for feature fusion to obtain a first target feature map. Next, the first target feature map is input into a pre-constructed target detection frame generation module so that the target detection frame generation module outputs a second target feature map, wherein the second target feature map is marked with multiple target detection frames. Finally, the second target feature map is input into a preset classifier for category classification and detection frame regression to obtain a target detection result. In the embodiment of the present application, a target feature extraction network is constructed based on the ResNet101 deep residual network, and the target feature extraction network is used to map the remote sensing image into high-frequency and low-frequency output feature maps of different levels, which can improve the target feature extraction network's ability to extract features and obtain contextual information, thereby improving the accuracy of distinguishing target objects in complex backgrounds. Furthermore, the feature pyramid network (FPN) is used for feature fusion, which can combine feature information of different depths for detection, achieve better processing of targets of different scales and features in remote sensing images, and improve the efficiency and accuracy of target detection in remote sensing images.

[0055] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0057] Figure 1 A schematic diagram of a remote sensing image target detection method provided in an embodiment of the present application is shown;

[0058] Figure 2 A schematic diagram showing the comparison between a natural image and a remote sensing image in a remote sensing image target detection method provided in an embodiment of the present application is shown;

[0059] Figure 3 A target feature extraction network diagram of a remote sensing image target detection method provided by an embodiment of the present application is shown;

[0060] Figure 4 A schematic diagram of an octave convolution calculation method for a remote sensing image target detection method provided by an embodiment of the present application is shown;

[0061] Figure 5 A schematic diagram of a pyramid network improvement of a remote sensing image target detection method provided by an embodiment of the present application is shown;

[0062] Figure 6 A schematic diagram of a rotating anchor of a remote sensing image target detection method provided in an embodiment of the present application is shown;

[0063] Figure 7 A schematic diagram of an IoU calculation method of a remote sensing image target detection method provided in an embodiment of the present application is shown;

[0064] Figure 8 A schematic diagram of an IoU calculation method of a remote sensing image target detection method provided in an embodiment of the present application is shown;

[0065] Fig. 9 A schematic diagram of an IoU calculation method of a remote sensing image target detection method provided in an embodiment of the present application is shown;

[0066] Fig.10 A schematic diagram of a network structure of a remote sensing image target detection method provided by an embodiment of the present application is shown;

[0067] Fig.11 A schematic structural diagram of a remote sensing image target detection device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0068] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0069] With the continuous development of computer vision, the Faster R-CNN (Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks) target detection algorithm has emerged. The Faster R-CNN target detection algorithm can be used to detect targets in remote sensing images. At present, the ZF network (Zeiler and Fergus Network) and the VGG16 network are often used as feature extraction networks of the Faster R-CNN target detection algorithm to extract the physical signs of the input image. The RPN network generates candidate regions from the input feature map, and then performs target detection on each candidate region, and corrects the border of the area where the target exists to obtain a more accurate target frame. The RoiHead module is consistent with the target detection network in the Fast R-CNN network, and realizes the target category classification and precise adjustment of the bounding box for each candidate frame generated by the RPN module. However, the applicant recognizes that the high resolution of remote sensing images can capture more detailed information, but the target to be detected will become smaller. At the same time, the background of the remote sensing image may contain a variety of objects and complex landform features, such as trees, water bodies, mountains, etc. These complex backgrounds may interfere with the detection and segmentation of the target, resulting in a low detection accuracy of the target detection of remote sensing images using the Faster R-CNN target detection algorithm. Therefore, the present application provides a remote sensing image target detection method, first obtaining a remote sensing image to be detected, and inputting the remote sensing image to be detected into a target feature extraction network, wherein the target feature extraction network is constructed based on a specified residual network ResNet101 network. Furthermore, the target feature extraction layer based on the target feature extraction network maps the remote sensing image to be detected into high-frequency output feature maps and low-frequency output feature maps of different levels, and uses a feature pyramid network for feature fusion to obtain a first target feature map. Next, the first target feature map is input into a pre-constructed target detection frame generation module so that the target detection frame generation module outputs a second target feature map, wherein the second target feature map is marked with multiple target detection frames. Finally, the second target feature map is input into a preset classifier for category classification and detection frame regression to obtain a target detection result. In an embodiment of the present application, a target feature extraction network is constructed based on the ResNet101 deep residual network, and the target feature extraction network is used to map the remote sensing image into high-frequency and low-frequency output feature maps at different levels. This can enhance the target feature extraction network's ability to extract features and acquire contextual information, thereby improving the accuracy of distinguishing target objects in complex backgrounds.Furthermore, the feature pyramid network (FPN) is used for feature fusion, which can combine feature information of different depths for detection, so as to better process targets of different scales and features in remote sensing images and improve the efficiency and accuracy of target detection in remote sensing images, such as. Figure 1 As shown, the method includes:

[0070] 101. Obtain a remote sensing image to be detected, and input the remote sensing image to be detected into a target feature extraction network, wherein the target feature extraction network is constructed based on a specified residual network, and the specified residual network is a ResNet101 network.

[0071] The target detection method provided in the embodiment of the present application is mainly aimed at detecting targets in remote sensing images. Most remote sensing images are formed by taking pictures of the ground by remote sensing platforms such as airplanes, drones and satellites. These images often show a bird's-eye view, and the range is also very large, which is significantly different from natural images. Remote sensing images have a huge amount of data and high resolution, dense targets and a large scale span, diverse target directions, scale scaling, complex backgrounds, and some targets are small. Figure 2 As shown in the figure, it can be seen that the imaging angles and imaging ranges of remote sensing images and natural images are different. With the development of remote sensing technology, the area covered by the remote sensing images obtained is getting larger and larger. Compared with natural images with a narrow imaging range, remote sensing images can often accommodate more targets to be detected. When there are more targets, the span between targets will also be larger. Therefore, remote sensing images also put forward higher requirements for multi-scale detection of models. Since the imaging angle of remote sensing images is a bird's-eye view, the captured targets often appear at different angles relative to natural images. Figure 2 As shown in Figure 1, the positions and arrangements of aircraft and cars in remote sensing images are different. The original Faster R-CNN (Region-based Convolutional Neural Network) network is difficult to accurately calibrate the target position.

[0072] The main application field of the original Faster R-CNN network is natural images. The Faster R-CNN network can be divided into three modules, namely, a feature extraction network, a region generation module, and a target classification and regression module. In order to detect targets on remote sensing images, the embodiment of the present application optimizes the original Faster R-CNN network. In the embodiment of the present application, the feature extraction network is first optimized, and the original feature extraction network is constructed using a ZF (Zeiler-Fergus Network) convolutional neural network. In the development of convolutional neural networks, AlexNet and VGGNet have been successfully applied in the field of image processing. People have been trying to deepen the number of layers of the network in order to achieve better training effects. Although deeper network layers can enhance the learning ability of the network to learn more features, they will also bring a huge amount of parameters, and will also be accompanied by problems such as gradient disappearance. The residual network aims to solve the problem of gradient disappearance and decreased expression ability of deep neural networks during training. The key idea of ​​ResNet is to retain the features before convolution calculation by introducing residual connections, so that the network can more easily learn the residual part, thereby improving the training effect and expression ability of the network. Before building the target feature extraction network, the relevant parameters of several common neural networks and the relevant parameters of different residual networks are compared, as shown in Table 1 below:

[0073] Table 1

[0074]

[0075] It can be seen from the table that although the ZF network has fewer parameters than VGG16, the Top-1 and Top-5 error rates of the ZF network are higher. Compared with the residual network, both the ResNet50 and ResNet101 networks significantly reduce the Top-1 and Top-5 error rates when the number of parameters is increased by no more than 3 times. At the same time, due to the existence of the residual module, the use of the residual network can achieve a deeper network layer without worrying about the gradient disappearance problem. Compared with the residual networks, although ResNet152 performs better in error rate, the improvement compared to the ResNet101 network is not significant, and the number of parameters is larger. Therefore, Figure 3 As shown, in the embodiment of the present application, the ResNet101 network is selected as the designated residual network to construct the target feature extraction network, wherein conv1 to conv4 in ResNet101 are selected as feature extraction layers.

[0076] 102. The target feature extraction layer based on the target feature extraction network maps the remote sensing image to be detected into high-frequency output feature maps and low-frequency output feature maps at different levels, and uses a feature pyramid network to perform feature fusion to obtain a first target feature map.

[0077] In the embodiment of the present application, it is considered that there is a large area of ​​background information in the remote sensing image, which will affect the feature extraction and thus reduce the efficiency of target detection. Therefore, in the present application, the octave convolution operation is used to replace the conventional convolution operation of conv1 to conv4 in Resnet101, so that the target feature extraction layer of the target feature extraction network can map the remote sensing image to be detected into high-frequency output feature maps and low-frequency output feature maps of different levels. It should be noted that there are high-frequency components and low-frequency components in the image. In daily images, high-frequency components usually refer to the image pixels with the strongest intensity (brightness / grayscale) changes in the image, such as important parts of the boundaries, contours, and other image details. Low-frequency components generally refer to pixels with smooth changes in image intensity (brightness / grayscale), such as the same area of ​​large color blocks. In image processing, more attention is usually paid to the high-frequency feature map in the image target, which contains information such as contours and edges. On the contrary, the information contained in the low-frequency feature map is less. In remote sensing images, large areas of background information are low-frequency components.

[0078] In this step, the octave convolution operation is used to replace the conventional convolution operation of conv1 to conv4 in Resnet101. Specifically, the feature map is mapped into high-frequency and low-frequency parts through a division factor α, wherein the division factor is set to 0.5, that is, the high-frequency part is the same as the low-frequency part. In the actual operation process, for the first convolution layer (top convolution layer) in the target feature extraction layer, the remote sensing image to be detected is mapped to the remote sensing feature map to be detected through the first convolution layer, and according to the preset division factor, the remote sensing feature map to be detected is mapped into a high-frequency input feature map and a low-frequency input feature map, wherein the resolution of the low-frequency input feature map is one-fourth of the high-frequency input feature map, and the length and width of the low-frequency input feature map are respectively one-half of the high-frequency input feature map. Furthermore, an octave convolution calculation is performed on the high-frequency input feature map and the low-frequency input feature map to obtain the high-frequency output feature map and the low-frequency output feature map corresponding to the first convolution layer. The process of octave convolution calculation is as follows Figure 4 As shown, where X H is the high-frequency input feature map, X L is the low-frequency input feature map. The resolution needs to be adjusted between different frequencies. Pooling downsampling operations are required from high frequency to low frequency, and pooling upsampling operations are required from low frequency to high frequency. Finally, the high-frequency output feature map and the low-frequency output feature map are obtained through point addition operations. Specifically, the high-frequency output feature map output by the first convolutional layer is determined. Figure X H and low frequency output characteristics Figure X L , for high frequency output characteristics Figure X H and low frequency output characteristics Figure X L Perform frequency internal convolution calculation f(X H ; W H→H ) and f(X L ; W L→L ), and get the high-frequency internal convolution result Y H→H And the low-frequency internal convolution result Y L→L . Perform pooling downsampling operation on high-frequency output feature maps (pool(X H ,2), obtain the high-frequency output feature map after frequency adjustment, and perform frequency convolution calculation f(pool(X H ,2);W H→L ), and get the low-frequency external convolution result Y H→L The low-frequency output feature map is pooled and upsampled to obtain the frequency-adjusted low-frequency output feature map. The frequency-interval convolution calculation upsample(f(X L ; W L→H ),2), get the high-frequency external convolution result Y L→H . For the high-frequency internal convolution result Y H→H And the high-frequency external convolution result Y L→H Perform point addition operation to obtain the high-frequency output feature map Y corresponding to the current convolutional layer H . For the low-frequency internal convolution result Y L→L And the low-frequency outer convolution result Y H→L Perform point addition operation to obtain the low-frequency output feature map Y corresponding to the current convolutional layer L. For the second convolution layer in the target feature extraction layer, the high-frequency output feature map and the low-frequency data feature map output by the first convolution layer are received through the second convolution layer, and the high-frequency output feature map and the low-frequency output feature map corresponding to the first convolution layer are calculated by octave convolution, so as to obtain the high-frequency output feature map and the low-frequency output feature map corresponding to the second convolution layer. The second convolution layer is the next layer of the first convolution layer. For the third convolution layer in the target feature extraction layer, the high-frequency output feature map and the low-frequency output feature map output by the second convolution layer are calculated by octave convolution through the third convolution layer until the last convolution layer, so as to obtain the high-frequency output feature map and the low-frequency output feature map corresponding to each convolution layer. Compared with ordinary convolution, OctConv (octave convolution) can effectively expand the receptive field by two times, further helping the feature extraction layer to capture more contextual information, thereby improving the detection capability of remote sensing image target detection. Furthermore, the feature pyramid network is used for feature fusion, which can combine feature information of different depths for detection, avoid the situation where small targets are easily missed, and improve the detection accuracy of small targets.

[0079] In this step, considering that the top-level features contain more semantic information but less location information of the target, the bottom-level features, on the contrary, contain rich and clear location information, feature fusion through the pyramid network can realize the detection of targets of different scales. Specifically, the pyramid feature network is used to fuse the high-frequency output feature map and the low-frequency output feature map corresponding to each convolutional layer to obtain the first target feature map. It should be noted that when fusing features at different layers, the conventional pyramid network uses the add method, which easily causes the aliasing effect of features. It is necessary to follow the add operation with a 3×3 convolution layer to further fuse the feature information and eliminate the aliasing effect. The embodiment of the present application uses the Concat operation instead of the add operation to splice the features of different layers in the channel direction, which can not only eliminate the aliasing effect, but also reduce the amount of calculation of using a convolution layer followed by an add operation. The improved fusion method is as follows Figure 5 As shown in the figure, the high-frequency output feature map and the low-frequency output feature map corresponding to each convolution layer are obtained, and the resolution of the high-frequency output feature map and the low-frequency output feature map are aligned to perform feature fusion to obtain the feature map to be spliced. Then, starting from the top convolution layer, the splicing operation is performed layer by layer along the channel direction to perform feature fusion on the feature map to be spliced ​​corresponding to each convolution layer to obtain the first target feature map.

[0080] 103. Input the first target feature map into a pre-built target detection frame generation module, so that the target detection frame generation module outputs a second target feature map, wherein the second target feature map is marked with multiple target detection frames.

[0081] In the embodiment of the present application, the region generation module of the original Faster R-CNN network uses RPN (Region Proposal Network) to generate proposal boxes. When different angles and dense arrangements occur, the horizontal proposal boxes generated by RPN are difficult to accurately calibrate the target position and are not robust to remote sensing image scenes. Therefore, the present application optimizes RPN and uses RRPN (Rotated Region Proposal Network) to construct a target detection box generation module. Compared with RPN, RRPN adds an angle parameter to control the rotation of the region proposal. At the same time, Figure 6 As shown in the figure, the rotation acnhor is redesigned to improve the accuracy of rotated target detection. Specifically, an initial detection box generation module is constructed to obtain multiple sample feature maps. The initial detection box generation module slides on each sample feature map through the rotation region generation network to generate a predicted rotation anchor box for each sample feature map. For each sample feature map, the intersection over union (IoU) between the predicted rotation anchor box corresponding to the sample feature map and the true detection box corresponding to the sample feature map is calculated, and the sample feature map is divided into positive samples or negative samples according to the intersection over union (IoU) result. Specifically, RRPN simplifies the calculation of tilted IoU into the area of ​​multiple triangles by designing triangulation, for example, Figures 7 to 9 As shown, Figure 7 Where IoU is the sum of the areas of ΔNHM and ΔNMB. Figure 8 Where IoU is the sum of the areas of ΔALK, ΔAKC, ΔACJ, and ΔAJI. Fig. 9 In the above formula, IoU is the sum of the areas of ΔIPO, ΔION, ΔINM, ΔIML, ΔILK, and ΔIKJ. During training, samples with IoU greater than 0.7 are set as positive samples, and samples with an angle between the rotation box corresponding to the anchor point and the true box less than π / 12 are recorded as positive samples. For each sample feature map, the predicted rotation anchor box corresponding to the sample feature map is classified and regressed to obtain the predicted category and predicted detection box, and the classification loss and regression loss are calculated using a predefined loss function. Through the back-propagation algorithm, the parameters of the target detection box generation module are updated according to the classification loss and regression loss to obtain the target detection box generation module.

[0082] Finally, the target detection box generation module is used to slide on the first target feature map through the rotation region generation network to generate multiple rotation anchor boxes for each position. The rotation anchor boxes with confidence higher than the preset threshold are selected as candidate detection boxes from all the rotation anchor boxes, and the candidate detection boxes are marked on the first target feature map to obtain the second target feature map, where the representation of the rotation box is represented by a five-tuple (x, y, w, h, θ), where (x, y) is the coordinate of the geometric center point of the angled bounding box, h is the height of the angled bounding box, w is the length of the angled bounding box, and θ is the angle between the positive direction of the x-axis and the long side direction of the rotation frame.

[0083] 104. Input the second target feature map into a preset classifier for category classification and detection frame regression to obtain a target detection result.

[0084] In an embodiment of the present application, each candidate detection frame in the second target feature map is classified by a preset classifier, and a category classification probability is generated according to the classification result, wherein the category classification probability is used to indicate the probability that the candidate detection frame contains the target. The candidate detection frames are screened according to the category classification probability to obtain the detection frame to be regressed. Finally, a bounding box regression is performed on each detection frame to be regressed, and the anchor box parameters of each detection frame to be regressed are corrected to obtain the target detection result.

[0085] In summary, the remote sensing image target detection method provided in this embodiment improves the original Faster R-CNN model in a targeted manner to solve the problem that small targets, random target arrangement directions, and dense situations in remote sensing images make it difficult to accurately mark and detect targets. The improved Faster R-CNN network is as follows: Fig.10 As shown in the figure, the original feature extraction network is optimized by using octave convolution and ResNet101 network, and the remote sensing image is mapped into high-frequency and low-frequency output feature maps at different levels, which can improve the target feature extraction network's ability to extract features and obtain contextual information, thereby improving the accuracy of distinguishing target objects in complex backgrounds. The optimized feature pyramid network can combine feature information of different depths for detection, achieve better processing of targets of different scales and features in remote sensing images, improve the efficiency and accuracy of target detection in remote sensing images, and introduce a rotation target detection algorithm and improve the target detection frame generation stage, so that the network model can detect targets with random and dense arrangement directions, thereby improving the detection performance. Finally, the feature map with the detection frame is input into the preset classifier for category classification and detection frame regression, and the target detection result of the remote sensing image is obtained.

[0086] Further, as Figure 1 In a specific implementation of the method, the present application provides a remote sensing image target detection device, such as Fig.11As shown, the device includes: an acquisition module 1101 , a first generation module 1102 , a second generation module 1103 , and a detection module 1104 .

[0087] The acquisition module 1101 is used to acquire a remote sensing image to be detected, and input the remote sensing image to be detected into a target feature extraction network, wherein the target feature extraction network is constructed based on a specified residual network, and the specified residual network is a ResNet101 network;

[0088] The first generating module 1102 is used to map the remote sensing image to be detected into high-frequency output feature maps and low-frequency output feature maps at different levels based on the target feature extraction layer of the target feature extraction network, and use a feature pyramid network to perform feature fusion to obtain a first target feature map;

[0089] The second generating module 1103 is used to input the first target feature map into a pre-built target detection frame generating module, so that the target detection frame generating module outputs a second target feature map, wherein the second target feature map is marked with a plurality of target detection frames;

[0090] The detection module 1104 is used to input the second target feature map into a preset classifier for category classification and detection box regression to obtain a target detection result.

[0091] In a specific application scenario, the first generation module 1102 is used for the first convolution layer in the target feature extraction layer, to map the remote sensing image to be detected into a remote sensing feature map to be detected through the first convolution layer, and to map the remote sensing feature map to be detected into a high-frequency input feature map and a low-frequency input feature map according to a preset division factor, and to perform octave convolution calculation on the high-frequency input feature map and the low-frequency input feature map to obtain a high-frequency output feature map and a low-frequency output feature map corresponding to the first convolution layer, wherein the resolution of the low-frequency input feature map is one-fourth of the high-frequency input feature map, and the length and width of the low-frequency input feature map are respectively one-half of the high-frequency input feature map, wherein the first convolution layer is the top convolution layer; for the second convolution layer in the target feature extraction layer, through The high-frequency output feature map and the low-frequency data feature map output by the first convolution layer are received through the second convolution layer, and the high-frequency output feature map and the low-frequency output feature map corresponding to the first convolution layer are performed with octave convolution calculation to obtain the high-frequency output feature map and the low-frequency output feature map corresponding to the second convolution layer, and the second convolution layer is the next layer of the first convolution layer; for the third convolution layer in the target feature extraction layer, the high-frequency output feature map and the low-frequency output feature map output by the second convolution layer are performed with octave convolution calculation through the third convolution layer until the last convolution layer, so as to obtain the high-frequency output feature map and the low-frequency output feature map corresponding to each convolution layer; the high-frequency output feature map and the low-frequency output feature map corresponding to each convolution layer are feature fused by using a pyramid feature network to obtain the first target feature map.

[0092] In a specific application scenario, the first generation module 1102 is used to determine the high-frequency output feature map and the low-frequency output feature map output by the first convolution layer, perform frequency internal convolution calculation on the high-frequency output feature map and the low-frequency output feature map respectively, and obtain a high-frequency internal convolution result and a low-frequency internal convolution result; perform a pooling downsampling operation on the high-frequency output feature map to obtain a frequency-adjusted high-frequency output feature map, perform an inter-frequency convolution calculation based on the frequency-adjusted high-frequency output feature map and the low-frequency output feature map, and obtain a low-frequency external convolution result; perform a pooling upsampling operation on the low-frequency output feature map to obtain a frequency-adjusted low-frequency output feature map, perform an inter-frequency convolution calculation based on the frequency-adjusted low-frequency output feature map and the high-frequency output feature map, and obtain a high-frequency external convolution result; perform a point addition operation on the high-frequency internal convolution result and the high-frequency external convolution result to obtain a high-frequency output feature map corresponding to the current convolution layer; perform a point addition operation on the low-frequency internal convolution result and the low-frequency external convolution result to obtain a low-frequency output feature map corresponding to the current convolution layer.

[0093] In a specific application scenario, the first generation module 1102 is used to obtain the high-frequency output feature map and the low-frequency output feature map corresponding to each convolution layer, align the resolution of the high-frequency output feature map and the low-frequency output feature map to perform feature fusion, and obtain the feature map to be spliced; starting from the top convolution layer, the splicing operation is used layer by layer along the channel direction to perform feature fusion on the feature map to be spliced ​​corresponding to each convolution layer to obtain the first target feature map.

[0094] In a specific application scenario, the preset division factor is 0.5.

[0095] In a specific application scenario, the second generation module 1103 is used for the target detection frame generation module to slide on the first target feature map through the rotating region generation network to generate multiple rotated anchor frames for each position; select the rotated anchor frame with a confidence level higher than a preset threshold from all the rotated anchor frames as the candidate detection frame, and mark the candidate detection frame on the first target feature map to obtain the second target feature map; wherein the rotated anchor frame is represented by a quintuple, including the coordinates of the center point of the rotated anchor frame, the width and height of the rotated anchor frame, and the rotation angle of the rotated anchor frame.

[0096] In a specific application scenario, the detection module 1104 is used to classify each of the candidate detection boxes in the second target feature map through the preset classifier, and generate a category classification probability based on the classification result, wherein the category classification probability is used to indicate the probability that the candidate detection box contains the target; the candidate detection boxes are screened according to the category classification probability to obtain the detection boxes to be regressed; a bounding box regression is performed on each of the detection boxes to be regressed, and the anchor box parameters of each of the detection boxes to be regressed are corrected to obtain the target detection result.

[0097] In a specific application scenario, the detection module 1104 is used to construct an initial detection frame generation module to obtain multiple sample feature maps. The initial detection frame generation module slides on each of the sample feature maps through a rotation region generation network to generate a predicted rotation anchor frame for each of the sample feature maps; for each of the sample feature maps, the intersection-and-union ratio between the predicted rotation anchor frame corresponding to the sample feature map and the real detection frame corresponding to the sample feature map is calculated, and the sample feature map is divided into a positive sample or a negative sample according to the intersection-and-union ratio result; for each of the sample feature maps, the predicted rotation anchor frame corresponding to the sample feature map is classified and regressed to obtain a predicted category and a predicted detection frame, and a predefined loss function is used to calculate the classification loss and the regression loss; through the back propagation algorithm, the parameters of the target detection frame generation module are updated according to the classification loss and the regression loss to obtain the target detection frame generation module.

[0098] The device provided in the embodiment of the present application first obtains the remote sensing image to be detected, and inputs the remote sensing image to be detected into the target feature extraction network, wherein the target feature extraction network is constructed based on the specified residual network ResNet101 network. Further, the target feature extraction layer based on the target feature extraction network maps the remote sensing image to be detected into high-frequency output feature maps and low-frequency output feature maps of different levels, and uses the feature pyramid network for feature fusion to obtain a first target feature map. Next, the first target feature map is input into a pre-constructed target detection frame generation module so that the target detection frame generation module outputs a second target feature map, wherein a plurality of target detection frames are marked in the second target feature map. Finally, the second target feature map is input into a preset classifier for category classification and detection frame regression to obtain a target detection result. In the embodiment of the present application, a target feature extraction network is constructed based on the ResNet101 deep residual network, and the target feature extraction network is used to map the remote sensing image into high-frequency and low-frequency output feature maps of different levels, which can improve the target feature extraction network's ability to extract features and its ability to obtain context information, thereby improving the accuracy of distinguishing target objects in complex backgrounds. Furthermore, the feature pyramid network (FPN) is used for feature fusion, which can combine feature information of different depths for detection, achieve better processing of targets of different scales and features in remote sensing images, and improve the efficiency and accuracy of target detection in remote sensing images.

[0099] It should be noted that for other corresponding descriptions of the functional units involved in the remote sensing image target detection device provided in the embodiment of the present application, reference can be made to Figures 1 to 10 The corresponding description in will not be repeated here.

[0100] Based on the above Figures 1 to 10 The method shown, accordingly, this embodiment also provides a storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the remote sensing image target detection method are implemented.

[0101] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product. The software product to be identified can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0102] In order to achieve the above purpose, in an exemplary embodiment, a device is also provided, which includes a communication bus, a processor, a memory and a communication interface, and may also include an input and output interface and a display device, wherein each functional unit can communicate with each other through the bus. The memory stores a computer program, and the processor is used to execute the program stored in the memory and execute the remote sensing image target detection method in the above embodiment.

[0103] Optionally, the physical device may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0104] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware of the physical device and the software resources to be identified, and supports the operation of the information processing program and other software and / or programs to be identified. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the information processing physical device.

[0105] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by hardware. By applying the technical solution of the present application, the remote sensing image to be detected is first obtained, and the remote sensing image to be detected is input into the target feature extraction network, wherein the target feature extraction network is constructed based on the specified residual network ResNet101 network. Furthermore, the target feature extraction layer based on the target feature extraction network maps the remote sensing image to be detected into high-frequency output feature maps and low-frequency output feature maps of different levels, and uses a feature pyramid network for feature fusion to obtain a first target feature map. Next, the first target feature map is input into a pre-constructed target detection frame generation module so that the target detection frame generation module outputs a second target feature map, wherein a plurality of target detection frames are marked in the second target feature map. Finally, the second target feature map is input into a preset classifier for category classification and detection frame regression to obtain a target detection result. Compared with the prior art, in the embodiment of the present application, a target feature extraction network is constructed based on the ResNet101 deep residual network, and the target feature extraction network is used to map the remote sensing image into high-frequency and low-frequency output feature maps of different levels, which can improve the target feature extraction network's ability to extract features and obtain contextual information, thereby improving the accuracy of distinguishing target objects in complex backgrounds. Furthermore, the feature pyramid network (FPN) is used for feature fusion, which can combine feature information of different depths for detection, achieve better processing of targets of different scales and features in remote sensing images, and improve the efficiency and accuracy of target detection in remote sensing images.

[0106] Those skilled in the art will appreciate that the accompanying drawings are merely schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily required for implementing the present application.

[0107] Those skilled in the art will appreciate that the modules in the device in the implementation scenario can be distributed in the device in the implementation scenario according to the implementation scenario description, or can be changed accordingly and located in one or more devices different from the implementation scenario. The modules in the above implementation scenario can be combined into one module, or can be further split into multiple sub-modules.

[0108] The above application serial numbers are for description only and do not represent the advantages or disadvantages of the implementation scenarios.

[0109] The above disclosure only discloses several specific implementation scenarios of the present application. However, the present application is not limited thereto, and any changes that can be conceived by technicians in this field should fall within the scope of protection of the present application.

Claims

1. A remote sensing image target detection method, characterized in that: include: Acquire a remote sensing image to be detected, and input the remote sensing image to be detected into a target feature extraction network, wherein the target feature extraction network is constructed based on a specified residual network, and the specified residual network is a ResNet101 network; Based on the target feature extraction layer of the target feature extraction network, the remote sensing image to be detected is mapped into high-frequency output feature maps and low-frequency output feature maps at different levels, and a feature pyramid network is used to perform feature fusion to obtain a first target feature map; Inputting the first target feature map into a pre-built target detection frame generation module, so that the target detection frame generation module outputs a second target feature map, wherein the second target feature map is marked with a plurality of target detection frames; The second target feature map is input into a preset classifier for category classification and detection frame regression to obtain a target detection result.

2. The method according to claim 1, characterized in that The target feature extraction layer based on the target feature extraction network maps the remote sensing image to be detected into high-frequency output feature maps and low-frequency output feature maps at different levels, and uses a feature pyramid network to perform feature fusion to obtain a first target feature map, including: For the first convolution layer in the target feature extraction layer, the remote sensing image to be detected is mapped to a remote sensing feature map to be detected through the first convolution layer, and according to a preset division factor, the remote sensing feature map to be detected is mapped to a high-frequency input feature map and a low-frequency input feature map, and an octave convolution calculation is performed on the high-frequency input feature map and the low-frequency input feature map to obtain a high-frequency output feature map and a low-frequency output feature map corresponding to the first convolution layer, wherein the resolution of the low-frequency input feature map is one-fourth of the high-frequency input feature map, and the length and width of the low-frequency input feature map are respectively one-half of the high-frequency input feature map, wherein the first convolution layer is a top convolution layer; For the second convolution layer in the target feature extraction layer, the high-frequency output feature map and the low-frequency data feature map output by the first convolution layer are received through the second convolution layer, and the high-frequency output feature map and the low-frequency output feature map corresponding to the first convolution layer are performed with octave convolution calculation to obtain the high-frequency output feature map and the low-frequency output feature map corresponding to the second convolution layer, and the second convolution layer is the next layer of the first convolution layer; For the third convolution layer in the target feature extraction layer, an octave convolution calculation is performed on the high-frequency output feature map and the low-frequency output feature map output by the second convolution layer through the third convolution layer, until the last convolution layer, to obtain the high-frequency output feature map and the low-frequency output feature map corresponding to each convolution layer; A pyramid feature network is used to perform feature fusion on the high-frequency output feature map and the low-frequency output feature map corresponding to each convolutional layer to obtain the first target feature map.

3. The method according to claim 2, characterized in that The method of receiving the high-frequency output feature map and the low-frequency data feature map output by the first convolution layer through the second convolution layer, performing octave convolution calculation on the high-frequency output feature map and the low-frequency output feature map corresponding to the first convolution layer, and obtaining the high-frequency output feature map and the low-frequency output feature map corresponding to the second convolution layer includes: Determine a high-frequency output feature map and a low-frequency output feature map output by the first convolutional layer, perform frequency internal convolution calculations on the high-frequency output feature map and the low-frequency output feature map, respectively, to obtain a high-frequency internal convolution result and a low-frequency internal convolution result; Performing a pooling downsampling operation on the high-frequency output feature map to obtain a frequency-adjusted high-frequency output feature map, and performing an inter-frequency convolution calculation based on the frequency-adjusted high-frequency output feature map and the low-frequency output feature map to obtain a low-frequency external convolution result; Performing a pooling upsampling operation on the low-frequency output feature map to obtain a frequency-adjusted low-frequency output feature map, and performing an inter-frequency convolution calculation based on the frequency-adjusted low-frequency output feature map and the high-frequency output feature map to obtain a high-frequency external convolution result; Performing a point addition operation on the high-frequency internal convolution result and the high-frequency external convolution result to obtain a high-frequency output feature map corresponding to the current convolution layer; Perform a point addition operation on the low-frequency internal convolution result and the low-frequency external convolution result to obtain a low-frequency output feature map corresponding to the current convolution layer.

4. The method according to claim 2, characterized in that: The step of using a pyramid feature network to perform feature fusion on the high-frequency output feature map and the low-frequency output feature map corresponding to each convolutional layer to obtain the first target feature map includes: Obtaining a high-frequency output feature map and a low-frequency output feature map corresponding to each convolutional layer, aligning the resolutions of the high-frequency output feature map and the low-frequency output feature map to perform feature fusion, and obtaining a feature map to be spliced; Starting from the top convolutional layer, a splicing operation is performed layer by layer along the channel direction to perform feature fusion on the to-be-splice feature map corresponding to each convolutional layer to obtain the first target feature map.

5. The method according to claim 1, characterized in that The step of inputting the first target feature map into a pre-built target detection frame generation module so that the target detection frame generation module outputs a second target feature map comprises: The target detection frame generation module slides on the first target feature map through a rotation region generation network to generate multiple rotation anchor frames for each position; Selecting a rotation anchor frame whose confidence is higher than a preset threshold from all the rotation anchor frames as a candidate detection frame, and marking the candidate detection frame on the first target feature map to obtain the second target feature map; The rotation anchor frame is represented by a quintuple, including the coordinates of the center point of the rotation anchor frame, the width and height of the rotation anchor frame, and the rotation angle of the rotation anchor frame.

6. The method according to claim 5, characterized in that The step of inputting the second target feature map into a preset classifier for category classification and detection frame regression to obtain a target detection result includes: Classify each of the candidate detection frames in the second target feature map by the preset classifier, and generate a category classification probability according to the classification result, wherein the category classification probability is used to indicate the probability that the candidate detection frame contains the target; The candidate detection frames are screened according to the category classification probability to obtain the detection frame to be regressed; Perform bounding box regression on each of the detection boxes to be regressed, correct the anchor box parameters of each of the detection boxes to be regressed, and obtain the target detection result.

7. The method according to claim 5, characterized in that Construct the target detection box generation module, including: Constructing an initial detection frame generation module to obtain multiple sample feature maps, wherein the initial detection frame generation module slides on each of the sample feature maps through a rotation region generation network to generate a predicted rotation anchor frame for each of the sample feature maps; For each of the sample feature maps, calculating the intersection-over-union ratio between the predicted rotated anchor frame corresponding to the sample feature map and the real detection frame corresponding to the sample feature map, and dividing the sample feature map into a positive sample or a negative sample according to the intersection-over-union ratio result; For each of the sample feature maps, classify and regress the predicted rotated anchor frame corresponding to the sample feature map to obtain a predicted category and a predicted detection frame, and use a predefined loss function to calculate the classification loss and regression loss; By using a back propagation algorithm, the parameters of the target detection frame generation module are updated according to the classification loss and the regression loss to obtain the target detection frame generation module.

8. A remote sensing image target detection device, characterized in that: include: An acquisition module is used to acquire a remote sensing image to be detected, and input the remote sensing image to be detected into a target feature extraction network, wherein the target feature extraction network is constructed based on a specified residual network, and the specified residual network is a ResNet101 network; A first generating module is used to map the remote sensing image to be detected into high-frequency output feature maps and low-frequency output feature maps at different levels based on the target feature extraction layer of the target feature extraction network, and perform feature fusion using a feature pyramid network to obtain a first target feature map; A second generating module is used to input the first target feature map into a pre-built target detection frame generating module, so that the target detection frame generating module outputs a second target feature map, wherein the second target feature map is marked with a plurality of target detection frames; The detection module is used to input the second target feature map into a preset classifier for category classification and detection frame regression to obtain a target detection result.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images