Remote sensing target detection method based on convolutional neural network

Abstract

The invention discloses a remote sensing target detection method based on a convolutional neural network, and the method comprises the steps: 1, constructing a full-scale enhanced alignment detection network model OSEADet comprising Backbone, Neck and Head, the Neck is an enhanced feature pyramid network EFPN, adding a P2 detection layer on the PAFPN network, enabling a feature map P2 to pass through SPDConv, outputting a feature map rich in small target information, fusing the feature map with a feature map P3, and obtaining a feature map P3; carrying out feature integration by using a CSP thought and a CSPF module; the Head comprises three dynamic task alignment sharing detection heads DSTA, the DSTA comprises a bounding box regression branch and a classification branch which are respectively used for predicting a target position and a target category, the tasks of the three DSTA are combined through group convolution, task interaction features are learned from a plurality of convolution layers, joint features are obtained, interaction among the tasks is increased, and the tasks are dynamically adjusted and aligned through a TaskDecomposition layer; 2, carrying out training and verification on the OSEADet; and 3, the trained OSEADet is used for predicting the position and category of the remote sensing target, so that the detection precision is high, and the remote sensing target can be detected in real time.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and remote sensing image processing, and specifically is a remote sensing target detection method based on convolutional neural network. Background Art

[0002] As an important means of obtaining information about the earth's surface, remote sensing images have extensive application value in fields such as environmental monitoring and urban planning. With the rapid growth of remote sensing image data, accurate detection of target objects from a large number of remote sensing images has received increasing attention. In recent years, deep learning technology has provided a new direction for remote sensing image data analysis with its powerful feature extraction and learning capabilities. It can efficiently learn the complex regular features in remote sensing images, significantly improving the accuracy of remote sensing image detection and recognition.

[0003] The object detection of remote sensing images mainly relies on the Deep Convolution Neural Network (DCNN) framework. Its core design idea is the combination of feature extraction and bounding box regression, that is, the object detector can adaptively extract image features and locate objects through an end-to-end learning framework. It is mainly divided into two-stage detection methods (Fast RCNN, Faster RCNN, Mask RCNN, R-FCN) and single-stage detection methods (SSD, RetinaNet, YOLO). They have been well applied in common task scenarios. In particular, the YOLO series adopts a single-stage design idea and has efficient feature extraction capabilities. It has become the fastest and most accurate model and is widely used in various computer vision tasks.

[0004] Among the two-stage target detection algorithms based on region recommendation, the representative methods are region-based convolutional neural network (R-CNN), Fast R-CNN and Faster The main idea of ​​R-CNN is to use the selective search method to generate proposed regions, and then perform regression and classification in the proposed regions. Some variants proposed based on the RCNN framework, such as ReDet, can capture the features of the target in different directions by embedding a rotation equivariant module in the network structure, and integrate these features into the detection process, thereby improving the recognition ability of rotated targets. Among them: R3Det adopts a coarse-to-fine progressive regression method, designs a feature refinement module FRM, uses feature interpolation to obtain the position information corresponding to the refined anchor point, and reconstructs the entire feature map in a pixel-level manner to achieve feature alignment; SCRDet designs a feature fusion structure SF-Net from the perspective of feature fusion and anchor sampling, highlights the features of the remote sensing foreground and weakens the background features, and adds an IoU constant factor to the loss function to achieve coordinate prediction of any corner of the target; SCRDet++[7] performs instance-level denoising on the feature map and specifies a new InLD component to approximately decouple the features of different object categories into their own channels. In the spatial domain, the features of the object and the background are enhanced and weakened respectively to enhance the detection of small and cluttered objects.

[0005] The single-stage target detection algorithm simplifies the detection problem into a regression problem. Only a convolutional neural network is needed to directly obtain the class probability and position coordinates of the target. The most representative algorithms include YOLOv1-10, SSD] and RetinaNe. The YOLO (You Only Look Once) series of models have good real-time and high efficiency, and have become the preferred method in many practical applications. Unlike traditional target detection methods such as the R-CNN series, YOLO transforms the target detection problem into a regression problem. Its core idea is to perform a global analysis of the input image through a single neural network and directly output the bounding box and category probability in the image. The innovation of YOLO lies in its end-to-end training and reasoning method, which greatly improves the detection speed. Therefore, since it was first proposed in 2015, it has been gradually evolving. Among them: YOLOv1 (2015) transforms the target detection problem into a regression problem, directly outputs the bounding box and category probability through a single neural network, realizes end-to-end target detection, and has high real-time performance, but has shortcomings in small target detection and positioning accuracy; YOLOv2 (2016) introduces Batch Normalization, Anchor Boxes and multi-scale training improve model performance; YOLOv3 (2018) adopts multi-scale feature maps and Darknet-53 backbone network, which significantly improves detection accuracy and small target detection capabilities; YOLOv4 (2020) integrates CSP Darknet53, Mish activation function and CIoU loss to further optimize detection effect and speed; YOLOv5 (2020), released by Ultralytics, optimizes the training process and enhances scalability; YOLOv7 and YOLOv8 (2022, 2023) introduce more complex network design and cutting-edge technologies such as Transformer on the basis of the former, further improving the expressiveness and robustness of the model; YOLOv9 combines programmable gradient information (PGI) with general ELAN (GELAN) architecture to achieve higher parameter utilization than deep convolution design based on state-of-the-art technology, while showing great advantages of lightness, speed and accuracy.

[0006] Among the existing single-stage target detection algorithms, YOLOv8 performs best in remote sensing target detection. Its structure includes Backbone, Neck and Head, among which: the customized CSPDarknet53 is used as the backbone network. This structure improves the information flow between different network stages through cross-stage partial connections, thereby improving accuracy; Neck uses the C2f module to merge feature maps from different stages of the backbone network to capture information of different scales. This module combines high-level semantic features with low-level spatial information, especially for the detection accuracy of small objects; Head is responsible for prediction, multiple detection modules, which predict the bounding box, target score and category probability of each feature map grid cell, and then summarize them into the final detection result.

[0007] Although YOLOv8 can detect remote sensing images well, there are the following problems: on the one hand, due to changes in factors such as the shooting angle and sensor posture of remote sensing images, the targets in remote sensing images rotate in multiple directions. In addition, remote sensing images have a wide viewing angle and often contain a large amount of background information, such as terrain, vegetation, and buildings. The extreme richness of background information makes the foreground information account for a very small proportion of the entire image. For example, targets such as vehicles and ships only occupy a small part of the pixels in remote sensing images. The number of pixels is very limited, and the feature representation is often weak. As a result, when the YOLOv8 model is building a pyramid, the features of small targets are easily lost in the deep convolution process, resulting in a decrease in detection accuracy. In addition, it is more biased in the training process. It tends to learn background features while ignoring the effective learning of foreground information; at the same time, there are overlapping occlusions of targets in the image and small targets are often similar to the background in color, texture, etc., which makes it difficult for the model to extract features, and detailed information is difficult to identify and extract, which is easy to cause confusion, resulting in YOLOv8's low detection accuracy for remote sensing targets under complex backgrounds; on the other hand, the YOLOv8 detection head uses a decoupling head, and the three branches of the decoupling head detect targets of different scales respectively. This design can reduce errors and mutual influence and improve the performance of the model, but the two task branches work independently and lack information interaction with each other. They cannot fully utilize the advanced features extracted by the backbone network, resulting in low model utilization and increased model complexity and computational complexity.

[0008] At present, in order to obtain satisfactory detection accuracy, more complex network structures, multi-layer splicing and fusion, and attention addition are usually used in BackBone and Neck to improve feature extraction and fusion capabilities. However, this increases the computational complexity of the algorithm and results in poor real-time performance. On the contrary, for real-time performance, a high frame rate (FPS) is usually pursued, resulting in insufficient attention to details or smaller targets during the detection process, which affects detection accuracy. It can be seen that the existing YOLO model has an imbalance between speed and accuracy when facing remote sensing images with severe background interference, multi-scales, and multi-directions. Summary of the invention

[0009] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a remote sensing target detection method based on convolutional neural network, which not only has high detection accuracy and can accurately perceive small remote sensing targets in complex scenes, but also has low computational complexity, realizes real-time detection of remote sensing targets, and improves detection efficiency.

[0010] In order to achieve the above object, the present invention adopts the following technical solutions:

[0011] A remote sensing target detection method based on convolutional neural network includes the following steps:

[0012] Step 1: Construct a full-scale enhanced alignment detection network model OSEADet including Backbone, Neck and Head, where:

[0013] The Backbone uses a five-layer feature pyramid structure built based on a CNN network to extract feature maps P1, P2, P3, P4 and P5 of different scales of remote sensing images;

[0014] The Neck is an enhanced feature pyramid network EFPN. EFPN adds a P2 detection layer on the basis of the PAFPN network structure. The feature map P2 is processed by SPDConv to output a feature map rich in small target information. The feature map rich in small target information is fused with the feature map P3, and then the cross-layer connection CSP idea and CSPF module are used for feature integration.

[0015] The CSPF module includes a global branch, a large-scale branch and a local branch. The CSPF module performs multi-level processing on the feature map through the global branch, the large-scale branch and the local branch, so that it captures information of different scales, and then fuses the multi-level features. Then, through the fast Fourier transform, the features are converted to the frequency domain for processing, thereby retaining the global information of the features;

[0016] The head includes three dynamic task alignment shared detection heads DSTA, which combine the advantages of the decoupling head and the coupling head, including a bounding box regression branch and a classification branch, which are used to predict the target position and category respectively, wherein: the tasks of the three DSTAs are merged through group convolution, task interaction features are learned from multiple convolutional layers, joint features are obtained, the interaction between tasks is increased, and tasks are dynamically adjusted through the TaskDecomposition layer to align the tasks;

[0017] Step 2: Use existing public datasets to train and validate the network model OSEADet. During the training process, the difference between the predicted value and the true label value is reduced by minimizing the classification loss and regression loss.

[0018] Step 3: Use the trained network model OSEADet to predict the location and category of the remote sensing target.

[0019] Furthermore, the process of integrating features using the cross-layer connection CSP concept and the CSPF module in step 1 is as follows:

[0020] Step 1.1, the CSPF module extracts features from the input feature map X through a convolution layer, and divides the feature map into X1 and X2 according to the ratio of 1 / 4 and 3 / 4. The 1 / 4 feature map X1 is channel compressed through a 1×1 convolution, and then used as the input of the local branch, large-scale branch and global branch respectively, for three-level feature extraction, where:

[0021] The local branch extracts the local features of the feature map through 1x1 depth convolution, which is expressed as:

[0022] y=C 1×1 (X1)

[0023] O local =DC 1×1 (y)

[0024] In the formula, O local is the output of the local branch, C 1×1 Represents 1×1 convolution, DC 1×1 Represents a 1×1 depth convolution;

[0025] The large-scale branch extracts the large-scale features of the feature map through three deep convolutions, which can be expressed as:

[0026] O large =DC 1×61 (y)+DC 61×61 (y)+DC 61×1 (y)

[0027] Where y is the input of the large-scale branch, DC 1×61, D.C. 61×61 and DC 61×1 They are the depth convolutions of three large kernels, O 1arge is the output of the large-scale branch;

[0028] The global branch first converts the feature map to the frequency domain through the fast Fourier transform (FFT), performs feature transformation, and then converts it back to the spatial domain through the inverse fast Fourier transform (IFFT), and restores the feature map through 1×1 convolution, which is expressed as:

[0029] Z = IFFT(FFT(y) × C 1×1 (y))

[0030] O gloobal =Z×C 1×1 (Z)

[0031] Where, FFT() stands for Fast Fourier Transform, C 1×1 represents 1×1 convolution, IFFT() represents inverse fast Fourier transform, O global Represents the output of the global branch;

[0032] Step 1.2: The outputs of the local branch, large-scale branch, and global branch are fused by element-wise addition, and the 3 / 4 feature map X2 and the fused features are element-wise added, and then the final feature map F is output through a convolutional layer. out , expressed as:

[0033] F out =Conv(X2+C 1×1 (O local +O large +O global )).

[0034] Furthermore, all convolutional layers in the group convolution in step 1 share parameters.

[0035] Furthermore, the classification loss in step 2 adopts cross entropy loss, which is expressed as:

[0036]

[0037] In the formula, M is the number of categories, y ic is a sign function, taking a value of 0 or 1. If the true category of sample i is equal to c, then y ic Take 1, otherwise y ic Take 0, p ic is the probability that the observed sample i belongs to c, and N is the target number.

[0038] Furthermore, the regression loss of step 2 adopts confidence loss and bounding box loss, where:

[0039] The confidence loss is expressed as:

[0040]

[0041] Where q is the intersection over union (IoU) of the predicted box bbox and the real box gt; p is the score. If the two boxes intersect, q>0, which is a positive sample. If the two boxes do not intersect, q=0, which is a negative sample. α and γ are both hyperparameters.

[0042] The bounding box loss is expressed as:

[0043]

[0044] Where IoU is the intersection over union ratio, b and b gt They represent the center points of the two rectangular boxes, ρ represents the Euclidean distance between the two rectangular boxes, c represents the diagonal distance of the closed areas of the two rectangular boxes, v is used to measure the consistency of the relative proportions of the two rectangular boxes, and α is the weight coefficient.

[0045] Compared with the prior art, the present invention has the following technical effects:

[0046] The network model OSEADet of the present invention introduces an enhanced feature pyramid network EFPN and a dynamic task alignment shared detection head DSTA. On the one hand, the EFPN enhances the representation capability of information of different scales in the feature map, thereby improving the accuracy of small target detection. This is because the local branch, the large-scale branch and the global branch can perform multi-level processing on the feature map, so that the feature map can capture information of different scales, and then the multi-level features are fused, thereby enhancing the expression capability of the feature map and improving the model's perception capability of small targets in remote sensing images. Then, the fast Fourier transform is used to convert the features into the frequency domain for processing, which can better retain the global information and improve the integrity of the features. On the other hand, DSTA fully combines the advantages of the decoupling head and the coupling head, first completes the prediction of the target position and category through the bounding box regression branch and the classification branch, and then calculates the bounding box regression loss to ensure the accuracy of the prediction, and then merges the three DSTA tasks through group convolution, learns the task interaction features from multiple convolutional layers, enhances the interaction between different tasks, makes the information more effectively transmitted and shared between tasks, improves the detection accuracy and speed, and makes the model more lightweight; It can be seen that the network model OSEADet of the present invention achieves a balance between speed and accuracy compared with the existing network models for remote sensing target detection, and can accurately and real-time detect remote sensing targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 : A schematic diagram of the structure of the OSEADet network model of the present invention;

[0048] Figure 2 : A schematic diagram of the structure of the CSPF module of the present invention;

[0049] Figure 3 : A schematic diagram of the structure of the coupling head and the decoupling head of the present invention;

[0050] Figure 4 : A schematic diagram of the structure of the DSTA detection head of the present invention;

[0051] Figure 5 : Visualization results of the OSEADet network model of the present invention on the RSOD dataset. DETAILED DESCRIPTION

[0052] The specific contents of the present invention are further explained in detail below in conjunction with embodiments.

[0053] A remote sensing target detection method based on convolutional neural network includes the following steps:

[0054] Step 1: Build Figure 1 The Omni-Scale Enhanced Align Detection Network (OSEADet) shown in the figure is composed of three parts: Backbone, Neck and Head. The remote sensing image data flows from left to right in the network model, passing through Backbone, Neck and DSTA in turn, realizing the interaction and parameter sharing between different tasks of the detector, reducing the number of parameters and improving the detection accuracy. Among them:

[0055] The Backbone uses a five-layer feature pyramid structure built based on a CNN network to extract feature maps P1, P2, P3, P4 and P5 of different scales of remote sensing images;

[0056] Considering the following two problems of existing network models in detecting remote sensing targets: on the one hand, due to the wide viewing angle of remote sensing images, the size of the obtained images is often larger than that of general images, which makes the foreground information and background information in the image extremely unbalanced, which easily causes the network model to fail to learn high-quality foreground information and also causes problems such as overlapping and occlusion of targets. In addition, small targets in the image provide less pixel information and may only occupy a few pixels on the feature map with lower resolution, which makes it difficult for these layers to capture enough spatial information to accurately identify and locate small targets;

[0057] On the other hand, in the target detection task, the traditional feature pyramid network FPN usually uses feature layers of different resolutions to detect targets of different sizes. The P1 to P5 layers correspond to feature maps of higher, medium and lower resolutions, respectively. The resolution of the feature map is inversely proportional to its receptive field. Feature maps with higher resolutions (such as P1) usually contain more fine-grained spatial information, while feature maps with lower resolutions (such as P5) contain richer semantic information but less spatial details. Although feature maps with higher resolutions (such as P1) retain more spatial information, their receptive fields are smaller, resulting in a lack of contextual information, which may be detrimental to distinguishing small targets from backgrounds or other adjacent objects. Therefore, when only P3, P4 and P5 layers are used, the fine-grained information provided by feature maps with higher resolutions (such as P2 layers) cannot be fully utilized, and this information is particularly important for the accurate detection of small targets in remote sensing images. In the prior art, a feature layer with higher resolution (such as P2) is usually introduced to better retain the spatial details of small targets and improve the detection accuracy. However, after adding the P2 detection layer, the computational complexity of the network model is too large, and post-processing is more time-consuming.

[0058] Therefore, in this embodiment, Neck is designed as an Enhanced Feature Pyramid Network (EFPN). EFPN is proposed on the basis of the PAFPN network structure. Compared with the traditional FPN which only performs feature fusion in the P3, P4 and P5 layers, EFPN adds a P2 detection layer. The feature map P2 is subjected to SPDConv to output a feature map rich in small target information, which is fused with the feature map P3. Then, the cross-stage partial (CSP) idea and the CSPF module are used for feature integration, thereby effectively learning feature representations from global to local, and improving the detection performance of the network model for small targets in remote sensing images.

[0059] Since the resolution of P1 is relatively high, resulting in a too low receptive field, this embodiment only uses P2, P3, P4 and P5 for target detection. The resolutions of P2, P3, P4 and P5 are 1 / 4, 1 / 8, 1 / 16 and 1 / 32 times of the original image, respectively.

[0060] like Figure 2As shown in the figure, the CSPF module includes three branches, namely, the global branch (Global-level), the large-scale branch (Large-level) and the local branch (Local-level). First, the local features, large-scale features and global features are processed at multiple levels, so that the feature map can capture information of different scales. Then, the multi-level features are fused to enhance the expression ability of the feature map and improve the model's perception of small targets. Then, the features are converted to the frequency domain for processing by fast Fourier transform, which can better retain global information and help improve the integrity of the features. In short, the CSPF module can effectively learn feature representations from global to local, enhance the representation ability of feature maps for information of different scales, improve the network's detection accuracy for small targets, and reduce the amount of calculation. That is, CSPF can effectively improve the detection accuracy by enhancing the perception ability of feature maps for small targets, especially in the case of processing high-resolution images and complex backgrounds.

[0061] The CSPF module first extracts features from the input feature map X through a convolution layer, and divides the feature map into X1 and X2 according to the ratio of 1 / 4 and 3 / 4. The 1 / 4 feature map X1 is channel compressed through a 1×1 convolution, and then three-level feature extraction is performed through the local branch, large-scale branch and global branch respectively, where:

[0062] The local branch extracts local features through 1x1 depth convolution (DC), which is expressed as:

[0063] y=C 1×1 (X1)

[0064] O local =DC 1×1 (y)

[0065] In the formula, O local is the output of the local branch, C 1×1 Represents 1×1 convolution, DC 1×1 Represents a 1×1 depth convolution;

[0066] The large-scale branch extracts large-scale features through three depth convolutions (DCs), which can be expressed as:

[0067] O large =DC 1×61 (y)+DC 61×61 (y)+DC 61×1 (y)

[0068] Where y is the input of the large-scale branch, DC 1×61 , D.C. 61×61 and DC 61×1They are the depth convolutions of three large kernels, O large is the output of the large-scale branch;

[0069] The global branch first converts the feature map to the frequency domain through the fast Fourier transform (FFT), performs feature transformation, then converts it back to the spatial domain through the inverse fast Fourier transform (IFFT), and restores the feature map through 1×1 convolution, which is expressed as:

[0070] Z = IFFT(FFT(y) × C 1×1 (y))

[0071] O global =Z×C 1×1 (Z)

[0072] Wherein, FFT() represents fast Fourier transform, IFFT() represents inverse fast Fourier transform, global Represents the output of the global branch, Z is an intermediate quantity and has no actual meaning;

[0073] The outputs of the local branch, large-scale branch, and global branch are fused by element-wise addition, and the 3 / 4 feature map X2 of the initial convolution and the fused features are element-wise added, and then the final feature map F is output through a convolution layer. out , expressed as:

[0074] F out =Conv(X2+C 1×1 (O local +O large +O global ));

[0075] In the target detection task, the head is used to predict the target category and position. The head is usually designed as a coupled head or a decoupled head. For example, versions before YOLOv5 use a coupled head, while versions after YOLOv6 use a decoupled head. Figure 3 As shown in (a), the coupling head interweaves the prediction of target category and position information and shares the network layer. The advantage is that the model structure is simple and the computational efficiency is high. However, since the category prediction and position prediction are combined in one head, the error of one task affects the other task. In addition, the goal of category prediction is classification, and the goal of position prediction is regression. That is, category prediction and position prediction have different optimization goals. Using the same network layer for prediction increases the difficulty of optimization. Figure 3As shown in (b), the decoupling head processes the category prediction and location prediction tasks separately and uses different network layers for prediction. Although this can reduce the mutual influence of errors and improve the performance of the model, the two task branches work independently and lack information interaction. They cannot fully utilize the high-level features extracted by the backbone network, resulting in low model utilization and increased model complexity and computational complexity.

[0076] In order to effectively improve the accuracy and speed of remote sensing rotating target detection and overcome the above problems existing in the coupling head and the decoupling head, the Head proposed in this embodiment includes three dynamic task alignment shared detection heads (Dynamic shared task alignment, DSTA), which fully combines the advantages of the decoupling head and the coupling head. The structure of DSTA is as follows: Figure 4 As shown, it includes a bounding box regression branch (Reg.) and a classification branch (Cls.), which are used to predict the target position and category respectively;

[0077] The bounding box regression branch and the classification branch are not independent. The three DSTA tasks are merged through group convolution, and the task interaction features are learned from multiple convolutional layers to obtain joint features, which realizes the migration and interaction of multiple tasks, making information more effectively transmitted and shared between tasks, improving detection accuracy while achieving lightweight.

[0078] The proposed DSTA dynamically adjusts the shared detection head between tasks, especially Figure 4 The TaskDecomposition layer in , enables the detection head to better align the detection results of different tasks, which means that when performing bounding box regression and classification, the outputs of the two can remain highly consistent, thereby improving the overall detection effect;

[0079] The DSTA utilizes Figure 4 The group convolution shown in the red box replaces the ordinary convolution. All convolution layers in the group convolution share parameters, which can also be called shared group convolution. It not only greatly reduces the number of parameters, but also makes the model lighter, especially on resource-constrained devices. At the same time, shared group convolution is also used in the subsequent classification branch and regression branch, as shown in Figure 4 As shown in the blue box, because the classification branch learns the same target when detecting targets at three different scales, F3, F4, and F5, parameters can be shared. When sharing convolution in the regression task, in order to deal with the problem of inconsistent target scales detected by each detection head, the Scale layer is used to scale the feature scale, thereby achieving parameter sharing between different scales.

[0080] Step 2: Use the existing public datasets as training sets and validation sets to train and validate the network model OSEADet;

[0081] When training the network model OSEADet, the initialization parameters are set as follows: training cycle (epochs) is 300, batch size (batchsize) is 16; the number of processes (works) is 8; the input image size is 640*640; the SGD optimizer is used to optimize the learning rate, and the maximum learning rate is set to 1×10 -3 , the minimum learning rate is 1×10 -5 ; The weight decay value is set to 5×10 -4 , the momentum is 0.937, and the weight decay strategy is used to prevent the network model OSEADet from overfitting. The network model training uses Early-Stopping. When the model detects that the validation loss Validation-Loss tends to be stable, the training is automatically terminated, and the network model reaches basic convergence;

[0082] During the training process, by minimizing the classification loss and regression loss, the difference between the predicted value and the true label value is reduced, so that the network model can learn more accurate parameters, thereby improving the prediction ability of the input data;

[0083] The classification loss adopts cross entropy loss, which is expressed as:

[0084]

[0085] In the formula, M is the number of categories, y ic is a sign function, taking a value of 0 or 1. If the true category of sample i is equal to c, then y ic Take 1, otherwise y ic Take 0, p ic is the probability that the observed sample i belongs to c, and N is the target number;

[0086] The regression loss uses confidence loss and bounding box loss, where:

[0087] The confidence loss is expressed as:

[0088]

[0089] Where q is the intersection over union (IoU) of the predicted box bbox and the real box gt; p is the score. If the two boxes intersect, q>0, which is a positive sample. If the two boxes do not intersect, q=0, which is a negative sample. α and γ are both hyperparameters.

[0090] The bounding box loss is expressed as:

[0091]

[0092] Where IoU is the intersection over union ratio, b and b gtRepresent the center points of the two rectangular boxes, ρ represents the Euclidean distance between the two rectangular boxes, c represents the diagonal distance of the closed area of ​​the two rectangular boxes, v is used to measure the consistency of the relative proportions of the two rectangular boxes, and α is the weight coefficient;

[0093] Step 3: Input the remote sensing image data into the trained network model to predict the location and category of the remote sensing target.

[0094] In order to evaluate the performance of the OSEADet network model proposed in this embodiment, comparative experiments are conducted on two public remote sensing datasets, RSOD and NWPU-10, respectively, between the existing network model and OSEADet. The experimental results are shown in Tables 1 and 2.

[0095] The RSOD (Remote Sensing Object Detection) dataset is an open dataset designed for remote sensing image target detection tasks. The dataset was released by Wuhan University in 2017 and contains 976 images and 6950 targets in four categories: airplanes, oil tanks, playgrounds, and overpasses.

[0096] The NWPU VHR-10 dataset is a 10-category geospatial object detection dataset released by Northwestern Polytechnical University, mainly for research purposes. The dataset contains a total of 650 very high resolution (VHR) remote sensing images, which are divided into 10 different types of targets. There is at least one target in each image, and most images contain more than one target. These images are cropped from Google Earth and Vaihingen datasets and manually annotated by experts. The ten categories of geospatial objects in the dataset include aircraft, ships, storage tanks, baseball diamonds, tennis courts, basketball courts, ground runways, ports, bridges, and vehicles.

[0097] The RSOD dataset and the NWPU VHR-10 dataset both contain scenes such as complex backgrounds and small targets.

[0098] Table 1 Comparative experimental results of different models in RSOD

[0099]

[0100] Table 2 Comparative experimental results of different models on NWPU

[0101]

[0102] As shown in Tables 1 and 2, on the RSOD dataset, the single-stage detection method achieved good results. On the NWPU dataset, the two-stage detection method (such as Faster R-CNN and Cascade R-CNN) achieved better results than the one-stage detection method. The reason is that compared with RSOD, the NWPU dataset has a wide variety of objects, and the boundaries between various objects may not be clear, which increases the difficulty of classification. In addition, there are more small objects or objects in dense areas, and the background is more complex, with occlusions and shadows, such as buildings and trees. These noises affect the accuracy of classification. The two-stage method first generates a series of high-quality candidate regions (Region Proposals) through RPN (Region Proposal Network), and then finely classifies and locates these regions. The two-step approach helps to locate the target more accurately, especially when the target size, shape and proportion vary greatly. The two-stage detection method can play an advantage; however, neither the single-stage detection method nor the two-stage detection method can achieve good detection results on both the RSOD dataset and the NWPU dataset at the same time;

[0103] In comparison, the mAP of the network model OSEADet proposed in this embodiment on the RSOD dataset and the NWPU dataset is improved compared with the existing detection methods, indicating that OSEADet has greatly improved the detection accuracy of small targets in remote sensing images compared with the existing one-stage and two-stage detection methods, proving the effectiveness of the remote sensing target detection method based on convolutional neural network in this embodiment.

[0104] Figure 5 The visualization results of the OSEADet network model on the RSOD dataset are shown. As can be seen from the figure, small targets in the remote sensing image are marked in rectangular boxes, including airplanes, ships, and cars, indicating that the OSEADet network model can accurately detect small targets in satellite images.

Claims

1. A remote sensing target detection method based on convolutional neural network, characterized in that: The steps include: Step 1: Construct a full-scale enhanced alignment detection network model OSEADet including Backbone, Neck and Head, where: The Backbone uses a five-layer feature pyramid structure built based on a CNN network to extract feature maps P1, P2, P3, P4 and P5 of different scales of remote sensing images; The Neck is an enhanced feature pyramid network EFPN. EFPN adds a P2 detection layer on the basis of the PAFPN network structure. The feature map P2 is processed by SPDConv to output a feature map rich in small target information. The feature map rich in small target information is fused with the feature map P3, and then the cross-layer connection CSP idea and CSPF module are used for feature integration. The CSPF module includes a global branch, a large-scale branch and a local branch. The CSPF module performs multi-level processing on the feature map through the global branch, the large-scale branch and the local branch, so that it captures information of different scales, and then fuses the multi-level features. Then, through the fast Fourier transform, the features are converted to the frequency domain for processing, thereby retaining the global information of the features; The head includes three dynamic task alignment shared detection heads DSTA, which combine the advantages of the decoupling head and the coupling head, including a bounding box regression branch and a classification branch, which are used to predict the target position and category respectively, wherein: the tasks of the three DSTAs are merged through group convolution, task interaction features are learned from multiple convolutional layers, joint features are obtained, the interaction between tasks is increased, and tasks are dynamically adjusted through the TaskDecomposition layer to align the tasks; Step 2: Use existing public datasets to train and validate the network model OSEADet, and reduce the difference between the predicted value and the true label value by minimizing the classification loss and regression loss; Step 3: Use the trained network model OSEADet to predict the location and category of the remote sensing target.

2. The remote sensing target detection method based on convolutional neural network according to claim 1, characterized in that: The process of integrating features using the cross-layer connection CSP concept and the CSPF module in step 1 is as follows: Step 1.1, the CSPF module extracts features from the input feature map X through a convolution layer, and divides the feature map into X1 and X2 according to the ratio of 1 / 4 and 3 / 4. The 1 / 4 feature map X1 is channel compressed through a 1×1 convolution, and then used as the input of the local branch, large-scale branch and global branch respectively, for three-level feature extraction, where: The local branch extracts the local features of the feature map through 1x1 depth convolution, which is expressed as: y=C 1×1 (X1) EITHER local =DC 1×1 (and) In the formula, O 1ocal is the output of the local branch, C 1×1 Represents 1×1 convolution, DC 1×1 Represents a 1×1 depth convolution; The large-scale branch extracts the large-scale features of the feature map through three deep convolutions, which can be expressed as: EITHER large =DC 1×61 (y)+DC 61×61 (y)+DC 61×1 (and) Where y is the input of the large-scale branch, DC 1×61 , D.C. 61×61 and DC 61×1 They are the depth convolutions of three large kernels, O large is the output of the large-scale branch; The global branch first converts the feature map to the frequency domain through the fast Fourier transform (FFT), performs feature transformation, and then converts it back to the spatial domain through the inverse fast Fourier transform (IFFT), and restores the feature map through 1×1 convolution, which is expressed as: Z=IFFT(FFT(y)×C 1×1 (y)) ABOUT global =Z×C 1×1 (WITH)) Where, FFT() stands for Fast Fourier Transform, C 1×1 represents 1×1 convolution, IFFT() represents inverse fast Fourier transform, and Oglobal represents the output of the global branch; Step 1.2: The outputs of the local branch, large-scale branch, and global branch are fused by element-wise addition, and the 3 / 4 feature map X2 and the fused features are element-wise added, and then the final feature map F is output through a convolutional layer. out , expressed as: F out =Conv(X2+C 1×1 (THE local +O large +O global ))。 3. The remote sensing target detection method based on convolutional neural network according to claim 1, characterized in that: All convolutional layers in the group convolution of step 1 share parameters.

4. The remote sensing target detection method based on convolutional neural network according to claim 1, characterized in that: The classification loss in step 2 adopts cross entropy loss, which is expressed as: In the formula, M is the number of categories, y ic is a sign function, taking a value of 0 or 1. If the true category of sample i is equal to c, then y ic Take 1, otherwise y ic Take 0, p ic is the probability that the observed sample i belongs to c, and N is the target number.

5. The remote sensing target detection method based on convolutional neural network according to claim 1, characterized in that: The regression loss of step 2 adopts confidence loss and bounding box loss, where: The confidence loss is expressed as: Where q is the intersection over union (IoU) of the predicted box bbox and the real box gt; p is the score. If the two boxes intersect, q>0, which is a positive sample. If the two boxes do not intersect, q=0, which is a negative sample. α and γ are both hyperparameters. The bounding box loss is expressed as: Where IoU is the intersection over union ratio, b and b gt They represent the center points of the two rectangular boxes, ρ represents the Euclidean distance between the two rectangular boxes, c represents the diagonal distance of the closed areas of the two rectangular boxes, v is used to measure the consistency of the relative proportions of the two rectangular boxes, and α is the weight coefficient.

Images