A spatial small target detection method based on position coding

Abstract

The application discloses a space small target detection method based on position coding and belongs to the technical field of aerospace. The application is implemented by the following method: a plurality of video frame images containing small celestial bodies are acquired to construct a small celestial body data set. h and R w Image information is decomposed in x and y dimensions, global self-attention is realized on a 2D feature map, an attention module is added in a feature fusion network PANet, an ASFF_Detect is used in a detection head network, and a small target detection MA-YOLO model based on position coding is constructed based on the feature extraction network, the feature fusion network and the detection head network. The model is trained on the small celestial body data set by using a transfer learning method, and an optimal small target detection MA-YOLO model based on position coding is obtained after training, so that the small target detection efficiency and accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of aerospace, and particularly relates to a space small target detection method based on position coding. BACKGROUND

[0002] Deep space exploration is a detection activity of human beings on deep space environment and celestial bodies, and has become the research focus of space agencies of various countries. The existing research is mainly for the detection of planets, and the research on small celestial body target detection is less. Since the deep space exploration target is often far away, when the target volume is small, a small target will also be generated in imaging, and the target needs to be detected from an image with a large background pixel ratio. The small target mentioned in the present application refers to a target with a pixel area less than 32x32 pixels. Moreover, the field of view range of the deep space exploration camera is small, and the target stays in the field of view for a short time, so it needs to be detected quickly. In view of the characteristics of small celestial body detection target, such as small size and short detection time, the existing target detection technology cannot guarantee the accuracy and real-time requirements.

[0003] With the continuous development of artificial intelligence technology, the target detection method based on deep convolutional neural network improves the accuracy and speed of detection. The R-CNN series algorithm has good detection effect on small targets, but needs multi-stage training and has slow running speed; the YOLO series algorithm can realize end-to-end training and has fast running speed, but has limited detection ability on small targets. Therefore, it is necessary to propose a small target detection method with strong small target detection ability and fast running speed, to ensure the accuracy and real-time of detection. SUMMARY

[0004] The main purpose of the present application is to provide a space small target detection method based on position coding, by acquiring a plurality of video frame images containing small celestial bodies, and using a filtering algorithm to pre-process the images, constructing a small celestial body data set and labeling, dividing the labeled images into a training set, a verification set and a test set. A feature extraction network CSPMHSA including a multi-head self-attention MHSA network and a CSPDarkNet53 is constructed, and the position coding characteristics of the MHSA network are comprehensively utilized, and two-dimensional position coding R h and R wThe image information is decomposed in x and y dimensions, so that the target information can be effectively associated with the position information, and global self-attention is realized on the 2D feature map; an attention module is added to the feature fusion network PANet, information of different scales is fused, and then important information is strengthened through the attention module, so that the multi-scale feature fusion network improves the extraction ability of small targets; the detection head network uses ASFF_Detect, and the adaptive spatial feature fusion ASFF can solve the inconsistency problem of different scales of the feature pyramid, and useful information is reserved for combination, and the detection head Detect is used to detect the category and position of the target; based on the feature extraction network, the feature fusion network and the detection head network, a small target detection MA-YOLO model based on position coding is constructed. The position coding-based small target detection network model is pre-trained on the general data set to obtain a general data set pre-training model. Using the transfer learning method, the small celestial body data set is trained, and the optimal position coding-based small target detection MA-YOLO model is obtained after training; the small target data image to be identified is input into the pre-trained optimal MA-YOLO model, and the optimal MA-YOLO model is used to realize real-time and high-accuracy position coding-based small target detection. The small target detection method has the advantages of high real-time and accuracy, improves the small target detection efficiency and accuracy, and realizes intelligent perception of deep space small targets.

[0005] To achieve the above object, the technical scheme adopted by the present application is as follows:

[0006] The position coding-based space small target detection method disclosed by the present application comprises the following steps:

[0007] Step one: obtain a plurality of video frame images containing small celestial bodies, and use a filtering algorithm to pre-process the images, construct a small celestial body data set and label, divide the labeled images into a training set, a verification set and a test set.

[0008] As a preferred embodiment, a bilateral filtering algorithm is used to filter the plurality of video frame images containing small celestial bodies, so as to reduce the influence of image blur on target recognition.

[0009] Step two: introduce a multi-head self-attention mechanism into the network, create Query (q), Key (k), and Value (v), and divide the obtained images into three parts, and the self-attention expression is as follows:

[0010]

[0011] When using multi-head self-attention, the expression is as follows:

[0012] MultiHead(Q,K,V)=Concat(head1,...,head h )W o

[0013] where head i =Attention(QW i Q ,KW i K ,VW i V )

[0014] Replace the Linear layer in the multi-head self-attention mechanism with a Pointwise (PW) convolution, which further reduces the parameters of the algorithm due to its shared parameter feature.

[0015] The feature extraction network CSPMHSA is constructed by fusing the multi-head self-attention MHSA network and the CSPDarkNet53 backbone. In the fifth, seventh, and tenth layers of the CSPDarkNet53 backbone network, the multi-head self-attention MHSA network is used to replace the feature extraction module in the CSPDarkNet53 backbone network, which utilizes the position encoding feature of the MHSA network and uses two-dimensional position encodings R h and R w to decompose the image information in the x and y dimensions, effectively associating the target information with the position information, and realizing global self-attention on the 2D feature map.

[0016] Attention modules are added to the fourteenth, eighteenth, and twenty-first layers of the feature fusion network PANet. The attention modules strengthen important information and fuse information of different scales, improving the small target extraction capability of the multi-scale feature fusion network.

[0017] The detection head network uses the Detect network that integrates adaptive spatial feature fusion ASFF. The adaptive spatial feature fusion ASFF can solve the inconsistency problem of different scales of the feature pyramid, retain useful information for combination, and use the detection head Detect to detect the category and position of the target. According to the loss function, the weight parameters of the network model are constantly optimized and adjusted, which continuously improves the recognition accuracy of the small target detection MA-YOLO based on position encoding until the optimal parameters of the MA-YOLO model are obtained.

[0018] The loss function is composed of three types of losses: confidence loss function, bounding box loss function, and classification loss function. The loss function is the weighted sum of the three losses, as shown in the formula:

[0019] L = λ1L object + λ2L box + λ3L class

[0020] wherein, L object is a confidence loss function, L box is a bounding box loss function, and L class is a classification loss function.

[0021] Based on the feature extraction network, the feature fusion network and the detection head network, a small target detection MA-YOLO model is constructed.

[0022] As a preferred, the attention module uses SimAM, compared with the existing one-dimensional channel attention module and two-dimensional spatial attention module, SimAM is a three-dimensional attention module and does not need to add parameters, and better attention can be realized.

[0023] Step three: pre-training the small target detection model based on position encoding on a general data set to obtain a general data set pre-training model; using a transfer learning method, training on the small celestial body data set, and obtaining an optimal small target detection MA-YOLO model based on position encoding after training;

[0024] Step four: inputting a small target image to be identified into the pre-trained optimal MA-YOLO model, and realizing real-time and high-accuracy small target detection by using the optimal MA-YOLO model;

[0025] Further comprising step five, obtaining a detection result according to the optimal MA-YOLO model of step four, improving the small target detection efficiency and accuracy, and realizing intelligent perception of deep space small targets.

[0026] Advantages:

[0027] 1. The small target detection method based on position encoding disclosed in the application adopts a feature extraction network CSPMHSA, fuses a multi-head self-attention MHSA network and a CSPDarkNet53 main body, comprehensively utilizes the position encoding characteristics of the MHSA network, uses two-dimensional position encodings R h and R w decomposes image information in x and y dimensions, thereby effectively associating target information with position information, realizing global self-attention on 2D feature mapping, and improving the accuracy of small target recognition of the model.

[0028] 2. The spatial small target detection method based on position coding disclosed in the application fuses a feature fusion network and an attention module, enhances key information in target features extracted at each scale through the attention module, then fuses features of different scales, improves the extraction capability of key information of target features, enhances the representation capability of the small target detection model, and improves the detection precision and efficiency of small targets.

[0029] 3. The spatial small target detection method based on position coding disclosed in the application uses a filtering algorithm to pre-process an image, improves the quality of images obtained through deep space exploration, and reduces the training time of the small target detection model based on position coding under the condition of maintaining the same detection precision.

[0030] 4. The spatial small target detection method based on position coding disclosed in the application uses a YOLO series algorithm as a framework, utilizes the advantage of end-to-end training to reduce the training time, and simultaneously improves the algorithm by using the advantage of fast detection speed of the YOLO series, so that the small target detection model MA-YOLO based on position coding can also meet the real-time requirement.

[0031] 5. The spatial small target detection method based on position coding disclosed in the application has strong global feature extraction and feature fusion capability, has strong detection capability for targets, and has the characteristics of strong expansibility and wide application range.

[0032] 6. The spatial small target detection method based on position coding disclosed in the application can output the detection result in the form of text, including a target category, a target number and target position coordinates, and provides data support for subsequent space tasks. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a flowchart of the spatial small target detection method based on position coding in the application;

[0034] Figure 2 is a schematic diagram of the MA-YOLO target detection network structure in the application;

[0035] Figure 3 is a schematic diagram of the SimAM attention module in the application;

[0036] Figure 4 is a schematic diagram of the multi-head self-attention MHSA structure in the application;

[0037] Figure 5 is a schematic diagram of the path aggregation network PANet structure in the application;

[0038] Figure 6Part of the test result figure of the position coding small target MA-YOLO detection model in the application. DETAILED DESCRIPTION

[0039] In order to better illustrate the purposes and advantages of the application, the summary of the application is further illustrated below in combination with the drawings and examples.

[0040] Reference Figure 1 The embodiment discloses a position coding-based spatial small target detection method, and the specific implementation steps are as follows:

[0041] Step one: a plurality of video frame images containing small celestial bodies are acquired, and a bilateral filtering algorithm is used to pre-process the images, a small celestial body data set is constructed and labeled, and the labeled images are divided into a training set, a verification set and a test set. The bilateral filtering is a nonlinear filter, which can achieve the effects of edge preservation and noise reduction smoothing. The formula of the bilateral filtering is

[0042]

[0043] In the formula, I p is the pixel point brightness value after final filtering processing; G σd is a spatial domain kernel; G σr is an image pixel value domain kernel; w p is a normalization factor, which is the product of G σd and G σr ; I q is a field pixel point brightness value, I p is an intermediate point pixel brightness value; S is the range of the center point; and ||p-q|| is the spatial distance between the center point and the field point.

[0044]

[0045] w d and w r respectively represent the spatial domain kernel two-dimensional Gaussian filtering function G σd and the image pixel value domain kernel G σr , and the calculation formula is as follows:

[0046]

[0047]

[0048] In the formula, I(i,j) and I(m,n) are respectively the brightness values of the center point and a certain pixel point in the sliding window; σ d is a spatial standard deviation; and σ rThe value range standard deviation is selected. A window size of 3x3 is selected, the entire image is traversed, all pixel values within the window are compared with the window center pixel, the spatial domain kernel and the image pixel domain kernel size are calculated, and the product of the two is taken as the feature value of the point. The product of the feature value of each point and the pixel value of the point is added, and the sum of all feature values is divided by the sum of all feature values, and the result is taken as the pixel value of the center point.

[0049] The obtained picture is preprocessed by a bilateral filtering algorithm, and then the size of the processed picture is adjusted to 1024x1024, and the picture is named in a random order as a digital number for subsequent use. LabelImg is used as a marking tool, and after the picture is imported into LabelImg, the position of the small celestial body in the picture is marked with a box, and the type name is marked. In this way, all pictures are marked, xml format marking files are generated, and output to the specified folder to generate a small celestial body dataset.

[0050] Step two:

[0051] The multi-head self-attention mechanism is introduced into the network, and the obtained image is divided into three parts, Query (q), Key (k), and Value (v). The self-attention expression is as follows:

[0052]

[0053] When using multi-head self-attention, the expression is as follows:

[0054] MultiHead(Q,K,V)=Concat(head1,...,head h )W o

[0055] Where head i =Attention(QW i Q ,KW i K ,VW i V )

[0056] We will replace the Linear layer in the multi-head self-attention mechanism with Pointwise (PW) convolution. Since PW convolution has the feature of sharing parameters, the parameters of the algorithm are further reduced.

[0057] The feature extraction network CSPMHSA is constructed by fusing a multi-head self-attention MHSA network and a CSPDarkNet53 backbone, at the fifth layer, the seventh layer and the tenth layer of the CSPDarkNet53 backbone network, the feature extraction module in the CSPDarkNet53 backbone network is replaced by a multi-head self-attention MHSA network, the characteristics of the MHSA network with position encoding are comprehensively utilized, two-dimensional position encoding R h and R w The image information is decomposed in the x and y dimensions, so that the target information can be effectively associated with the position information, and global self-attention is realized on the 2D feature map;

[0058] Attention modules are added to the fourteenth layer, the eighteenth layer and the twenty-first layer of the feature fusion network PANet, important information is strengthened through the attention modules, then different scale information is fused, and the extraction ability of the multi-scale feature fusion network for small targets is improved;

[0059] The detection head network uses a Detect network fused with adaptive spatial feature fusion ASFF, the adaptive spatial feature fusion ASFF can solve the inconsistency problem of different scales of the feature pyramid, useful information is reserved for combination, and then the detection head Detect is used to detect the category and position of the target. According to the loss function, the weight parameters of the network model are continuously optimized and adjusted, so that the recognition accuracy of the small target detection MA-YOLO based on position encoding is continuously improved, until the best parameters of the MA-YOLO model are obtained.

[0060] A small target detection MA-YOLO model is constructed based on the feature extraction network, the feature fusion network and the detection head network.

[0061] The present application takes the YOLO series algorithm as the framework, introduces the MHSA with two-dimensional position encoding to enhance the recognition ability of small targets, and uses multiple attention mechanisms to enhance the extraction and detection ability of multi-scale features, forming the MA-YOLO small target detection model. The MA-YOLO small target detection model can be divided into three parts of a feature extraction network, a feature fusion network and a detection head network as shown in Figure 2 .

[0062] (1) Feature extraction network

[0063] The YOLOv5 network uses CSPDarkNet53 as the backbone network, and the structure of the CSPDarkNet53 is as shown in Figure 3As shown in the figure. As a feature extraction network, it has good performance in detection accuracy, but when facing small targets, the feature extraction capability is limited. In order to enhance the feature extraction capability of the feature extraction network for small targets, the feature extraction network CSPMHSA which combines the multi-head self-attention MHSA network and the CSPDarkNet53 backbone is used. The MHSA network has the characteristics of position coding, and the two-dimensional position coding R h and R w The image information is decomposed in x and y dimensions, so that the target information can be effectively associated with the position information, and the global self-attention is realized on the 2D feature map. The multi-head self-attention MHSA structure is as shown in the figure. Figure 4 In this example, we select a 4-head MHSA, and considering the real-time problem of the algorithm detection, we only add the MHSA structure in the last CSPDarkNet53 module.

[0064] (2) Feature fusion network

[0065] The feature fusion network can fuse the multi-scale information extracted by the feature extraction network to obtain a feature with better judgment ability than the input image. The application uses the path aggregation network (PANet) to fuse the parameter-free attention module SimAM as the feature fusion network, as shown in the figure. Figure 4 Specifically, after top-down feature fusion, bottom-up feature fusion is performed to fuse features of different scales together, enhance semantic information and position information, and facilitate small target detection. In order to further improve the feature extraction capability, the attention module SimAM is added in the last three modules of the path aggregation network to strengthen important information and improve the extraction capability of the multi-scale feature fusion network for small targets. The unique advantage of the SimAM attention mechanism is that it can directly derive three-dimensional weights for the feature map without increasing additional parameters, so that the network can learn more discriminative neurons to improve the feature extraction capability of the network. Most operations of the attention mechanism are based on optimization energy function selection, which avoids excessive structure adjustment and accelerates the calculation of attention weights, so as to ensure that the calculation amount is not increased while the effectiveness and flexibility of the attention mechanism SimAM are better exerted. According to the definition of the attention mechanism, the feature is enhanced:

[0066]

[0067] In the formula, X represents the neurons in a single channel of the input feature, represents the neurons in a single channel of the output feature, E represents all cross-channel and spatial dimension grouping numbers, and sigmod represents the activation function.

[0068] (3) Detection head network

[0069] The detection head network uses ASFF_Detect, and the adaptive spatial feature fusion ASFF can solve the inconsistency problem of different scales of the feature pyramid, retain useful information for combination, and then use the detection head Detect to detect the category and position of the target. First, through the adaptive spatial feature fusion ASFF, the inconsistency is suppressed by learning the spatial filter to suppress the contradictory information, thereby improving the scale invariance of the feature and the effectiveness of the feature fusion network, and then using the Detect module to predict the category and position of the target.

[0070] The loss function is often used as parameter estimation of machine learning, and the loss function designed by the application is composed of three types of losses, namely confidence loss function, boundary box loss function and classification loss function, and the loss function is obtained by weighted summation of the three losses, as shown in the formula:

[0071] L = λ1L object + λ2L box + λ3L class

[0072] Wherein, L object is the confidence loss function, L box is the boundary box loss function, and L class is the classification loss function.

[0073] The loss function L object and L class both use the binary cross-entropy loss function

[0074]

[0075] Wherein,

[0076] The loss function L box uses CIoU_loss, as shown in the formula:

[0077]

[0078] b and b gt represent the center points of the predicted box and the real box, and v is used to describe the proportional consistency of the predicted box and the real box, as shown in the formula:

[0079]

[0080] Alpha is a weight function used to balance the proportion, as shown in the formula:

[0081]

[0082] The CIoU_Loss function increases the measurement of the intersection scale, which is beneficial to solve the problem that the bounding box does not coincide sometimes, and considers the information of the distance of the center point of the bounding box and the scale information of the width-height ratio of the bounding box, so that the network has faster and better convergence effect.

[0083] Step three: pre-training the small target detection model based on position encoding on the VisDrone2021 dataset to obtain a general dataset pre-training model. Using the transfer learning method, training is performed on the small celestial body dataset, and the optimal small target detection MA-YOLO model based on position encoding is obtained after training;

[0084] The following training software environment is Windows 10, and the hardware configuration is NVIDIA RTX3070. Taking the training process of the lightweight deep space unknown target detection model as an example, the training parameters of the network are initialized, the pre-training is performed on the VisDrone2021 dataset, and then the pre-training model is trained on the small celestial body dataset. The parameter settings in the configuration file are as follows: the training iteration number is 2000 times, the learning rate is set to 0.001, the batch size is set to 16, the NMS is set to 0.5, the model is saved once every 100 iterations, the gradient descent algorithm (SGD algorithm) is used, the network parameters are updated, and finally the network training related parameters are modified to start model training.

[0085] Step four: inputting the small target image to be identified into the pre-trained optimal MA-YOLO model, and using the optimal MA-YOLO model to realize real-time and high-accuracy small target detection.

[0086] Further comprising step five, obtaining the detection result according to the optimal MA-YOLO model of step four. The small target detection method has high real-time performance and high accuracy, and can solve the problems of low detection efficiency and low accuracy of the prior art, and realizes intelligent perception of deep small targets.

[0087] The specific description above further describes the purpose, technical scheme and beneficial effects of the application. It should be understood that the above description is only a specific embodiment of the application, which is used to explain the application and does not limit the protection scope of the application. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the application shall be included in the protection scope of the application.

Claims

1. A method for detecting small targets in space based on position encoding, characterized in that: Comprising the following steps, Step one: by acquiring a plurality of video frame images containing small celestial bodies, and using a filtering algorithm to pre-process the images, constructing a small celestial body data set and labeling, dividing the labeled images into training set, validation set and test set; Step two: introduce multi-head self-attention mechanism in the network, create Query (q), Key (k), Value (v) to divide the obtained image into three parts, the self-attention expression is as follows: When using multi-head self-attention, the expression is as follows: MultiHead(Q, K, V) = Concat(head1,...,head h ) W o wherein head i = Attention(QW i Q ,KW i K ,VW i V ) Replace the Linear layer in the multi-head self-attention mechanism with Pointwise (PW) convolution. Because PW convolution has the feature of sharing parameters, the parameters of the algorithm are further reduced; The feature extraction network CSPMHSA including the fusion of the multi-head self-attention MHSA network and the CSPDarkNet53 backbone is constructed, in the fifth layer, the seventh layer and the tenth layer of the CSPDarkNet53 backbone network, the feature extraction module in the CSPDarkNet53 backbone network is replaced by the multi-head self-attention MHSA network, the characteristics of the MHSA network having position coding are comprehensively utilized, two-dimensional position coding R h and R w The image information is decomposed in the x and y dimensions, so that the target information can be effectively associated with the position information, and the global self-attention is realized on the 2D feature mapping; Add attention modules to the 14th, 18th and 21st layers of the feature fusion network PANet. The attention modules strengthen important information and fuse information of different scales to improve the extraction ability of the multi-scale feature fusion network for small targets. The detection head network uses the Detect network of the adaptive spatial feature fusion ASFF. The adaptive spatial feature fusion ASFF can solve the inconsistency problem of different scales of the feature pyramid, retain useful information for combination, and use the detection head Detect to detect the category and position of the target. According to the loss function, the weight parameters of the network model are constantly optimized and adjusted, so that the recognition accuracy of the small target detection MA-YOLO based on position encoding is constantly improved until the best parameters of the MA-YOLO model are obtained. The loss function is composed of three types of losses: confidence loss function, bounding box loss function and classification loss function. The loss function is the weighted sum of the three losses, as shown in the formula: L = λ1L object + λ2L box + λ3L class wherein, L object is a confidence loss function, L box is a bounding box loss function, L class is a classification loss function; Based on the feature extraction network, the feature fusion network and the detection head network, a small target detection MA-YOLO model is constructed. Step three: pre-train the small target detection model based on position encoding on a general data set to obtain a general data set pre-training model; use transfer learning method to train on the small celestial body data set, and obtain the optimal small target detection MA-YOLO model based on position encoding after training. Step four: input the small target image to be identified into the pre-trained optimal MA-YOLO model, and use the optimal MA-YOLO model to realize real-time and high-accuracy small target detection.

2. The method of claim 1, wherein: Step five: according to the detection result of the optimal MA-YOLO model in step four, improve the small target detection efficiency and accuracy, and realize intelligent perception of deep space small targets.

3. The method of claim 1 or 2, wherein: Use the bilateral filtering algorithm to filter the plurality of video frame images containing small celestial bodies to reduce the influence of image blur on target recognition.

4. The method of claim 1 or 2, wherein the method is based on position encoding. The attention module uses SimAM.

Images