Low-altitude small target identification method under high interference condition

Abstract

The invention belongs to the field of image recognition, and relates to a low-altitude small target recognition method under a high interference condition, which comprises a single-stage deep network and is characterized in that the single-stage deep network is used as a basic detection framework, and the feature extraction capability and robustness of the network are improved through transfer learning and an attention enhancement mechanism; comprising the following steps: acquiring an input image, performing target detection by taking a single-stage deep network as a detection network, and performing feature extraction on the input image through a feature enhancement module and a spatial perception module; channels of similar information are concluded and sorted through a space attention mechanism and a channel attention mechanism, and similar images are distinguished; a convolutional neural network backbone architecture CSPDarknet53 cross-stage local fusion network is adopted as a backbone network, channel pruning is carried out, model acceleration is realized, and lightweight processing of image features is executed; a student model and a teacher model are continuously iterated in a knowledge migration network, and a soft label for collaborative consistency guidance of a prediction label is constructed in a training process, so that the precision of the pruned model is further improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of image recognition, and particularly relates to a low-altitude small target recognition method under high interference conditions. BACKGROUND

[0002] When image acquisition is performed, small targets are formed on the image due to the long distance of the aircraft, and the small targets have few recognizable features, are prone to missed detection and false detection; in order to cope with possible malicious behaviors of the aircraft, real-time countermeasures are performed after recognition, so it is important to make the aircraft target detection algorithm meet the end-side real-time detection, and the inference speed needs to be improved; the aircraft is prone to be disturbed by complex backgrounds such as clouds, bird flocks, sea clutter and sea surface reflected light during flight, which leads to a relatively high misrecognition rate; model training relies on a large amount of labeled data, data acquisition is difficult, and label noise in the aircraft image training process brings great disturbance, and the generalization ability of the model is inhibited.

[0003] Traditional image processing methods such as background difference method and optical flow method are prone to false detection when the environment dynamically changes, such as cloud layer and leaf shaking, and it is difficult to extract features for small targets due to small pixel ratio. Pure single-stage detectors such as YOLOv5 and YOLOv8 and pure two-stage detectors such as FasterR-CNN cannot simultaneously meet the requirements of high recall rate and high real-time performance, and need to rely on a large amount of labeled data for training, and the robustness is poor.

[0004] The application realizes a small target detection and positioning method for low-altitude aircraft, a high-precision target recognition method under high interference conditions, improves the recall rate of small targets, reduces the model inference time, improves the recall rate under interference conditions, and improves the generalization ability. SUMMARY

[0005] The application provides a low-altitude small target recognition method under high interference conditions.

[0006] The low-altitude small target recognition method under high interference conditions comprises a single-stage deep network, uses the single-stage deep network as a basic detection architecture, and improves the feature extraction capability and robustness of the network through transfer learning and attention enhancement mechanism; and comprises the following steps.

[0007] An input image is acquired, a single-stage deep network is used as a detection network for target detection, and a feature enhancement module and a spatial perception module are used for feature extraction on the input image.

[0008] Similar information channels are summarized and sorted through a spatial attention mechanism and a channel attention mechanism, and similar images are distinguished.

[0009] A convolutional neural network backbone architecture CSPDarknet53 cross-stage partial fusion network is used as the backbone network, channel pruning is performed to realize model acceleration, and light processing of image features is performed;

[0010] Through the continuous iteration of the student model and the teacher model in the knowledge transfer network, a soft label guided by the cooperative consistency of the predicted label is constructed in the training process, further improving the accuracy of the pruned model.

[0011] In S1, the feature enhancement module FEM is used to enrich semantic information, and the spatial awareness module SAM introduces deformable convolution and adaptive spatial learning technology to capture more discriminative features of small targets.

[0012] The single-stage deep network is used as the detection network for target detection, and the feature enhancement module and the spatial awareness module are used for feature extraction of the input image, including the following sub-steps:

[0013] Based on the pre-set anchor points, the features of the image at different scales are extracted by the convolutional neural network;

[0014] The offset of the detection frame and the anchor point coordinates in four dimensions and the confidence and classification score of the target in the frame are predicted;

[0015] The non-maximum suppression method is used to remove and denoise the prediction results with low confidence and large overlap area to obtain the final detection result.

[0016] The channel pruning also includes the following content: through the means of scaling control of model depth and the number of convolution kernels in each layer, the channel pruning is performed to further compress the model parameter quantity.

[0017] The student network and the teacher network are included in S4, including the following sub-steps:

[0018] S41: First, train the teacher network to achieve a pre-set threshold;

[0019] S42: Fit the teacher network through the student network, so as to transfer the knowledge learned by the teacher network to the student network;

[0020] S43: Train the knowledge distillation of the student network through the output of the teacher network normalization exponential function layer.

[0021] The spatial attention mechanism and the channel attention mechanism include a method based on a dual-channel attention mechanism, which focuses on more discriminative areas using the spatial attention mechanism, and integrates and induces channels containing similar semantic information using the channel attention mechanism, so that the classification result is more discriminative.

[0022] The channel of similar information is induced and arranged through the spatial attention mechanism and the channel attention mechanism, and the spatial attention module and the channel attention module are integrated into the network in the form of a residual.

[0023] The precision of the pruned model is further improved by iteratively updating the student model and the teacher model in the knowledge transfer network and constructing the soft label guided by the collaborative consistency of the predicted labels during the training process, and the following contents are included:

[0024] A student model and a teacher model are set, and the teacher model is integrated for the student model multiple times to predict model parameters;

[0025] In several rounds of iterations, the teacher model parameters are updated, and the loss function of the collaborative consistency of the predicted labels of the student model and the teacher model is constructed;

[0026] An input sample set is set, and based on the input sample set and its noisy true value label, the corresponding predicted label is obtained after the teacher model;

[0027] The predicted label and the true value label are mixed according to a preset proportion to construct a soft label of the aircraft image sample;

[0028] The soft label is used as a supervision signal to guide the training of the aircraft feature extraction model, and the student model parameters are updated according to this.

[0029] The soft label is used as a supervision signal to guide the training of the aircraft feature extraction model, and the student model parameters are updated according to this, and in order to ensure the stability of the prediction, the calculation of the loss function is further included, and the following sub-steps are included:

[0030] The loss based on the aircraft fine-grained image recognition task is calculated;

[0031] The collaborative consistency loss of the predicted label of the student model and the predicted label of the teacher model is calculated;

[0032] The loss function is calculated and obtained according to the fine-grained image recognition task loss, the collaborative consistency loss, the coordination parameter of the task loss and the collaborative consistency loss.

[0033] The application also discloses a low-altitude small target recognition device under high interference conditions, comprising a processor and a memory, wherein the memory stores a computer program that can be called by the processor; the processor executes the low-altitude small target recognition method under high interference conditions in the background by calling the computer program stored in the memory.

[0034] The application further discloses a computer readable storage medium, which stores an erasable computer program; when the computer program runs on a computer device, the computer device is caused to execute the low-altitude small target identification method under a high-interference condition in the background.

[0035] The application has the following beneficial effects:

[0036] 1. Based on a single-stage deep network architecture, a feature enhancement module FEM and a spatial perception module SAM are added, on one hand, the FEM is used to enrich the semantic information of an image, and on the other hand, the deformable convolution and adaptive spatial learning technology in the SAM are used to accurately capture the discriminative features of small targets, so that the recall rate of the model is significantly improved when facing small targets such as low-altitude aircraft, and the missed detection and false detection are reduced, and meanwhile, the multi-scale prediction mechanism ensures the real-time detection, and meets the real-time countermeasure requirement of the end side;

[0037] 2. The CSPDarknet53 cross-stage local fusion network is used as the backbone, the channel pruning technology is used to control the depth and the number of convolution kernels of the model, the model parameter quantity is greatly compressed, the knowledge distillation method is combined to guide the training of the pruned student network by the high-precision teacher network, the performance is prevented from being reduced while the inference time is reduced, the model is more suitable for end-side deployment, the real-time inference speed of the algorithm is effectively improved, and the high-power energy efficiency requirement is met;

[0038] 3. The dual-channel attention mechanism of spatial attention and channel attention is used to focus on the discriminative regions in the image, integrate similar semantic information, and reduce the interference of complex backgrounds such as cloud layers and sea clutter and image noise, meanwhile, the soft label is constructed by using the integrated iterative self-distillation collaborative method, the noisy true value label and the prediction label of the teacher model are mixed in proportion, the influence of label noise on training is reduced, the model is more robust in the feature extraction process, and the target identification accuracy and the generalization ability under high-interference conditions are significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 FIG. 1 is a step flow diagram of the low-altitude small target identification method under a high-interference condition according to the application;

[0040] Figure 2 FIG. 5 is a small target detection model diagram of the single-stage deep network of the low-altitude small target identification method under a high-interference condition according to the application;

[0041] Figure 3 FIG. 7 is a model compression method of the low-altitude small target identification method under a high-interference condition according to the application;

[0042] Figure 4A feature extraction module based on an attention mechanism for a low-altitude small target recognition method under high interference conditions according to the present application is shown in the schematic diagram.

[0043] Figure 5 A feature extraction model based on an attention mechanism for a low-altitude small target recognition method under high interference conditions according to the present application is shown in the schematic diagram.

[0044] Figure 6 A self-distillation collaborative framework for a low-altitude small target recognition method under high interference conditions according to the present application is shown in the schematic diagram.

[0045] Figure 7 A low-altitude small target recognition device under high interference conditions according to the present application is shown in the schematic diagram. DETAILED DESCRIPTION

[0046] In order to enable personnel in the technical field to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly described below in conjunction with embodiments.

[0047] As shown in Figure 1 , the present application proposes a low-altitude small target recognition method under high interference conditions, which includes the following contents:

[0048] I. Small target detection and positioning method for low-altitude aerial vehicles

[0049] The present application intends to use a single-stage deep network as a basic detection architecture, and to optimize and improve the network structure, loss function design, training method, etc. of the model. The transfer learning and attention enhancement mechanism are used to improve the feature extraction capability and robustness of the network, and to focus on solving the problem of small target detection difficulty in the unmanned aerial vehicle detection scene. In addition, model compression means is used for end-side deployment algorithm, so as to reduce the parameters and inference time of the model.

[0050] Specifically, the following technical route is intended to be used:

[0051] (1) Small target detection model based on single-stage deep network

[0052] The single-stage deep network is an end-to-end detection network that completes the tasks of target detection and classification in the same network. For a single input image,

[0053] First, based on pre-set anchor points, the features of the image at three different scales are extracted through a convolutional neural network,

[0054] and the four-dimensional offset of the detection frame and anchor coordinates, as well as the confidence and classification score of the target in the frame, are predicted.

[0055] Then, through the non-maximum suppression method, the prediction results with low confidence and large overlap area are de-duplicated and de-noised to obtain the final detection results.

[0056] The four dimensions include:

[0057] The center coordinate offset in the x-axis direction: the displacement of the center of the detection frame relative to the center of the anchor point in the horizontal direction x-axis; the center coordinate offset in the y-axis direction: the displacement of the center of the detection frame relative to the center of the anchor point in the vertical direction y-axis;

[0058] The offset in the width direction: the scaling ratio or absolute difference of the width of the detection frame relative to the width of the anchor point;

[0059] The offset in the height direction: the scaling ratio or absolute difference of the height of the detection frame relative to the height of the anchor point.

[0060] The classification score acquisition method includes: extracting class-related features through a convolution layer, and then performing normalization through a softmax normalization exponential function to generate a quantized classification score.

[0061] Since the above operations are implemented in a neural network, the detection speed is greatly improved compared to traditional detection networks, and the multi-scale prediction method enhances the network's ability to detect targets of different sizes, especially in local exposure small target detection. With the above advantages, the present scheme focuses on the local exposure target detection task and makes targeted improvements,

[0062] As shown in the following figure, a feature enhancement module and a spatial perception module are added,

[0063] The feature enhancement module FEM can enrich semantic information, and the spatial perception module SAM introduces deformable convolution, adaptive spatial learning and other technologies, which can capture more discriminative features of small targets to improve the comprehensive performance and robustness of the detection model.

[0064] (2) End-side model compression

[0065] Model compression is an important way to model lightweight, and there are many model compression techniques based on artificial methods, such as parameter pruning, low-rank decomposition, network quantization, knowledge transfer network, etc.

[0066] The idea of network weight sparsification or parameter pruning is to reduce the storage and computation of the network by removing part of the connection in the neural network, according to whether to constrain the position of the elements that can be pruned.

[0067] Figure 3The detection model shown adopts CSPDarknet53 cross-stage local fusion network as the backbone network, learns from the idea of dense cross-layer connection in DenseNet, adopts means of scaling control on model depth and number of convolution kernels of each layer for channel pruning, and then compresses the model parameter quantity, so as to achieve the purpose of model acceleration.

[0068] Due to the reduction of parameter quantity, the model performance will naturally decrease to a certain extent, therefore the scheme further adopts the knowledge transfer method to improve the accuracy of the pruned model. The knowledge transfer network is different from the above network acceleration and compression method, this method uses two types of networks, one is the teacher network, and the other is the student network.

[0069] This method first trains the teacher network to achieve very high performance, and then uses the student network to fit the teacher network, so as to transfer the knowledge learned by the teacher network to the student network.

[0070] The knowledge distillation (KD) of the student network can be trained by the output of the softmax normalization exponential function layer of the teacher network, therefore the scheme adopts the model with higher accuracy as the teacher network, the pruned model as the student network, and uses the knowledge distillation method for training.

[0071] II. High-precision target recognition method under high interference condition

[0072] The existing aircraft target image or video data usually has a certain degree of noise, which is very challenging for network training. It is necessary to learn robust and robust features in samples with noise.

[0073] In order to solve this problem, an aircraft image feature extraction method based on attention focusing and self-distillation cooperation is proposed.

[0074] Specifically, in the process of learning image feature distribution, model training is a more detailed sub-class division of the same basic class. Compared with ordinary image recognition tasks, local target image recognition has the problem of large intra-class difference and small inter-class difference, which is more difficult.

[0075] We need to distinguish similar images and obtain a discriminative feature representation, so that samples of the same fine classification are close in feature space, and samples of different fine classifications are far apart. To this end, we propose a method based on a dual-channel attention mechanism, as shown in the figure. The spatial attention mechanism focuses on more discriminative areas, and the channel attention mechanism integrates and induces channels containing similar semantic information, making the classification result more discriminative. At the same time, the spatial attention module and the channel attention module are integrated into the network in the form of a residual, making the network more robust and stable, and enabling the extraction of more robust aircraft features.

[0076] Specifically, more discriminative areas refer to areas that contain features unique to the target and can be used for accurate identification, such as the contour, wing, tail, and other morphological features of low-altitude aircraft, or unique textures, local pixel distributions related to motion trajectories, etc.

[0077] The spatial attention mechanism automatically enhances the feature signals of key areas by calculating the importance weights of different positions in the feature map, and weakens the influence of irrelevant background or interference areas, so that the model focuses more on the core areas that can effectively distinguish the target from the interference, improving the accuracy of the classification result.

[0078] Specifically, taking the channel attention module as an example, set the input feature map F ∈ R CxHxW , reshape F twice to get F 1 ∈ R CxN , 2 , NxC , 3 , CxN , N = H x W.

[0079] Multiply F 1 and F 2 matrices, and normalize through softmax to get the channel attention mask M ∈ R CxC .

[0080] Then, apply M to F 3 in the form of matrix multiplication, and reshape to get the feature map F 4 ∈ R CxHxW after channel attention.

[0081] Finally, the channel attention module is integrated into the network in the form of a residual, and F' ∈ R CxHxW is obtained, i.e.:

[0082] F ' = F + F 4

[0083] The feature extraction model structure is as follows Figure 4 ,5 As shown in the figure, the feature extraction model adopts a residual network resnet as the skeleton model, and integrates spatial self-attention mechanism and channel self-attention mechanism into the model in a modular form, and obtains the aircraft features through generalized average pooling GEM.

[0084] In actual application scenarios, there are certain errors in picture labels, which will introduce noise in the training process, reduce the recognition accuracy of the model, and affect its generalization performance. Therefore, we propose a self-distillation collaborative method based on integrated iteration, which constructs a soft label guided by the collaborative consistency of the predicted label in the training process, reduces the interference brought by label noise, and improves the generalization performance of the model.

[0085] As Figure 5 shown, the sample soft label is constructed, and a student model and a teacher model are set. The teacher model is an integration of the student model's multiple prediction model parameters. In each iteration, the teacher model updates the parameters according to the sliding average principle, and constructs a loss function of the collaborative consistency of the predicted labels of the student model and the teacher model to ensure the stability of the predicted labels. Set the current iteration as t, the student model parameter as θ t , the teacher model parameter as θ' t , and the student model parameter θ t is updated to the teacher model parameter θ' t , that is:

[0086]

[0087] Then, the obtained predicted label is mixed with the original noisy true value label according to the proportion α to construct the soft label of the aircraft sample, which is used as the supervision signal for fine-grained aircraft image recognition.

[0088] Specifically, set the input sample set X = {x1, x2, x3……x N}, N is the number of training samples, and the corresponding noisy true value label is Y = {y1, y2, y3……y N}. The predicted label of the sample after the teacher model is Y' = {y'1, y'2, y'3……y' N}, and the soft label Y'' = {y''1, y''2, y''3……y'' N} after mixing with the proportion α is:

[0089] Y'' = αY + (1-α)Y'

[0090] We will use Y'' as a new supervision signal to guide the training of the aircraft feature extraction model. As Figure 6 shown, we will use the parameters of the pre-trained model as the initial parameters of the student model and the teacher model, and then update the student model parameters.

[0091] This step contains two parts, the first part is the loss function L of the aircraft fine-grained image recognition task task .

[0092] The second part is the collaborative consistency loss L of the student model prediction label and the teacher model prediction label consistency , which makes the prediction result Y of the student network S and the prediction result Y of the teacher network as similar as possible, theoretically, the prediction label should not have too much jitter regardless of the sample.

[0093] Set the input sample set X = {x1, x2, x3……x M}, the calculation method of the loss function L is:

[0094] L = λL task +(1-λ)L consistency

[0095]

[0096] Wherein, λ is the coordination parameter of task loss and collaborative consistency loss.

[0097] In a feasible embodiment, the present application is developed based on the python language, pytorch framework or PaddlePaddle framework.

[0098] The embodiment of the low-altitude small target recognition device under high interference condition can be applied to any device with data processing capability, which can be a device or apparatus such as a computer. The device embodiment can be realized by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logical device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory and running through the processor of the device with data processing capability. From the hardware level, as shown in Figure 7 , it is a hardware structure diagram of the device with data processing capability of the low-altitude small target recognition device under high interference condition, in addition to the processor, memory, network interface and non-volatile memory shown in Figure 7 , the device with data processing capability in the embodiment usually includes other hardware according to the actual function of the device with data processing capability, which will not be repeated here. The implementation process of the functions and roles of each unit in the above device is specifically described in the implementation process of the corresponding steps in the above method, which will not be repeated here.

[0099] For the device embodiment, since it basically corresponds to the method embodiment, the relevant part can be seen from the part of the method embodiment. The device embodiment described above is only illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place or distributed to multiple network units. Part or all of the modules can be selected to achieve the purpose of the present application according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0100] The embodiment of the present application also provides a computer readable storage medium, which stores a program, and the program is executed by a processor to realize the low-altitude small target recognition method in a high interference condition in the above embodiment.

[0101] The computer readable storage medium can be an internal storage unit of any data processing device in the above embodiment, such as a hard disk or a memory. The computer readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of any data processing device. The computer readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0102] It should be noted that: the above description is only the preferred embodiment of the present application, and is not used to limit the present application, any modification, equivalent replacement, improvement, etc. within the principle of the present application, should be included in the protection scope of the present application.

Claims

1. A method for identifying low-altitude small targets under high interference conditions, characterized in that, This includes single-stage deep networks, which serve as the basic detection architecture. Transfer learning and attention enhancement mechanisms are used to improve the network's feature extraction capabilities and robustness. The process includes the following steps: The input image is acquired, and a single-stage deep network is used as the detection network for target detection. Feature extraction is performed on the input image through a feature enhancement module and a spatial awareness module. By using spatial attention and channel attention mechanisms, channels of similar information are summarized and organized to distinguish similar images; The convolutional neural network backbone architecture CSPDarknet53 cross-stage local fusion network is adopted as the backbone network to perform channel pruning, thereby accelerating the model and performing lightweight processing of image features. By continuously iterating the student and teacher models in the knowledge transfer network, soft labels guided by the collaborative consistency of prediction labels are constructed during the training process, which further improves the accuracy of the pruned model.

2. The method according to claim 1, characterized in that, The process involves acquiring an input image, using a single-stage deep network as the detection network for target detection, and extracting features from the input image through a feature enhancement module and a spatial awareness module. The feature enhancement module (FEM) is used to enrich semantic information, while the spatial awareness module (SAM) introduces deformable convolution and adaptive spatial learning techniques to capture more discriminative features of small targets.

3. The method according to claim 1, characterized in that, The method of using a single-stage deep network as the detection network for target detection, and extracting features from the input image through a feature enhancement module and a spatial awareness module, includes the following sub-steps: Based on pre-defined anchor points, features of the image at several different scales are extracted using a convolutional neural network; Predict the offset of the detection box and anchor point coordinates in four dimensions, as well as the confidence and classification score of the target in the box; Non-maximum suppression is used to denoise and deduplicate predictions with low confidence and large overlap areas to obtain the final detection results.

4. The method according to claim 1, characterized in that, The channel pruning also includes the following: channel pruning is performed by scaling and controlling the model depth and the number of convolutional kernels in each layer, thereby compressing the number of model parameters.

5. The method according to claim 1, characterized in that, The student network and teacher network in the knowledge transfer network include the following: First, train the teacher network to achieve the preset performance threshold; By fitting the teacher network to the student network, the knowledge learned by the teacher network can be transferred to the student network. Knowledge distillation of the student network is trained by using the output of the normalized exponential function layer of the teacher network.

6. The method according to claim 1, characterized in that, The spatial attention mechanism and channel attention mechanism include a method based on a dual-channel attention mechanism. The spatial attention mechanism focuses the viewpoint on a more discriminative region, while the channel attention mechanism allows the model to integrate and summarize channels containing similar semantic information, making the classification results more distinguishable.

7. The method according to claim 6, characterized in that, The method of summarizing and organizing channels of similar information through spatial attention and channel attention mechanisms also includes the following: spatial attention modules and channel attention modules are integrated into the network in the form of residuals.

8. The method according to claim 1, characterized in that, The method of continuously iterating the student and teacher models in the knowledge transfer network and constructing soft labels guided by the collaborative consistency of prediction labels during training to further improve the accuracy of the pruned model includes the following: Define a student model and a teacher model, where the teacher model is an ensemble of multiple prediction model parameters from the student model; In several iterations, the teacher model parameters are continuously updated, and a loss function is constructed to assess the collaborative consistency of the student model and the teacher model in predicting labels. Given an input sample set, and based on the input sample set and its noisy ground truth labels, obtain the corresponding predicted labels after passing through the teacher model; The predicted labels and ground truth labels are mixed in a preset ratio to construct soft labels for aircraft image samples; The soft labels are used as supervisory signals to guide the training of the aircraft feature extraction model, and the parameters of the student model are updated accordingly.

9. The method according to claim 8, characterized in that, The training of the aircraft feature extraction model is guided by soft labels as supervision signals, and the parameters of the student model are updated accordingly. To ensure the stability of the prediction, the loss function is also calculated, which includes the following sub-steps: Calculate the loss for aircraft-based fine-grained image recognition tasks; Calculate the collaborative consistency loss between the student model's predicted labels and the teacher model's predicted labels; Based on the fine-grained image recognition task loss and the collaborative consistency loss, and combining the coordination parameters of the task loss and the collaborative consistency loss, the loss function is calculated and obtained.

Images