Small target detection method based on improved RT-DETR

Abstract

The invention belongs to the technical field of computer vision, and discloses a small target detection method based on improved RT-DETR, and the method comprises the steps: employing public data / self-built data to make a data set according to a small target task; expanding the data set by adopting a data enhancement method, and dividing the data set into a training set, a test set and a verification set; a small target recognition model based on the improved RT-DETR is constructed, deformable convolution is adopted, an attention mechanism is introduced, the relation of all positions of a target is fully considered, a target area with important information is focused on, and other irrelevant feature information is inhibited; training the constructed model by using the training set and the verification set to obtain a trained detection model; and detecting a small target to be detected by using the trained detection model. According to the invention, the network is reset according to the specific features of the small target, the precision of small target detection is improved, and the balance between the detection processing speed and precision under a complex background is realized.

Description

Technical Field

[0001] The invention relates to the technical field of computer vision, and in particular to a small target detection method based on improved RT-DETR. Background Art

[0002] As one of the core research directions in the field of computer vision, the basic principle of target detection is to find and locate the target object of interest in the input image through image processing and machine learning technology. As a difficult and hot spot of target detection, small target detection has important application value in many fields such as ship detection, environmental monitoring, remote sensing image processing, automatic driving, medical diagnosis, face recognition, military reconnaissance, etc.

[0003] The definition of small targets has different interpretations in different scenarios. Currently, there are two types of small target definitions. One is the absolute size definition, which defines small targets based on the absolute pixel size of the target. Taking the commonly used dataset COCO as an example, objects with targets smaller than 32×32 pixels in the image are absolutely small targets; the other is the relative definition, where the target object is smaller than 10% of the image size or even smaller, which is relatively small. Due to the diversity and variability of the size of small targets, the number of pixels they occupy in a video or image is relatively small. The small size contains less information in the image, which easily causes the target to be blurred and the details to be unclear, thus restricting the further development of small target detection performance.

[0004] Target detection technology is mainly divided into two categories: traditional target detection algorithms and deep learning-based target detection algorithms. The former mainly relies on traditional feature extractors to extract image features and uses sliding windows to generate a large number of target candidate regions. This method has cumbersome steps and serious window redundancy problems. It also has problems such as slow detection speed and low detection accuracy. With the development of deep learning, researchers have applied network models such as convolutional neural networks (CNN) and RT-DETR (Real-Time DEtection TRansformer) to target detection. Target detection algorithms have made major breakthroughs, but there are still challenges in detecting small targets with low resolution of the recognition area, blurred images, and low signal-to-noise ratio. Taking complex sea surface targets as an example, under the background of complex sea conditions in multiple scenes, the characteristics of sea surface targets are mainly affected by environmental factors such as waves, sea fog, and light reflection on the water surface. The main features of sea surface targets include a small number of pixels in the image, large changes in target scale, weak texture information, and few available features. When the target size is too small, the detection algorithm may find it difficult to accurately identify and locate the target, resulting in missed detection; due to the similarity between small targets and backgrounds or other objects, false detections are also prone to occur. In addition, in practical applications, the limitation of hardware equipment is also an important factor affecting the effect of small target detection.

[0005] Many researchers have introduced the attention mechanism into the small target detection network to improve the detection performance of the network. The attention mechanism mainly sets a weight for the features of the target area, so that the network can focus on the target area with important information during the feature extraction process, while suppressing other irrelevant feature information and reducing the interference of irrelevant background on the detection results. The adoption of the attention mechanism will enable the model to pay more attention to the small target area and improve the detection accuracy of small targets, but it is not a lightweight network structure. The parameters and calculation amount of the model increase and the detection speed slows down. Therefore, how to balance the processing speed and accuracy of the small target detection model based on the attention mechanism is an urgent problem to be solved. Summary of the invention

[0006] The technical problem to be solved by the present invention is to overcome the above technical defects and provide a small target detection method based on improved RT-DETR, which can maintain high accuracy while taking into account high efficiency, and achieves a balance between detection processing speed and precision under complex backgrounds.

[0007] In order to solve the above problems, the technical solution of the present invention is to provide a small target detection method based on improved RT-DETR, comprising the following steps:

[0008] S1, make dataset, the dataset source is public dataset / self-built dataset, the self-built dataset is obtained by collecting small target datasets in different scene environments and manually annotating them.

[0009] S2, expand the data set, use data enhancement methods to enhance the data to form an expanded data set, and divide the expanded data set into training set, test set and validation set according to a certain ratio.

[0010] S3, build a small target recognition model based on the improved RT-DETR, which includes a backbone network, a hybrid encoder and a decoder with an auxiliary prediction head;

[0011] S31, the backbone network uses the A-DCN (Attention Deformable Convolutional Networks) feature extraction network module, and introduces a global attention mechanism to improve the offset of deformable convolution; A-DCN adds an attention mechanism to improve the deformable convolution;

[0012] S32, the features of the last three stages of the backbone network are input to the encoder;

[0013] S33, the hybrid encoder converts multi-scale features into image feature sequences through the intra-scale feature interaction module (AIFI) and the multi-scale spatial feature pyramid network (Muti-FPN);

[0014] S34, IoU-aware query selection selects a fixed number of features from the feature sequence output by the encoder as the initial target query for the decoder;

[0015] S35, the decoder iteratively optimizes the target query through the auxiliary prediction head to generate a bounding box and a confidence score.

[0016] S4, training the target detection model, using the training set and the validation set to train the constructed model to obtain the trained detection model.

[0017] S5, detecting small targets to be detected, using the trained detection model to detect small targets to be detected.

[0018] Furthermore, the test set described in step S1 is used to test the model operating environment before building the model.

[0019] Furthermore, the data enhancement method in step S2 includes but is not limited to supervised data enhancement methods such as Mosaic, Mixup, geometric transformation, color transformation, and unsupervised data enhancement methods such as GAN and Rand Augment.

[0020] Furthermore, in step S31, the A-DCN adds an attention mechanism to improve the variable convolution step as follows:

[0021] 1) By defining a new mask network in the original convolution layer, the mask network is used to generate the offsets and masks required in the convolution operation. These offsets and masks are used to control the sampling position and weight of the convolution kernel on the input feature map during the convolution process;

[0022] 2) The generated offset and mask are passed to the attention mechanism module. The attention mechanism adopts a hybrid attention module (CBAM), which adjusts the values ​​of the offset and mask so that when performing convolution operations, more attention is paid to important feature areas and irrelevant noise or redundant information is suppressed;

[0023] 3) Improve the representation ability of deformable convolution by mixing the offset and mask through the attention mechanism.

[0024] Furthermore, the intra-scale feature interaction module (AIFI) in step S33 processes the feature map, combines it with multi-head self-attention after changing the vector dimension, and finally corrects the obtained output to two dimensions as the return amount for cross-scale feature fusion.

[0025] Furthermore, the multi-scale spatial feature pyramid network (Muti-FPN) feature fusion network in step S33 resets the network according to the unique features of small targets, screens the feature vectors output by the intra-scale feature interaction module (AIFI) and the two different scale feature maps after passing through the DCN network, and extracts important information in each channel through pooling operations and weight calculations.

[0026] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described above.

[0027] The advantages of the present invention compared with the prior art are:

[0028] 1. The present invention reduces the number of overall algorithm parameters, while improving the effectiveness of the algorithm, achieving a balance between detection processing speed and accuracy under complex backgrounds. The present invention introduces an attention mechanism, fully considering the connection between various positions of the target, so that the network can focus on the target area with important information during the feature extraction process, while also suppressing other irrelevant feature information, reducing the interference of irrelevant background on the detection results, and improving the detection performance of the target detection network.

[0029] 2. The present invention adopts deformable convolution (Deformable Convolutional Networks), which allows the convolution kernel to flexibly move and adjust the shape on the input feature map by introducing offsets and masks, and can better adapt to the shape and position changes of small targets. A multi-scale spatial feature pyramid network structure is adopted, and the network is reset according to the unique features of small targets. The high-level feature map is used as a weight to filter low-level features. The filtered features are integrated with high-level features, enriching the low-level features, injecting important semantic information, and enhancing the representation ability of small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of the present invention.

[0031] Figure 2 It is a target detection network structure diagram of the present invention. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0033] like Figure 1 , Figure 2 As shown, the present invention discloses a small target detection method based on improved RT-DETR, comprising the following steps:

[0034] S1, make a dataset. The dataset comes from public datasets / self-built datasets. The self-built dataset is obtained by collecting small target datasets in different scene environments and manually annotating them.

[0035] S2, expand the data set, use data enhancement methods to enhance the data to form an expanded data set, and divide the expanded data set into training set, test set and validation set according to a certain ratio.

[0036] S3, build a small target recognition model based on the improved RT-DETR, which includes a backbone network, a hybrid encoder and a decoder with an auxiliary prediction head.

[0037] S4, training the target detection model. Use the training set and the validation set to train the constructed model to obtain a trained detection model.

[0038] S5, detection of small targets to be detected. The trained detection model is used to detect the small targets to be detected.

[0039] Furthermore, the test set described in step S1 is used to test the model operating environment before building the model.

[0040] Furthermore, the data enhancement method in step S2 includes but is not limited to supervised data enhancement such as Mosaic, Mixup, geometric transformation, color transformation, and unsupervised data enhancement methods such as generating new data GAN network and learning enhancement network Rand Augment.

[0041] Furthermore, the steps of constructing a small target recognition model based on improved RT-DETR in step S3 are as follows:

[0042] S31, the backbone network uses the A-DCN (Attention Deformable Convolutional Networks) feature extraction network module, and introduces a global attention mechanism to improve the offset of deformable convolution; A-DCN adds an attention mechanism to improve the deformable convolution;

[0043] S32, the features of the last three stages of the backbone network are input to the encoder;

[0044] S33, the hybrid encoder converts multi-scale features into image feature sequences through intra-scale feature interaction (AIFI) and multi-scale spatial feature pyramid network Muti-FPN;

[0045] S34, IoU-aware query selection selects a fixed number of features from the feature sequence output by the encoder as the initial target query for the decoder

[0046] S35, the decoder iteratively optimizes the target query through the auxiliary prediction head to generate a bounding box and a confidence score.

[0047] Furthermore, in step S31, the A-DCN innovatively adds an attention mechanism to improve the variable convolution step:

[0048] 1) Define a new mask network in the original convolution layer to generate the offset and mask required for the convolution operation. The offset and mask are used to control the sampling position and weight of the convolution kernel on the input feature map during the convolution process.

[0049] 2) The generated offset and mask are passed to the attention mechanism module. The attention mechanism adopts a hybrid attention module (CBAM), which adjusts the values ​​of the offset and mask, focuses on important feature areas, and suppresses irrelevant noise or redundant information.

[0050] 3) Improve the representation ability of deformable convolution by mixing the offset and mask through the attention mechanism.

[0051] When performing deformable convolution, not only can the shape changes of the input data be taken into account, but also the information at certain locations can be emphasized or suppressed according to the attention mask. The algorithm formula is as follows:

[0052]

[0053]

[0054]

[0055] Among them, the input feature map x, each position p on the output feature map y, G is the total number of aggregation groups, and for the g-th group, w g represents the projection weight of the group, m gk represents the modulation parameter of the kth sampling point in the gth group, and Δp gk represents the sampling position p k The corresponding offset.

[0056] Furthermore, the intra-scale feature interaction AIFI module in step S33 processes the feature map, combines it with multi-head self-attention after changing the vector dimension, and finally corrects the obtained output to two dimensions as the return amount for cross-scale feature fusion.

[0057] Furthermore, the Muti-FPN feature fusion network described in step S33 resets the network according to the specific features of small targets, screens the feature vector output by AIFI and the two different scale feature maps after passing through the DCN network, and effectively extracts the important information in each channel through pooling operations and weight calculations.

[0058] The embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer executable instructions are executed by a computer processor, the computer executable instructions are used to execute a small target detection method based on improved RT-DETR, the method comprising:

[0059] Create data sets, expand data sets, build small target recognition models based on improved RT-DETR, train target detection models, and detect small targets to be detected.

[0060] Through the above description of the implementation method, the technicians in the relevant field can clearly understand that the present application can be implemented with the help of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application can be essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0061] It is worth noting that in the embodiment of the above-mentioned data storage device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0062] The undisclosed parts in the present invention are all prior art, and their specific structures and working principles are not described in detail.

[0063] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0064] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0065] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A small target detection method based on improved RT-DETR, characterized in that: The following steps are involved: S1, make a dataset. The dataset comes from a public dataset or a self-built dataset. The self-built dataset is obtained by collecting small target datasets in different scene environments and manually annotating them. S2, expand the data set, use data enhancement methods to enhance the data to form an expanded data set, and divide the expanded data set into training set, test set and validation set according to a certain ratio; S3, constructing a small target recognition model based on the improved RT-DETR, wherein the small target recognition model includes a backbone network, a hybrid encoder, and a decoder with an auxiliary prediction head, and the steps for constructing the small target recognition model are as follows: S31, the backbone network adopts the A-DCN feature extraction network module and introduces a global attention mechanism to improve the offset of deformable convolution; A-DCN adds an attention mechanism to improve the deformable convolution; S32, the features of the last three stages of the backbone network are input to the encoder; S33, the hybrid encoder converts multi-scale features into image feature sequences through an intra-scale feature interaction module and a multi-scale spatial feature pyramid network; S34, IoU-aware query selection selects a fixed number of features from the feature sequence output by the encoder as the initial target query for the decoder; S35, the decoder iteratively optimizes the target query through the auxiliary prediction head to generate a bounding box and a confidence score; S4, training the target detection model, using the training set and the validation set to train the constructed model to obtain a trained detection model; S5, detecting small targets to be detected, using the trained detection model to detect small targets to be detected.

2. A small target detection method based on improved RT-DETR according to claim 1, characterized in that: The test set described in step S1 is used to test the model running environment before building the model.

3. A small target detection method based on improved RT-DETR according to claim 1, characterized in that: The data enhancement method described in step S2 includes supervised data enhancement methods and unsupervised data enhancement methods. The supervised data enhancement methods include Mosaic, Mixup, geometric transformation, and color transformation. The unsupervised data enhancement methods include GAN and RandAugment.

4. A small target detection method based on improved RT-DETR according to claim 1, characterized in that: The A-DCN described in step S31 adds an attention mechanism to improve the variable convolution step as follows: 1) By defining a new mask network in the original convolution layer, the mask network is used to generate the offsets and masks required in the convolution operation. These offsets and masks are used to control the sampling position and weight of the convolution kernel on the input feature map during the convolution process; 2) The generated offset and mask are passed to the attention mechanism module. The attention mechanism module adopts a hybrid attention module, and the attention mechanism module adjusts the values ​​of the offset and mask so that when performing convolution operations, it pays more attention to important feature areas and suppresses irrelevant noise or redundant information; 3) Improve the representation ability of deformable convolution by mixing the offset and mask through the attention mechanism.

5. A small target detection method based on improved RT-DETR according to claim 4, characterized in that: The A-DCN innovatively adds an attention mechanism to improve the algorithm formula of the variable convolution: Among them, the input feature map x, each position p on the output feature map y, G is the total number of aggregation groups, and for the g-th group, w g represents the projection weight of the group, m gk represents the modulation parameter of the kth sampling point in the gth group, Δp gk represents the sampling position p k The corresponding offset.

6. A small target detection method based on improved RT-DETR according to claim 1, characterized in that: The intra-scale feature interaction module in step S33 processes the feature map, combines it with multi-head self-attention after changing the vector dimension, and finally corrects the obtained output to two dimensions as the return amount for cross-scale feature fusion.

7. The small target detection method based on improved RT-DETR according to claim 1, characterized in that: The multi-scale spatial feature pyramid network in step S33 resets the network according to the unique features of small targets, screens the feature vectors output by the intra-scale feature interaction module and the two different scale feature maps after passing through the DCN network, and extracts important information in each channel through pooling operations and weight calculations.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a small target detection method based on improved RT-DETR as described in any one of claims 1 to 7 is implemented.

Images