A Small Target Detection Method and System Based on Deformable Recurrent Neural Networks
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-10
- Publication Date
- 2026-08-14
AI Technical Summary
然而，现有RWKV类方法中常用的偏移操作大多采用预定义的规则偏移或静态采样方式，其有效感受野固定，难以根据图像内容、目标形状及空间分布进行自适应调整
[0052]（1）本发明在可变形循环神经网络编码单元中，通过可变形偏移单元依据输入特征内容为各空间位置动态预测采样偏移，使采样位置能够随目标形状和上下文信息自适应变化，从而克服了固定感受野的限制。在空间维度上，可变形空间混合通过动态调整采样位置，使有效感受野能够灵活覆盖不同尺度和形态的目标区域，并结合双向WKV上下文建模建立远程依赖关系，使模型在保持线性计算复杂度的前提下实现全局上下文与局部关键特征的协同增强；在通道维度上，可变形通道混合通过内容相关的通道重排与门控调制，增强了特征通道之间的非线性耦合与判别性表达。可变形空间混合与可变形通道混合协同作用，使编码器输出的多尺度特征在目标边缘、纹理等细节信息和语义信息两个维度上均得到显著增强，为后续的多尺度融合提供了高质量的特征基础。
Smart Images
Figure CN122574597A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and specifically to a method and system for small target detection based on a deformable recurrent neural network. Background Technology
[0002] In the fields of computer vision and intelligent image processing, object detection is one of the core tasks. Among them, small object detection is particularly challenging because the target pixel area is small, the available texture and shape features are extremely scarce, and it is easily affected by factors such as complex backgrounds, lighting changes, and occlusion, resulting in weak effective feature signals and poor separation between the target and the background. At the same time, small objects are highly sensitive to scale changes and geometric deformations, and the bounding box regression accuracy requirements are stringent. Even slight positioning deviations can lead to detection failure, and its detection accuracy is far lower than that of conventionally sized targets.
[0003] Deep learning-based object detection methods, particularly those based on convolutional neural networks and the Transformer architecture, have achieved significant results in general object detection tasks. Classic detection frameworks such as Faster R-CNN (Faster Region-based Convolutional Neural Network) and SSD (SingleShot MultiBox Detector) have validated the effectiveness of deep semantic features for object localization and recognition. Researchers have focused on small object detection tasks, making targeted improvements to general object detection through data augmentation, context utilization, and multi-scale feature modeling. For example, copy-paste strategies have been used to increase the frequency of small objects in training samples; Feature Pyramid Networks (FPNs) use a top-down path to pass high-level semantic information to shallow high-resolution features, enhancing multi-scale representation capabilities. Furthermore, some methods have explored generative adversarial learning, dilated convolutions, center point prediction, or high-resolution feature expansion techniques to further improve small object detection performance.
[0004] Despite the progress made by the aforementioned methods for small target detection, significant shortcomings remain in practical applications. First, existing multi-scale feature fusion methods often rely on fixed interpolation, simple concatenation, or element-wise addition to achieve cross-scale interaction. These methods struggle to effectively eliminate spatial mismatches between features at different levels, resulting in insufficient fusion of high-level semantic information and shallow detail information, failing to form the high-quality joint representation crucial for small target localization and recognition. Second, while Transformer-based multi-scale modeling methods possess powerful global dependency modeling capabilities, their computational complexity typically increases quadratically with sequence length. When processing high-resolution inputs and multi-scale features, this leads to extremely high computational and memory overhead, limiting their deployment in practical applications.
[0005] To balance global context modeling efficiency with local feature extraction capabilities, linear complexity modeling mechanisms based on RWKV (Receptance-Weighted Key Value) offer a new technical approach for vision tasks. Vision-RWKV, Restore-RWKV, and related improvements enhance the interactivity of local spatial information to some extent by introducing different forms of offset operations into the network structure. However, most commonly used offset operations in existing RWKV-like methods employ predefined rule offsets or static sampling, resulting in a fixed effective receptive field that is difficult to adaptively adjust based on image content, target shape, and spatial distribution. When processing small or deformed targets, this fixed offset approach easily introduces background interference or misses crucial details, thus limiting further improvements in model detection accuracy and robustness.
[0006] Therefore, there is an urgent need in this field for a small target detection method that can take into account dynamic fusion of multi-scale features, content-adaptive spatial modeling, and low computational complexity. Summary of the Invention
[0007] To address the shortcomings of existing technologies, this invention provides a small target detection method and system based on deformable recurrent neural networks. Based on an end-to-end detection framework, this invention introduces a deformable recurrent neural network encoder in the encoding stage to achieve content-related modeling of target shape changes, scale changes, and contextual relationships. Simultaneously, a dynamic multi-scale feature fusion mechanism is introduced in the multi-scale feature integration stage to enhance the spatial alignment and information complementarity between features at different levels, thereby improving the accuracy and robustness of small target detection in complex scenes.
[0008] To achieve the above objectives, the present invention adopts the following technical solution:
[0009] This invention proposes a small target detection method based on a deformable recurrent neural network, comprising the following steps:
[0010] The image to be detected is acquired, and the image to be detected is preprocessed to obtain an image tensor;
[0011] The image tensor is input into a pre-trained target detection neural network, which performs target detection on the image to be detected and outputs the category label and bounding box coordinates of the detected targets.
[0012] The target detection neural network includes a backbone network, a deformable recurrent neural network encoder, a decoder, and a detection head connected in sequence.
[0013] The backbone network is used to extract hierarchical features at multiple different scales from the image tensor;
[0014] The deformable recurrent neural network encoder includes a dynamic fusion module and multiple deformable recurrent neural network encoding units. Each of the multiple deformable recurrent neural network encoding units corresponds one-to-one with multiple hierarchical features of different scales. The multiple deformable recurrent neural network encoding units are used to perform deformable spatial mixing processing and deformable channel mixing processing on the hierarchical features of the corresponding scales, and output multi-scale encoded features. The dynamic fusion module is used to perform adaptive alignment and fusion on the multi-scale encoded features to obtain fused enhanced features.
[0015] The decoder is used to perform target query interaction based on the fused enhanced features and output a target representation vector;
[0016] The detection head is used to generate the category label and bounding box coordinates based on the target representation vector.
[0017] Furthermore, the processing of hierarchical features at the corresponding scale by each deformable recurrent neural network encoding unit includes:
[0018] Group normalization is performed on the hierarchical features at the corresponding scale to obtain group-normalized hierarchical features;
[0019] The group-normalized hierarchical features are subjected to deformable spatial blending to obtain spatial blending features. The deformable spatial blending is used to establish long-range dependencies between different positions in the group-normalized hierarchical features in the spatial dimension, and to dynamically generate sampling offsets for each position of the group-normalized hierarchical features to adjust the receptive field.
[0020] The spatial mixing features are residually connected with the hierarchical features at the corresponding scale to obtain residual connection features;
[0021] The residual connectivity features are subjected to group normalization to obtain group-normalized residual connectivity features.
[0022] The group-normalized residual connection features are subjected to deformable channel mixing processing to obtain channel mixing features. The deformable channel mixing processing is used to sequentially perform recombination, mapping and gating modulation on the group-normalized residual connection features in the channel dimension to improve the discriminativeness of the channel mixing features.
[0023] Furthermore, the process of deformable space mixing includes:
[0024] The group-normalized hierarchical features are input into the deformable offset unit to predict the sampling offset corresponding to each reference position in the group-normalized hierarchical features.
[0025] Based on the sampling offset, deformable sampling is performed on the group-normalized hierarchical features to obtain spatial rearrangement features;
[0026] The spatial rearrangement features are linearly mapped to generate a first receiving feature, a first key feature, and a first value feature.
[0027] The first key feature and the first value feature are input into a bidirectional WKV (Weighted Key Value) attention unit to obtain the context response result;
[0028] The context response result is gated and modulated using the first receiving feature.
[0029] Furthermore, the processing procedure of the deformable channel mixing module includes:
[0030] The group-normalized residual connection feature is input into the deformable offset unit to predict the sampling offset corresponding to each reference position in the group-normalized residual connection feature.
[0031] Based on the sampling offset, the group normalized residual connectivity features are rearranged and sampled to obtain the channel rearranged features;
[0032] Linear mapping is performed on the channel rearrangement features to generate a second reception feature and a second key feature;
[0033] The second key feature is nonlinearly activated to obtain the activated key feature, and the second value feature is generated based on the activated key feature.
[0034] The second value feature is gated and modulated using the second received feature.
[0035] Furthermore, the deformable offset unit adopts a multi-head deformable offset structure, and the process of the deformable offset unit predicting the sampling offset is as follows:
[0036] For each reference position, multiple offset heads predict the offset of multiple sampling points to determine the actual sampling position of each sampling point.
[0037] Based on the actual sampling location of each sampling point, the features of each sampling point are obtained through bilinear interpolation.
[0038] Predict the sampling weights corresponding to each sampling point;
[0039] The sampling weights are normalized to obtain normalized sampling weights;
[0040] The features of each sampling point are weighted and aggregated according to the normalized sampling weights to obtain the sampling offset corresponding to the reference position.
[0041] Furthermore, the dynamic fusion module includes a top-down fusion path and a bottom-up fusion path. In the top-down fusion path, the deformable feature alignment fusion module adaptively aligns and fuses the encoded features at different scales step by step. The execution process of the deformable feature alignment fusion module includes:
[0042] Dynamic sampling enhancement is performed on the high-level encoded features to obtain enhanced high-level features;
[0043] The enhanced high-level features are channel-expanded and upsampled by pixel rearrangement to obtain the upsampled enhanced high-level features.
[0044] Channel attention and spatial attention are applied to the low-level encoded features to obtain enhanced low-level features; the enhanced low-level features are added to the low-level encoded features to obtain residual enhanced low-level features.
[0045] After the residual enhanced low-level features are spliced together with the upsampled enhanced high-level features, the output of the deformable feature alignment fusion module is obtained through fusion layer processing.
[0046] Furthermore, the target detection network is an end-to-end detector based on the Transformer architecture.
[0047] This invention also proposes a small target detection system based on a deformable recurrent neural network to implement the above-mentioned small target detection method, comprising:
[0048] The image acquisition module is used to acquire the image to be detected;
[0049] The preprocessing module is used to preprocess the image to be detected to obtain an image tensor;
[0050] The object detection module includes a pre-trained object detection neural network, which is used to perform object detection on the image tensor and output the category labels and bounding box coordinates of all detected objects.
[0051] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0052] (1) In the deformable recurrent neural network encoding unit, the present invention uses a deformable offset unit to dynamically predict the sampling offset for each spatial position based on the input feature content, so that the sampling position can adapt to the target shape and context information, thereby overcoming the limitation of a fixed receptive field. In the spatial dimension, deformable spatial mixing dynamically adjusts the sampling position, so that the effective receptive field can flexibly cover target regions of different scales and shapes, and establishes long-range dependencies by combining bidirectional WKV context modeling, so that the model can achieve synergistic enhancement of global context and local key features while maintaining linear computational complexity. In the channel dimension, deformable channel mixing enhances the nonlinear coupling and discriminative expression between feature channels through content-related channel rearrangement and gating modulation. The synergistic effect of deformable spatial mixing and deformable channel mixing significantly enhances the multi-scale features output by the encoder in both the dimensions of target edge, texture and other detailed information and semantic information, providing a high-quality feature foundation for subsequent multi-scale fusion.
[0053] (2) In the dynamic fusion module, the present invention performs content-related deformable sampling on high-level features through a dynamic sampling enhancement unit, replacing fixed bilinear interpolation with learnable sampling positions, and combining pixel rearrangement upsampling to achieve accurate restoration of the spatial resolution of high-level features to low-level features, effectively avoiding edge blurring and spatial misalignment caused by fixed interpolation. At the same time, the low-level features rich in detail information are enhanced through a dual screening mechanism of channel attention and spatial attention, so that the high-level semantic information and the low-level detail information are accurately matched in both spatial and channel dimensions. On this basis, the spatially aligned high-level features and the screened and enhanced low-level features are channel-seam-fused, so that the fused enhanced features have both high-resolution spatial localization capability and strong semantic discrimination capability, thereby significantly improving the detection accuracy of small targets in complex backgrounds.
[0054] (3) This invention uses bidirectional WKV context modeling units to establish long-range dependencies, requiring only linear computational complexity, which significantly reduces the computational load and memory usage in high-resolution feature processing. Compared to the quadratic complexity of standard self-attention mechanisms, this invention can effectively handle high-resolution multi-scale features while maintaining global context modeling capabilities, balancing small target detection performance with computational efficiency in practical deployment. Attached Figure Description
[0055] Figure 1 This is a schematic diagram of the overall structure of the target detection neural network in an embodiment of the present invention;
[0056] Figure 2 This is a schematic diagram of the deformable space mixing process in an embodiment of the present invention;
[0057] Figure 3 This is a schematic diagram of the deformable channel mixing process in an embodiment of the present invention;
[0058] Figure 4 This is a schematic diagram of the deformable feature alignment and fusion module in an embodiment of the present invention. Detailed Implementation
[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0060] Example
[0061] This embodiment proposes a small target detection method based on a deformable recurrent neural network, including the following steps:
[0062] Step 1: Obtain the image to be detected and preprocess the obtained image.
[0063] The image to be detected can be a picture taken by a surveillance camera or a high-resolution optical remote sensing image.
[0064] The preprocessing of the image to be detected includes size normalization and standardization. In this embodiment, size normalization scales the input image to 1200×1200 pixels. Standardization is then performed to map the pixel value range from [0,255] to [0,1], so that the distribution of the image data conforms to the input requirements of the target detection neural network, thereby obtaining the image tensor.
[0065] Step 2: Input the image tensor into the pre-trained target detection neural network, perform target detection on the target image through the target detection neural network, and output the category labels and bounding box coordinates of all detected targets.
[0066] refer to Figure 1 The target detection neural network is based on the Transformer architecture and includes a backbone network, a deformable recurrent neural network encoder, a decoder, and a detection head connected in sequence.
[0067] The backbone network is used to extract multi-scale hierarchical features from the input image tensor. In this embodiment, the backbone network uses the backbone feature extractor in RT-DETR (Real-Time Detection Transformer) to extract multiple hierarchical features of different scales from the input image, including shallow features, mid-level features, and high-level semantic features. Shallow features have high resolution and contain rich edge and texture details, while high-level semantic features have lower resolution and strong semantic information. The resolution and semantic information of mid-level features are between those of shallow and high-level features. In this embodiment, the backbone network extracts three hierarchical features of different scales from the input image tensor, denoted as C3, C4, and C5, where C3 is the shallow feature, C4 is the mid-level feature, and C5 is the high-level semantic feature.
[0068] The deformable recurrent neural network encoder includes a dynamic fusion module and multiple deformable recurrent neural network encoding units. Each of the multiple deformable recurrent neural network encoding units corresponds one-to-one with multiple hierarchical features of different scales. The multiple deformable recurrent neural network encoding units are used to perform deformable spatial mixing processing and deformable channel mixing processing on the hierarchical features of the corresponding scales, and output multi-scale encoded features. The dynamic fusion module is used to perform adaptive alignment and fusion on the multi-scale encoded features to obtain fused enhanced features.
[0069] The dynamic fusion module includes a top-down fusion path and a bottom-up fusion path. In the top-down fusion path, the deformable feature alignment fusion module adaptively aligns and fuses the encoded features at different scales step by step.
[0070] In this embodiment, C3, C4, and C5 are respectively subjected to 1×1 convolution for channel adjustment, resulting in F3, F4, and F5, each with 256 channels. F3, F4, and F5 are processed by their respective deformable recurrent neural network coding units to obtain three coding features B3, B4, and B5 at different scales. The dynamic fusion module receives B3, B4, and B5 and performs adaptive alignment and fusion on them. In the top-down fusion path, B5 and B4 are adaptively aligned and fused by the first deformable feature alignment and fusion module to obtain fused feature S45. Fusion feature S45 and B3 are adaptively aligned and fused by the second deformable feature alignment and fusion module to obtain enhanced feature Z3. In the bottom-up fusion path, Z3 is downsampled and then fused with S45 through element-wise addition or channel concatenation to obtain enhanced feature Z4. Z4 is downsampled and then fused with B5 through element-wise addition or channel concatenation to obtain enhanced feature Z5. Z3, Z4, and Z5 are concatenated along the channels to obtain the fused enhanced feature.
[0071] The decoder receives the fused and enhanced features output by the deformable recurrent neural network encoder, performs target query and feature interaction through multi-head self-attention mechanism and cross-attention mechanism, gradually optimizes the target representation, and outputs the target representation vector; in this embodiment, the decoder adopts the Transformer decoder structure.
[0072] After processing the target representation vector output by the decoder, the prediction head outputs the category labels and bounding box coordinates of all detected targets.
[0073] The processing of hierarchical features at the corresponding scale by the deformable recurrent neural network encoding unit includes: performing group normalization on the hierarchical features at the corresponding scale to obtain group normalized hierarchical features; performing deformable spatial blending on the group normalized hierarchical features to obtain spatial blending features; the deformable spatial blending is used to establish long-range dependencies between different positions in the group normalized hierarchical features in the spatial dimension, and dynamically generating sampling offsets for each position of the group normalized hierarchical features to adjust the receptive field.
[0074] The process of performing deformable spatial blending on the group-normalized hierarchical features is as follows: The group-normalized hierarchical features are input into a deformable offset unit to predict the sampling offset corresponding to each reference position in the group-normalized hierarchical features; the actual sampling position corresponding to each reference position is determined based on the sampling offset corresponding to each reference position in the group-normalized hierarchical features; features of each sampling point are obtained from the normalized hierarchical features through bilinear interpolation to obtain spatial rearrangement features. Three parallel linear mappings are performed on the spatial rearrangement features to generate a first receiving feature R1, a first key feature K1, and a first value feature V1; the first key feature K1 and the first value feature V1 are input into a bidirectional WKV attention unit for bidirectional key value calculation to obtain the context response result; the first receiving feature R1 is activated by a sigmoid function to generate a first gating weight; the first gating weight is multiplied element-wise with the context response result to obtain a gated context response result; the gated context response result is then subjected to linear mapping and layer normalization processing sequentially to obtain spatial blending features.
[0075] The spatial mixing features are residually connected to the hierarchical features at the corresponding scale to obtain residual connection features;
[0076] Group normalization is applied to the residual connectivity features to obtain group-normalized residual connectivity features.
[0077] The group-normalized residual connection features are subjected to deformable channel mixing processing to obtain channel mixing features. The deformable channel mixing processing is used to sequentially perform recombination, mapping and gating modulation on the group-normalized residual connection features in the channel dimension to improve the discriminativeness of the channel mixing features.
[0078] The process of performing deformable channel mixing on the group-normalized residual connection features is as follows: The group-normalized residual connection features are input into a deformable offset unit to predict the sampling offset corresponding to each reference position in the group-normalized residual connection features; the actual sampling position corresponding to each reference position is determined based on the sampling offset corresponding to each reference position in the group-normalized residual connection features; the features of each sampling point are obtained from the group-normalized residual connection features through bilinear interpolation to obtain channel rearrangement features; two parallel linear mappings are performed on the channel rearrangement features to generate a second receiving feature R2 and a second key feature K2; the second key feature K2 is squared-activated through a squared-corrected linear unit to obtain an activated key feature; the activated key feature is linearly mapped to generate a second value feature V2; the second receiving feature R2 is activated by a sigmoid function to generate a second gate weight; the second gate weight is multiplied element-wise with the second value feature V2 to obtain a gated value feature; the gated value feature is sequentially linearly mapped and layer-normalized to obtain the channel mixing features.
[0079] By performing a residual connection between the channel mixing features and the residual connection features, the output of the deformable recurrent neural network coding unit is obtained, which is the coding feature at the corresponding scale.
[0080] In the process of deformable space hybrid processing and deformable channel hybrid processing, the deformable offset unit adopts a multi-head deformable offset structure. The process of the deformable offset unit predicting the sampling offset is as follows: the input features of the deformable offset unit are flattened to obtain a flattened feature sequence. For each reference position in the flattened feature sequence, multiple offset heads predict the offset of multiple sampling points in the sequence direction to determine the actual sampling position of each sampling point. Based on the actual sampling position of each sampling point, the features of each sampling point are obtained from the input features of the deformable offset unit through bilinear interpolation. The flattened feature sequence is linearly mapped to predict the sampling weight corresponding to each sampling point. The sampling weight is Softmax normalized in the sampling point dimension to obtain normalized sampling weights. The features of each sampling point are weighted and aggregated according to the normalized sampling weights to obtain the sampling offset corresponding to the reference position.
[0081] In the dynamic fusion module, each deformable feature alignment and fusion module adaptively aligns and fuses two encoded features at different scales. The specific process is as follows:
[0082] The high-level coding features are sequentially processed by 1×1 convolution, batch normalization and ReLU activation to obtain preprocessed features; dynamic sampling enhancement is performed on the preprocessed features to obtain enhanced high-level features; after channel expansion of the enhanced high-level features, upsampling is completed by pixel rearrangement to obtain upsampled enhanced high-level features, and the upsampled enhanced high-level features have the same spatial resolution as the low-level coding features.
[0083] Channel attention and spatial attention processing are performed on the low-level encoded features. Channel attention processing involves sequentially performing global average pooling and 1×1 convolution on the low-level encoded features to obtain low-level channel description vectors. These vectors are then activated using a sigmoid function to obtain channel attention weights. Spatial attention processing involves sequentially performing global average pooling, 1×1 convolution, ReLU activation, and 1×1 convolution on the low-level encoded features to obtain low-level spatial description vectors. These vectors are then activated using a sigmoid function to obtain spatial attention weights. The low-level encoded features, channel attention weights, and spatial attention weights are then multiplied element-wise to obtain enhanced low-level features. Finally, these enhanced low-level features are added element-wise to the low-level encoded features to obtain residual enhanced low-level features.
[0084] The residual-enhanced low-level features and the upsampled enhanced high-level features are concatenated along the channel dimension to obtain the concatenated fused features. The concatenated fused features are then subjected to 1×1 convolution fusion, batch normalization, and ReLU activation to obtain the output of the deformable feature alignment fusion module.
[0085] In this embodiment, the object detection neural network is trained and validated on the publicly available VisDrone (Vision Meets Drone) dataset. In practical applications, other datasets can be selected according to requirements. During training, the AdamW optimizer is used, with an initial learning rate set to... The weight decay coefficient is set to The total number of training epochs was set to 300. The total loss function was a weighted sum of the classification loss, bounding box regression loss, and generalized intersection-union (GUIR) loss. The classification loss used the cross-entropy loss function, the bounding box regression loss used the L1 loss function, and the GUIR loss was used to optimize the overlap between the predicted and ground truth boxes. The network parameters were iteratively updated using the backpropagation algorithm until the model converged.
[0086] To better illustrate the beneficial effects of the present invention, the target detection neural network (hereinafter referred to as the "invention model"), DINO (DETR with Improved deNoising anchOr boxes, DETR model based on improved denoising anchor box strategy), DDQ (Dense Distinct Query) model, RT-DETR, improved RT-DETR-1 and improved RT-DETR-2 in the above embodiments were respectively verified on the publicly available VisDrone dataset.
[0087] The difference between the improved RT-DETR-1 and RT-DETR is that each coding layer in the encoder of RT-DETR is replaced with a deformable recurrent neural network coding unit as described in the above embodiments.
[0088] The difference between the improved RT-DETR-2 and RT-DETR is that the fusion module in RT-DETR is replaced with the dynamic fusion module in the above embodiments.
[0089] Each model was validated under the same input size and training strategy. The evaluation metrics included: average accuracy of all targets (mAP), average accuracy of small targets with less than 1024 spatial pixels (mAP_s), and average accuracy of medium targets with spatial pixels in the range of 1024~9216 (mAP_m). The validation results are shown in Table 1.
[0090] Table 1. Performance comparison of each model on the VisDrone dataset
[0091]
[0092] As shown in Table 1, the model of this invention achieved optimal results across all evaluation metrics. The model of this invention significantly outperforms other comparative models in both overall detection accuracy and small target detection accuracy, indicating that the synergistic effect of the deformable recurrent neural network encoding unit and the dynamic fusion module effectively improves the detection performance of small targets in complex scenes.
[0093] The improved RT-DETR-1 and improved RT-DETR-2 show improvements over the baseline model RT-DETR on all evaluation metrics, indicating that the deformable recurrent neural network encoding unit and the dynamic fusion module in this application can each bring effective performance gains.
[0094] The specific embodiments of the present invention are provided to enable those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention.
[0095] It should be understood that the present invention is not limited to the content already described above, and various modifications and changes can be made without departing from its scope. The scope of the present invention is limited only by the appended claims.
Claims
1. A small target detection method based on a deformable recurrent neural network, characterized in that, Includes the following steps: The image to be detected is acquired, and the image to be detected is preprocessed to obtain an image tensor; The image tensor is input into a pre-trained target detection neural network, which performs target detection on the image to be detected and outputs the category label and bounding box coordinates of the detected targets. The target detection neural network includes a backbone network, a deformable recurrent neural network encoder, a decoder, and a detection head connected in sequence. The backbone network is used to extract hierarchical features at multiple different scales from the image tensor; The deformable recurrent neural network encoder includes a dynamic fusion module and multiple deformable recurrent neural network encoding units. Each of the multiple deformable recurrent neural network encoding units corresponds one-to-one with multiple hierarchical features of different scales. The multiple deformable recurrent neural network encoding units are used to perform deformable spatial mixing processing and deformable channel mixing processing on the hierarchical features of the corresponding scales, and output multi-scale encoded features. The dynamic fusion module is used to perform adaptive alignment and fusion on the multi-scale encoded features to obtain fused enhanced features. The decoder is used to perform target query interaction based on the fused enhanced features and output a target representation vector; The detection head is used to generate the category label and bounding box coordinates based on the target representation vector.
2. The small target detection method based on deformable recurrent neural networks according to claim 1, characterized in that, The processing of hierarchical features at the corresponding scale by each deformable recurrent neural network encoding unit includes: Group normalization is performed on the hierarchical features at the corresponding scale to obtain group-normalized hierarchical features; The group-normalized hierarchical features are subjected to deformable spatial blending to obtain spatial blending features. The deformable spatial blending is used to establish long-range dependencies between different positions in the group-normalized hierarchical features in the spatial dimension, and to dynamically generate sampling offsets for each position of the group-normalized hierarchical features to adjust the receptive field. The spatial mixing features are residually connected with the hierarchical features at the corresponding scale to obtain residual connection features; The residual connectivity features are subjected to group normalization to obtain group-normalized residual connectivity features. The group-normalized residual connection features are subjected to deformable channel mixing processing to obtain channel mixing features. The deformable channel mixing processing is used to sequentially perform recombination, mapping and gating modulation on the group-normalized residual connection features in the channel dimension to improve the discriminativeness of the channel mixing features.
3. The small target detection method based on a deformable recurrent neural network according to claim 2, characterized in that, The process of deformable space mixing includes: The group-normalized hierarchical features are input into the deformable offset unit to predict the sampling offset corresponding to each reference position in the group-normalized hierarchical features. Based on the sampling offset, deformable sampling is performed on the group-normalized hierarchical features to obtain spatial rearrangement features; The spatial rearrangement features are linearly mapped to generate a first receiving feature, a first key feature, and a first value feature. The first key feature and the first value feature are input into the bidirectional WKV attention unit to obtain the context response result; The context response result is gated and modulated using the first receiving feature.
4. The small target detection method based on deformable recurrent neural networks according to claim 2, characterized in that, The processing steps of the deformable channel mixing module include: The group-normalized residual connection feature is input into the deformable offset unit to predict the sampling offset corresponding to each reference position in the group-normalized residual connection feature. Based on the sampling offset, the group normalized residual connectivity features are rearranged and sampled to obtain the channel rearranged features; Linear mapping is performed on the channel rearrangement features to generate a second reception feature and a second key feature; The second key feature is nonlinearly activated to obtain the activated key feature, and the second value feature is generated based on the activated key feature. The second value feature is gated and modulated using the second received feature.
5. The small target detection method based on a deformable recurrent neural network according to claim 3 or 4, characterized in that, The deformable offset unit adopts a multi-head deformable offset structure, and the process of predicting the sampling offset by the deformable offset unit is as follows: For each reference position, multiple offset heads predict the offset of multiple sampling points to determine the actual sampling position of each sampling point. Based on the actual sampling location of each sampling point, the features of each sampling point are obtained through bilinear interpolation. Predict the sampling weights corresponding to each sampling point; The sampling weights are normalized to obtain normalized sampling weights; The features of each sampling point are weighted and aggregated according to the normalized sampling weights to obtain the sampling offset corresponding to the reference position.
6. The small target detection method based on deformable recurrent neural networks according to claim 1, characterized in that, The dynamic fusion module includes a top-down fusion path and a bottom-up fusion path. In the top-down fusion path, a deformable feature alignment fusion module adaptively aligns and fuses encoded features at different scales step by step. The execution process of the deformable feature alignment fusion module includes: Dynamic sampling enhancement is performed on the high-level encoded features to obtain enhanced high-level features; The enhanced high-level features are channel-expanded and upsampled by pixel rearrangement to obtain the upsampled enhanced high-level features. Channel attention and spatial attention are applied to the low-level encoded features to obtain enhanced low-level features; the enhanced low-level features are added to the low-level encoded features to obtain residual enhanced low-level features. After the residual enhanced low-level features are spliced together with the upsampled enhanced high-level features, the output of the deformable feature alignment fusion module is obtained through fusion layer processing.
7. The small target detection method based on a deformable recurrent neural network according to claim 1, characterized in that, The target detection network is an end-to-end detector based on the Transformer architecture.
8. A small target detection system based on a deformable recurrent neural network, used to implement the small target detection method of claim 1, characterized in that, include: The image acquisition module is used to acquire the image to be detected; The preprocessing module is used to preprocess the image to be detected to obtain an image tensor; The object detection module includes a pre-trained object detection neural network, which is used to perform object detection on the image tensor and output the category labels and bounding box coordinates of all detected objects.