A multi-scale target detection method and device based on dynamic snake convolution feature enhancement and parameter-free attention mechanism

Abstract

This invention provides a multi-scale target detection method and apparatus based on dynamic serpentine convolution feature enhancement and a parameter-free attention mechanism. The method includes: acquiring an infrared image to be detected; processing the image using a target detection network within the YOLO framework, extracting an initial feature map from the infrared image, and inputting it into a backbone network embedded with a dynamic serpentine convolution feature enhancement module and a lightweight self-attention module, outputting a multi-scale feature map. The dynamic serpentine convolution generates a serpentine sampling trajectory along the X and Y axes through learnable offsets, while the lightweight self-attention calculates attention weights based on an energy function without introducing additional learnable parameters; after fusion by a feature pyramid network employing a dynamic sampling upsampling module, a fused feature map is output, where dynamic sampling upsampling replaces fixed interpolation with predicted sampling offsets; and the detection head outputs the detection result. This invention achieves improved accuracy in multi-scale, multi-morphological infrared target detection with relatively low parameter increments.

Description

Technical Field

[0001] This invention relates to the field of infrared imaging and infrared target detection technology, specifically providing a multi-scale target detection method and apparatus based on dynamic serpentine convolution feature enhancement and a parameter-free attention mechanism. Background Technology

[0002] Infrared imaging technology, by capturing the thermal radiation characteristics of objects, demonstrates strong all-weather adaptability and excellent penetration capabilities in fields such as remote sensing monitoring, drone reconnaissance, and security surveillance. It is particularly suitable for complex scenarios such as high-altitude, long-distance target identification. This technology can effectively overcome lighting limitations, providing stable target detection capabilities even under adverse conditions such as smoke and haze.

[0003] In recent years, deep learning-based target detection methods have provided a new technical path for infrared image processing, but significant limitations remain in practical applications. First, traditional convolutional networks use fixed rectangular grid sampling geometry, which is insufficient for extracting features from elongated, curved, and irregularly shaped targets. This leads to the gradual loss of local edge and contour details during propagation in deep networks, resulting in low recall rates for targets with varied shapes. Second, mainstream detection frameworks generally employ static upsampling strategies such as bilinear interpolation in the multi-scale fusion stage of the feature pyramid. The fixed sampling positions and lack of adaptive adjustment capabilities for feature content distribution cause information misalignment and detail loss during spatial alignment of high and low-level feature maps, hindering the effectiveness of multi-scale feature fusion. Furthermore, existing attention enhancement mechanisms often introduce a large number of learnable parameters to improve feature representation, significantly increasing the number of model parameters and computational complexity, making efficient deployment difficult on resource-constrained edge computing devices. Therefore, how to simultaneously enhance the network's ability to adaptively capture the morphological features of polymorphic targets and accurately fuse and reconstruct multi-scale features while maintaining model lightweight constraints has become a critical technical problem that urgently needs to be solved in the current target detection field. Summary of the Invention

[0004] To overcome the above-mentioned shortcomings, this invention is proposed to provide solutions or at least partially solve the problems of insufficient morphological feature extraction, low upsampling fusion accuracy, and high parameter overhead of attention mechanism in infrared target detection algorithms.

[0005] In a first aspect, the present invention provides a multi-scale target detection method based on dynamic serpentine convolution feature enhancement and a parameter-free attention mechanism, comprising the following steps: Acquire the infrared image of the object to be detected; The infrared image is processed using a target detection network based on the YOLO framework to obtain target detection results, including: Extract an initial feature map from the infrared image; The initial feature map is input into a backbone network for feature extraction. The backbone network embeds a dynamic serpentine convolution feature enhancement module and a lightweight self-attention module, and outputs multiple feature maps of different scales. The dynamic serpentine convolution feature enhancement module uses learnable offsets to make the convolution kernel generate a serpentine sampling trajectory along the X-axis and Y-axis. The lightweight self-attention module calculates attention weights for each neuron based on an energy function, and this calculation process does not introduce additional learnable parameters. The lightweight self-attention module is configured after each level of the backbone network output to perform self-attention calibration on the extracted features. The feature maps of different scales are input into a feature pyramid network for feature fusion. The feature pyramid network employs a dynamic sampling upsampling module and outputs a fused feature map for detection. In the top-down path of the feature pyramid network, the dynamic sampling upsampling module upsamples the feature maps from the deep layers of the backbone network using a predicted sampling offset, replacing the fixed interpolation operation. The fused feature map is input into the detection head, and the detection head outputs the target detection result.

[0006] Preferably, the dynamic snake-shaped convolution feature enhancement module is built based on the C3k2 architecture and includes at least one bottleneck unit; The dynamic snake-shaped convolution feature enhancement module expands the channels of the received input feature map and then divides it into a main branch feature map and a side branch feature map along the channel dimension. The main branch feature map is sequentially processed by channel compression, dynamic serpentine convolution, and channel splicing and fusion to obtain the bottleneck output feature map. After concatenating the bottleneck output feature map and the side branch feature map along the channel dimension, the number of channels is compressed by convolution to obtain the output feature map of the dynamic snake convolution feature enhancement module.

[0007] Preferably, the dynamic serpentine convolution processing is performed by a dynamic serpentine convolution module; the dynamic serpentine convolution module includes three parallel branches: a standard convolution branch, a serpentine convolution branch along the X-axis, and a serpentine convolution branch along the Y-axis. For the serpentine convolution branch along the X-axis, the convolution kernel has K sampling points arranged along the X-axis. The offset of each sampling point in the Y-axis is calculated using an iterative accumulation strategy: with the center sampling point as the reference, its Y-axis offset is set to zero; the cumulative offset in the Y-axis of the sampling points on both sides of the center sampling point is iteratively calculated according to the following formula: in, This represents the cumulative offset of the center sampling point in the Y direction. Setting it to zero means that the center sampling point is not offset. Indicates the rightmost point of the central sampling point The cumulative offset of each sampling point in the Y direction. This represents the cumulative offset in the Y direction of the previous adjacent sampling point. Represents the offset field. The predicted local offset increment in the Y direction for each channel at its corresponding spatial location. This represents the position index relative to the center sampling point, with a value range of 1. arrive , The index position of the center sampling point. Indicates the left side of the center sampling point The cumulative offset of each sampling point in the Y direction. Indicates the leftmost The cumulative offset of each sampling point in the Y direction, that is, the cumulative offset of the sampling point adjacent to the right of the current sampling point. Represents the offset field. The predicted local offset increment in the Y direction for each channel at its corresponding spatial location; For the serpentine convolution branch along the Y-axis, the convolution kernel has K sampling points arranged along the Y-axis. The offset of each sampling point in the X-axis is calculated using the same iterative accumulation strategy: with the center sampling point as the reference, its X-axis offset is set to zero; the cumulative X-axis offset of the sampling points on both sides of the center sampling point is iteratively calculated according to the following formula: in, This represents the cumulative offset of the center sampling point in the X direction. Its value is set to zero, meaning the center sampling point is not offset. Indicates the number of sampling points above the center. The cumulative offset of each sampling point in the X direction. This represents the cumulative offset in the X direction of the previous adjacent sampling point. Represents the offset field. The predicted local offset increment in the X direction for each channel at its corresponding spatial location. Indicates the number below the center sampling point The cumulative offset of each sampling point in the X direction. Indicates the first one below The cumulative offset of each sampling point in the X direction, that is, the cumulative offset of the adjacent sampling points above the current sampling point. Represents the displacement field. The local offset increment in the X direction predicted for each channel at its corresponding spatial location.

[0008] Preferably, in the serpentine convolution branch along the X-axis, the coordinates of each sampling point in the X-direction are uniformly distributed from -K / 2 to K / 2; after iterative accumulation of the offset, the... The actual sampling coordinates of each sampling point are: in, This represents the reference coordinates of the current output position on the feature map. Indicates the first The actual sampling coordinates of each sampling point on the feature map This is the offset range control factor. Indicates the first Cumulative offset in the Y direction of each sampling point; The actual sampling coordinates of the kth sampling point are sampled using bilinear interpolation to obtain a deformed feature map; the deformed feature map is then aggregated using a strip convolution kernel along the X-axis to obtain a horizontal serpentine feature map.

[0009] Preferably, the lightweight self-attention module processes the received input feature map. Perform self-attention calibration as follows: For position in channel C neurons at the location Calculate its energy function mapping value: in, This is the global mean of all pixels in this channel. To prevent division by zero constants, This represents the attention weight mapping value at the corresponding position after calculation using the energy function. , These represent the width and height of the feature map, respectively. , These represent the index variables for traversing along the width and height directions, respectively. After normalizing the energy function mapping value using a sigmoid activation function, it is multiplied element-wise with the input feature map to obtain the output feature map: in, This represents the Sigmoid activation function, which normalizes the energy function mapping values ​​to... Intervals are used to generate spatial attention weights.

[0010] Preferably, the dynamic sampling upsampling module adopts a linear prior style, and the upsampling ratio is s; The dynamic sampling upsampling module predicts a two-dimensional sampling offset field from the received input feature map through a convolutional layer; After the two-dimensional sampling offset field is suppressed by a scaling factor, it is compared with the preset initial sampling position. The samples are fused to obtain the fused sample coordinates; wherein, the initial sampling position... It consists of the normalized offset of each sampling point at the target resolution relative to the center of its corresponding low-resolution pixel; The fused sampling coordinates are added to the low-resolution standard grid coordinates and normalized. Then, the sampling point coordinates in the channel dimension are rearranged to the spatial dimension through a pixel rearrangement operation to obtain a high-resolution sampling coordinate grid. The input feature map is split into multiple groups according to the channel dimension. Each group of feature maps is sampled at the corresponding high-resolution sampling coordinates by bilinear interpolation. The sampling results of each group are then concatenated along the channel dimension to obtain the upsampled output feature map.

[0011] Preferably, the target detection network is trained using a two-stage training strategy: Freeze the network weights in the backbone network except for the dynamic serpentine convolution feature enhancement module and the lightweight self-attention module, and train only the parameters of the dynamic serpentine convolution feature enhancement module, the lightweight self-attention module, and the dynamic sampling upsampling module; Unfreeze all network weights and perform joint training on the entire object detection network.

[0012] Preferably, in the serpentine convolution branch along the Y-axis, the deformation feature map obtained after deformation sampling... By aggregating features using strip convolution kernels along the Y-axis, a vertical serpentine feature map is obtained. : The output feature map of the dynamic serpentine convolution module is derived from the output feature map of the standard convolution branch. The horizontal serpentine feature map output by the serpentine convolution branch along the X-axis direction. and the vertical serpentine feature map After splicing along the channel dimension, through The number of channels is obtained by convolution compression.

[0013] Preferably, the lightweight self-attention module is configured after the last convolutional layer of each stage in the backbone network; The dynamic sampling upsampling module upsamples the feature maps at the P5 and P4 levels of the backbone network in the top-down path of the feature pyramid network, respectively, so as to fuse them with the feature maps at the P4 and P3 levels.

[0014] In a second aspect, the present invention provides a multi-scale target detection device based on dynamic serpentine convolution feature enhancement and a parameter-free attention mechanism, comprising: Image acquisition unit, used to acquire infrared images to be detected; An initial feature extraction unit is used to extract an initial feature map from the infrared image; A backbone network module is used to extract features from the initial feature map and output multiple feature maps of different scales. The backbone network module embeds a dynamic serpentine convolutional feature enhancement unit and a lightweight self-attention unit. The dynamic serpentine convolutional feature enhancement unit uses learnable offsets to cause the convolution kernel to generate a serpentine sampling trajectory along the X and Y axes. The lightweight self-attention unit calculates attention weights for each neuron based on an energy function, and this calculation process does not introduce additional learnable parameters. The lightweight self-attention unit is configured after each level of the backbone network module's output and is used to perform self-attention calibration on the extracted features. The feature fusion module is used to fuse the features of the multiple feature maps at different scales and output a fused feature map for detection. The feature fusion module uses a dynamic sampling upsampling unit to construct a feature pyramid network. In the top-down path of the feature pyramid network, the dynamic sampling upsampling unit upsamples the feature maps from the deep layers of the backbone network module by predicting the sampling offset, in order to replace the fixed interpolation operation. The detection head module is used to output target detection results based on the fused feature map.

[0015] Compared with existing technologies, the multi-scale target detection method based on dynamic serpentine convolution feature enhancement and parameterless attention mechanism provided by this invention has the following advantages: By introducing a dynamic serpentine convolution feature enhancement module, this invention utilizes a learnable offset field to adaptively deform the convolution kernel along the horizontal and vertical directions, effectively capturing the edge and texture details of slender, curved, and irregularly shaped targets. This significantly enhances the model's ability to perceive the geometric adaptive features of polymorphic targets, solving the problem of insufficient feature extraction for targets with varied shapes using traditional fixed-grid convolution. By introducing a lightweight self-attention module, without adding any learnable parameters, a self-attention mechanism driven by an energy function is used to achieve collaborative calibration of channel and spatial dimensions, effectively suppressing background noise and enhancing the response of the target region. This solves the problem of large parameter counts and unfavorable deployment for edge devices in existing attention mechanisms. By introducing a dynamic sampling upsampling module, a learnable sampling offset replaces the traditional fixed interpolation operation, achieving content-adaptive multi-scale feature space alignment. This effectively reduces information misalignment and detail loss during feature pyramid fusion, significantly improving the detection accuracy and localization accuracy of small targets. Attached Figure Description

[0016] The disclosure of this invention will become more readily understood with reference to the accompanying drawings. It will be readily understood by those skilled in the art that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Furthermore, similar numbers in the drawings are used to denote similar components, wherein: Figure 1 This is a multi-scale target detection device based on dynamic snake convolution feature enhancement and parameterless attention mechanism according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall architecture of the detection network according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a dynamic snake-shaped convolution feature enhancement module according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the principle of a dynamic serpentine convolution sampling mechanism according to an embodiment of the present invention; Figure 5 This is a safety assessment of screws of different diameters according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a lightweight self-attention module according to an embodiment of the present invention. Detailed Implementation

[0017] Some embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0018] In the description of this invention, "module" and "processor" can include hardware, software, or a combination of both. A module can include hardware circuitry, various suitable sensors, communication ports, memory, and may also include software components, such as program code, or a combination of software and hardware. A processor can be a central processing unit, microprocessor, image processor, digital signal processor, or any other suitable processor. The processor has data and / or signal processing capabilities. The processor can be implemented in software, in hardware, or a combination of both. Non-transitory computer-readable storage media includes any suitable medium capable of storing program code, such as magnetic disks, hard disks, optical disks, flash memory, read-only memory, random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B, or A and B. The terms "at least one A or B" or "at least one of A and B" have a similar meaning to "A and / or B" and can include only A, only B, or A and B. The singular terms "a" or "this" can also include plural forms.

[0019] Example 1 like Figure 2-6 As shown, the multi-scale target detection method based on dynamic snake convolution feature enhancement and parameterless attention mechanism in this embodiment of the invention mainly includes the following steps S1-S4.

[0020] Step S1: Obtain the infrared image to be detected.

[0021] In this embodiment, an infrared camera mounted on a drone captures high-altitude scene images to construct a high-altitude infrared target detection dataset. , where the dataset The total number of elements in the middle is The image size is , 1 represents the number of image channels (single-channel infrared grayscale image). This defines the image height and width. Image annotation was performed using the open-source tool labelImg, with annotations including the infrared target category and the coordinates of the top-left and bottom-right corners of the detected target. The annotated information file is in XML format.

[0022] Step S2: Process the infrared image using a target detection network based on the YOLO framework to obtain target detection results, including: Step S21: Extract an initial feature map from the infrared image; Step S22: The initial feature map is input into the backbone network for feature extraction. The backbone network embeds a dynamic serpentine convolution feature enhancement module and a lightweight self-attention module, and outputs multiple feature maps of different scales. The dynamic serpentine convolution feature enhancement module generates a serpentine sampling trajectory along the X-axis and Y-axis directions through learnable offsets. The lightweight self-attention module calculates attention weights for each neuron based on an energy function, and this calculation process does not introduce additional learnable parameters. The lightweight self-attention module is configured after each level of the backbone network output to perform self-attention calibration on the extracted features.

[0023] In this embodiment, the following is constructed: Figure 2 The detection network model shown includes a Backbone feature extraction network with an embedded dynamic snake convolution feature enhancement module, a Neck feature fusion network with a dynamic sampling upsampling module, and a detection head structure.

[0024] The size is The infrared image is input into the Backbone of the detection network model. The Backbone extracts features through a series of convolutional layers, a dynamic serpentine convolutional feature enhancement module, and a lightweight self-attention module. The dynamic serpentine convolutional feature enhancement module adaptively deforms along the horizontal and vertical directions using learnable offsets, effectively capturing the edge and texture details of slender, curved, and irregularly shaped targets. The lightweight self-attention module achieves self-attention calibration of the channel and spatial dimensions through an energy function, effectively suppressing background noise and enhancing the response of the target region without introducing additional learnable parameters.

[0025] Backbone outputs three feature maps at different scales, namely P3, P4, and P5 levels. These are defined as follows: Its specific scale and number of channels are as follows: , ,as well as .

[0026] Step S23: Input the multiple feature maps of different scales into the feature pyramid network for feature fusion. The feature pyramid network adopts a dynamic sampling upsampling module and outputs a fused feature map for detection. In the top-down path of the feature pyramid network, the dynamic sampling upsampling module upsamples the feature maps from the deep layers of the backbone network by predicting the sampling offset, in order to replace the fixed interpolation operation.

[0027] In this embodiment, a feature pyramid network based on a dynamic sampling upsampling module is constructed in the Neck of the detection network model to achieve efficient feature map fusion. This network includes two paths: top-down and bottom-up. 1. The top-down path starts from the highest-level feature map of the backbone. Initially, the dynamic upsampling module upsamples the data level by level and compares it with the feature maps of the corresponding layers in the Backbone. The system performs splicing and fusion. The dynamic sampling upsampling module achieves adaptive spatial alignment through learnable sampling offsets, replacing traditional fixed interpolation upsampling and effectively reducing information misalignment and detail loss during multi-scale feature fusion.

[0028] 2. The bottom-up path starts from the P3 level features generated by the top-down path, downsamples them through convolution with a stride of 2, and then concatenates and fuses them with the feature maps of the corresponding level.

[0029] 3. After fusion and enhancement by the Neck network, three feature maps are finally output for detection, which are defined as follows: These three feature maps cover targets of small, medium, and large scales, with specific scales and number of channels as follows: , ,as well as .

[0030] Step S24: Input the fused feature map into the detection head, and the detection head outputs the target detection result.

[0031] In this implementation, These three output feature maps are fed into the detection head to complete the final classification and bounding box regression of the target.

[0032] In one embodiment, the dynamic serpentine convolution feature enhancement module, such as Figure 3 As shown, this module aims to address the problem that traditional standard convolution is limited by a fixed rectangular grid sampling geometry, making it difficult to effectively capture features of elongated, curved, and irregularly shaped targets. The dynamic serpentine convolution feature enhancement module is built on the C3k2 architecture and contains one bottleneck unit. The core innovation of each bottleneck unit is to replace the second standard convolution kernel in the standard bottleneck structure with a dynamic serpentine convolution branch, while introducing a two-branch channel splicing and fusion mechanism in the output stage.

[0033] Let the input features be ,in, This refers to the batch size (the number of samples processed simultaneously in one forward pass). The number of channels in the input feature map. The height of the input feature map, The input feature map width is [value], and the output channel number is [value]. The number of hidden channels is The bottleneck unit count is 1 (depth multiplied by repeat). The dynamic serpentine convolution feature enhancement module first expands the input channels to... It is divided into two paths along the channel dimension: in, This represents the feature map after the input features have been channel-expanded. express Convolutional module The main branch features are evenly distributed after the module-level channel expansion, including One channel; The lateral branch features after equal division also include One channel;

[0034] The main branch sequentially undergoes channel compression within the bottleneck, dynamic serpentine convolution processing, and the merging of the two branches: in This indicates a convolution module, used for dimensionality reduction or expansion of the channel dimension. This represents a dynamic serpentine convolution module; This indicates a channel dimension splicing operation.

[0035] Finally, the bottleneck output of the main branch is concatenated with the side branches and the channels are compressed to obtain the output of the dynamic serpentine convolution feature enhancement module. : in, The output features of the bottleneck unit are obtained by sequentially processing the main branch through channel compression, dynamic serpentine convolution, and splicing and merging the two branches.

[0036] In one embodiment, the dynamic serpentine convolution module, such as Figure 4 As shown, the dynamic serpentine convolution branch includes three parallel branches: a standard convolution branch, a serpentine convolution branch along the X-axis, and a serpentine convolution branch along the Y-axis. It achieves adaptive shape perception along the target geometric principal axis through a deformable convolution mechanism.

[0037] Let the input feature map be The kernel size is (default value) First, basic features are extracted using a standard convolutional branch, and the output is... At the same time, the input features enter two serpentine convolution branches, denoted as the serpentine convolution branches along the X-axis (…). ) and serpentine convolution branches along the Y-axis ( ).

[0038] for The core idea of ​​branching is to use a learnable offset field to make the convolution kernel extend horizontally while generating a serpentine swing in the vertical direction.

[0039] Specifically, firstly through a Offset prediction convolutional layer From input features The predicted two-dimensional offset field has the following output channels: , respectively corresponding Each sampling point is at direction and The offset in direction. This offset field is batch normalized and... Activation function normalized to Interval: in, This indicates a batch normalization operation. This represents the hyperbolic tangent activation function, which maps the offset value to... Intervals. Subsequently, the offset field is split along the channel dimension into... Directional offset component and Directional offset component .

[0040] for Branches (deformed along the X-axis), convolution kernels arranged along the X-axis. Each sampling point has its X-coordinate from arrive Uniformly distributed, with the Y-coordinate initially set to 0.

[0041] The offset of each sampling point in the Y direction employs an iterative accumulation strategy, causing the offsets of adjacent sampling points to form a chain-like propagation, simulating a serpentine oscillating trajectory. Specifically, with the center sampling point... As a reference, its Y-direction offset is set to 0, and the offsets of the remaining sampling points are calculated iteratively as follows: in, This represents the cumulative offset of the center sampling point in the Y direction. Setting its value to zero means the center sampling point remains in its original position without shifting. Indicates the rightmost point of the central sampling point The cumulative offset of each sampling point in the Y direction. This represents the cumulative offset in the Y direction of the previous adjacent sampling point. Represents the offset field. The predicted local offset increment in the Y direction for each channel at its corresponding spatial location. This represents the position index relative to the center sampling point, with a value range of 1. arrive , The index position of the center sampling point. Indicates the left side of the center sampling point The cumulative offset of each sampling point in the Y direction. Indicates the left side of the center sampling point The cumulative offset of each sampling point in the Y direction, that is, the cumulative offset of the sampling point adjacent to the right of the current sampling point. Represents the offset field. Each channel predicts the local offset increment in the Y direction at the corresponding spatial location. This iterative accumulation mechanism ensures that the actual offset of each sampling point depends not only on the predicted offset at the current location but also on the cumulative offset of the previous sampling point, thus forming a coherent serpentine sampling trajectory that can closely fit the curved and irregular shape features of the target.

[0042] After iterative accumulation of offsets, the first... The actual sampling coordinates of each sampling point on the feature map for: in, This represents the reference coordinates of the current output position on the feature map. Indicates the first The actual sampling coordinates of each sampling point on the feature map This is the offset range control factor (default value is 1).

[0043] Since the actual sampling coordinates may fall at non-integer pixel positions, feature sampling is performed at the deformed coordinates using bilinear interpolation to obtain the deformed feature map. Finally, a strip convolution kernel along the X-axis (kernel size is...) is applied. Step size is Perform convolution operation on the deformed feature map, and then... The features from each sampling point are aggregated into the final horizontal serpentine feature response. : in, For grouping normalization, For activation function, Indicates the kernel size as Step size is The bar convolution operation aggregates features from the deformed feature map along the X-axis. This represents the deformation feature map along the X-axis obtained after bilinear interpolation sampling.

[0044] Similarly, The process of handling branches (deformation along the Y-axis) and Symmetrical. The convolution kernels are arranged along the Y-axis. For each sampling point, the offset in the X direction is calculated using the same iterative accumulation strategy: in, This represents the cumulative offset of the center sampling point in the X direction. Setting its value to zero means the center sampling point remains in its original position without shifting. Indicates the number of sampling points above the center. i The cumulative offset of each sampling point in the X direction. This represents the cumulative offset in the X direction of the previous adjacent sampling point, i.e., the number of points above. Cumulative offset of each sampling point Represents the offset field. The predicted local offset increment in the X direction for each channel at its corresponding spatial location. Indicates the number below the center sampling point The cumulative offset of each sampling point in the X direction. Indicates the first one below The cumulative offset of each sampling point in the X direction, that is, the cumulative offset of the adjacent sampling points below the current sampling point. Represents the offset field. The local offset increment in the X direction predicted for each channel at its corresponding spatial location.

[0045] Deformation feature map obtained after deformation sampling By using a strip convolution kernel along the Y-axis (kernel size is...) Step size is Feature aggregation is performed to obtain the vertical serpentine feature response. : Wherein, GN() is grouping normalization.

[0046] Finally, output the standard convolution. Horizontal serpentine features and vertical serpentine features The output of the dynamic serpentine convolutional feature enhancement module is obtained by concatenating and fusing the data along the channel dimension. : To achieve compact feature representation and cross-scale interaction, the module... Convolution reduces the number of channels from Compress to The final output is a feature map with the same number of input channels. This design enables the network to effectively enhance its ability to extract fine-grained features from multi-morphological targets while maintaining computational efficiency, making it particularly suitable for detecting elongated, curved-edge, and irregularly contoured targets.

[0047] In one embodiment, a lightweight self-attention module such as Figure 5 As shown, this module is designed as a lightweight plug-and-play structure, aiming to achieve synergistic enhancement of channel and spatial dimensions through an energy function-driven self-attention mechanism, while avoiding computational overhead caused by introducing additional learnable parameters.

[0048] Let the input feature map be The number of channels is The space dimensions are The core idea of ​​the lightweight self-attention module originates from the sparse weighted attention mechanism in neuroscience, which measures the importance of each neuron by assigning it an independent energy function. Specifically, for channels... Middle position neurons at the location Its energy function is defined as: in, This is the global mean of all pixels in this channel. For global variance, To prevent division by zero, a small constant (usually taken as...) ), , These represent the width and height of the feature map, respectively. , These represent the index variables for traversing along the width and height directions, respectively.

[0049] To simplify calculations and ensure numerical stability, the above energy function is equivalently transformed as follows: in, Indicates the input feature map channel Medium spatial position Neuron value at that location, This represents the attention weight mapping value at the corresponding position after calculation using the energy function. , These represent the width and height of the feature map, respectively. , These represent the index variables for traversing along the width and height directions, respectively.

[0050] Furthermore, the energy function is simplified to a batch statistical form, denoted as... If we subtract one from the total number of pixels, then: Among them, global mean This can be efficiently computed using global average pooling. Finally, the attention weights are normalized to a sigmoid activation function. The interval is defined as the element-wise product of the input features and the attention weights. The core advantage of this module lies in its zero-parameter characteristic: the entire attention calculation process does not contain any learnable weight matrix, and adaptive channel-spatial co-calibration can be completed solely based on the batch statistics of the input features. After inserting the lightweight self-attention module into each stage of the backbone network, it can effectively enhance the feature response intensity of the target region without increasing the number of parameters and computational cost, while suppressing interference from the background region, thereby improving the network's comprehensive perception capability of multi-scale targets.

[0051] In one embodiment, the dynamic sampling upsampling module is as follows: Figure 6 As shown, this module aims to replace traditional static upsampling methods such as bilinear interpolation. It achieves adaptive content-aware feature map spatial resolution improvement through learnable sampling offsets, solving the problem of insufficient multi-scale feature alignment accuracy during upsampling.

[0052] The dynamic sampling upsampling module supports two upsampling styles: "Linear Prior" (hereinafter referred to as...). ) and "Pixel Shuffle Prior" (denoted as This invention employs a linear prior style (). Its core structure consists of three parts: a sampling offset prediction network, a pixel rearrangement operator, and a grouped grid sampling operator.

[0053] Let the input feature map be The upsampling ratio is (In this invention, take) First, through Convolutional layers predict two-dimensional sampling offset fields from input features. ,in Number of groups (default value) ), For each group at the target resolution The two-dimensional coordinate offset of each sampling point. This offset field is scaled by a scaling factor. After amplitude suppression, compared with the preset initial sampling position To merge: in, Indicates the final sampled coordinates after fusion. Indicates passage The sampling offset field of convolution prediction, initial sampling position Defined as the center point coordinates of the standard upsampling grid, it consists of the normalized offset of each sampling point at the target resolution relative to the center of its corresponding low-resolution pixel, and is calculated using the following formula: by For example, the coordinates of the four initial sampling positions are as follows: , , and These correspond to the center positions of the four sub-pixels within the target pixel region. After grouping and copying, these initial positions have the following shape: , and offset field Alignment of channel dimensions.

[0054] Then, the merged offset Rearranged into a five-dimensional tensor The second dimension corresponds to the Y and X coordinate axes, and the third dimension corresponds to the area within each group. Each sampling point. Based on this, a low-resolution standard grid coordinate system is constructed. The coordinates of each spatial location are the center pixel coordinates plus an offset of 0.5 (to achieve pixel center alignment). After adding the blended offset to the standard grid coordinates, it is normalized to a linear mapping. Interval: in, Represents low-resolution standard grid coordinates, with a preset reference sampling position for each spatial location. Here, are normalization factors, corresponding to the width and height of the feature map, respectively. The normalized coordinates are then expanded to a higher resolution using a pixel shuffle operation, extending the low-resolution spatial dimensions. The coordinates of each sampling point are rearranged from the channel dimension to the spatial dimension to obtain a high-resolution sampling coordinate grid. .

[0055] Finally, the input feature maps are grouped according to... The features are split into groups, and each group is sampled at high-resolution sampling coordinates using bilinear interpolation. The sampling results of each group are then concatenated along the channel dimension to obtain the final upsampled feature map. in, This represents the final output upsampled feature map. This represents a bilinear interpolation sampling operation, which extracts feature values ​​at corresponding positions from the input feature map based on a given coordinate grid. For the input feature map, the first Sub-features of the group These are the high-resolution sampling coordinates corresponding to this group. For bilinear interpolation sampling operators, Indicates will The sampling results of each group are concatenated along the channel dimension. The final output upsampled feature map size is... .

[0056] Dynamic upsampling, through an end-to-end offset prediction network, enables sampling points to cluster towards salient feature regions (such as target edges and texture details), thereby effectively preserving and enhancing key feature information while improving spatial resolution. This design significantly improves the spatial alignment accuracy in the multi-scale feature pyramid fusion process, providing higher-resolution feature representations for small target detection.

[0057] In one embodiment, the training method includes first freezing the weights of the backbone network, training only the parameters of the proposed dynamic serpentine convolutional feature enhancement module, lightweight self-attention module, and dynamic sampling upsampling module, training for 50 epochs, then unfreezing all weights and jointly training the entire network for 250 epochs. The loss function used is consistent with the loss function in the YOLO detection framework, including classification loss, bounding box regression loss, and distributed focus loss (DFL). Network training parameters are set: initial learning rate... The batch size is 16, the training set and validation set are divided in an 8:2 ratio, the optimizer is SGD (stochastic gradient descent), and the training cycle is 300 epochs.

[0058] The trained network is used for prediction. The input is a test image, and the output is a predicted bounding box for the infrared image target. First, the test image is... The image size is input into the network. After feature extraction by the backbone network, the features are enhanced by a dynamic serpentine convolutional feature enhancement module and a lightweight self-attention module for morphological feature enhancement and channel calibration. Subsequently, in the Neck feature fusion network, a dynamic sampling upsampling module achieves precise spatial alignment of multi-scale feature maps. Finally, the detection head completes network inference, yielding the output of the detection head. The scale of the output feature map is... , and The three feature maps correspond to the three detection layers P3, P4, and P5, respectively. The classification and regression prediction results are extracted from the feature maps at different scales and then concatenated and transformed in dimension. For ease of processing, the original channel dimension is moved to the end, resulting in the following shapes for the class prediction branch and bounding box prediction branch: and ,in Image batch processing size, For the number of prediction points, The preset number of categories is used. All targets are sorted in descending order of their presence confidence (conf), and then the IoU (Intersection over Union) with other predictions is calculated for each target from highest to lowest. Prediction with an IoU greater than 0.45 is discarded. Subsequently, based on the previous preprocessing steps, the remaining detection boxes are restored to the original image scale before network output, and non-maximum suppression (NMS) is performed to remove redundant detection boxes. The number of output detection boxes does not exceed the preset maximum number of detections, 300.

[0059] Obtain the coordinate position of the described detection box. The corresponding normalized position coordinates The bounding box is then plotted on the test image. If a detection box is output for the image to be detected, it means that the target has been detected in the infrared image; otherwise, it means that the target has not been detected.

[0060] Example 2 like Figure 1As shown, this invention provides a multi-scale target detection device based on dynamic serpentine convolution feature enhancement and a parameter-free attention mechanism, comprising: an image acquisition unit for acquiring an infrared image to be detected; an initial feature extraction unit for extracting an initial feature map from the infrared image; and a backbone network module for extracting features from the initial feature map and outputting multiple feature maps of different scales. The backbone network module embeds a dynamic serpentine convolution feature enhancement unit and a lightweight self-attention unit. The dynamic serpentine convolution feature enhancement unit uses learnable offsets to generate a serpentine sampling trajectory along the X-axis and Y-axis directions in the convolution kernel. The lightweight self-attention unit calculates attention for each neuron based on an energy function. The weights are calculated without introducing additional learnable parameters; the lightweight self-attention unit is configured after each level of the backbone network module to perform self-attention calibration on the extracted features; the feature fusion module is used to fuse the multiple feature maps of different scales and output a fused feature map for detection; the feature fusion module uses a dynamic sampling upsampling unit to construct a feature pyramid network; in the top-down path of the feature pyramid network, the dynamic sampling upsampling unit upsamples the feature maps from the deep layers of the backbone network module by a predicted sampling offset to replace the fixed interpolation operation; the detection head module is used to output the target detection result based on the fused feature map. In one embodiment, a description of the specific implementation function can be found in steps S1-S2.

[0061] The aforementioned multi-scale object detection device based on dynamic snake-like convolution feature enhancement and parameterless attention mechanism is used to perform... Figure 1 The embodiments of the multi-scale target detection method based on dynamic serpentine convolution feature enhancement and parameterless attention mechanism shown are similar in technical principle, technical problem solved and technical effect produced. Those skilled in the art can clearly understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the multi-scale target detection device based on dynamic serpentine convolution feature enhancement and parameterless attention mechanism can be found in the embodiments of the multi-scale target detection method based on dynamic serpentine convolution feature enhancement and parameterless attention mechanism, and will not be repeated here.

[0062] In one embodiment, the device is deployed as follows: Figure 1 The system shown consists of a drone 104, an infrared camera 103, a wireless image transmission module 102, and a computer 101. The drone 104 carries the infrared camera 103 to take pictures and transmits data to the computer 101 through the wireless image transmission module 102. The computer 101 is equipped with an image acquisition module, a feature extraction module, a feature fusion module, and a detection output module to realize the detection and positioning of high-altitude multi-scale targets.

[0063] Those skilled in the art will understand that the various modules in the device can be adaptively split or combined. Such splitting or combining of specific modules will not cause the technical solution to deviate from the principles of the present invention; therefore, the technical solutions after splitting or combining will fall within the protection scope of the present invention.

[0064] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the original technical features, and the technical solutions resulting from these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. A multi-scale object detection method based on dynamic snake convolution feature enhancement and parameter-free attention mechanism, characterized in that, Includes the following steps: Acquire the infrared image of the object to be detected; The infrared image is processed using a target detection network based on the YOLO framework to obtain target detection results, including: Extract an initial feature map from the infrared image; The initial feature map is input into a backbone network for feature extraction. The backbone network embeds a dynamic serpentine convolution feature enhancement module and a lightweight self-attention module, and outputs multiple feature maps of different scales. The dynamic serpentine convolution feature enhancement module uses learnable offsets to make the convolution kernel generate a serpentine sampling trajectory along the X-axis and Y-axis. The lightweight self-attention module calculates attention weights for each neuron based on an energy function, and this calculation process does not introduce additional learnable parameters. The lightweight self-attention module is configured after each level of the backbone network output to perform self-attention calibration on the extracted features. The feature maps of different scales are input into a feature pyramid network for feature fusion. The feature pyramid network employs a dynamic sampling upsampling module and outputs a fused feature map for detection. In the top-down path of the feature pyramid network, the dynamic sampling upsampling module upsamples the feature maps from the deep layers of the backbone network using a predicted sampling offset, replacing the fixed interpolation operation. The fused feature map is input into the detection head, and the detection head outputs the target detection result.

2. The method of claim 1, wherein, The dynamic snake-shaped convolution feature enhancement module is built on the C3k2 architecture and includes at least one bottleneck unit; The dynamic snake-shaped convolution feature enhancement module expands the channels of the received input feature map and then divides it into a main branch feature map and a side branch feature map along the channel dimension. The main branch feature map is sequentially processed by channel compression, dynamic serpentine convolution, and channel splicing and fusion to obtain the bottleneck output feature map. After concatenating the bottleneck output feature map and the side branch feature map along the channel dimension, the number of channels is compressed by convolution to obtain the output feature map of the dynamic snake convolution feature enhancement module.

3. The method of claim 2, wherein, The dynamic serpentine convolution processing is performed by a dynamic serpentine convolution module; the dynamic serpentine convolution module includes three parallel branches: a standard convolution branch, a serpentine convolution branch along the X-axis, and a serpentine convolution branch along the Y-axis; For the serpentine convolution branch along the X-axis, the convolution kernel has K sampling points arranged along the X-axis. The offset of each sampling point in the Y-axis is calculated using an iterative accumulation strategy: with the center sampling point as the reference, its Y-axis offset is set to zero; the cumulative offset in the Y-axis of the sampling points on both sides of the center sampling point is iteratively calculated according to the following formula: in, This represents the cumulative offset of the center sampling point in the Y direction. Setting it to zero means that the center sampling point is not offset. Indicates the rightmost point of the central sampling point The cumulative offset of each sampling point in the Y direction. This represents the cumulative offset in the Y direction of the previous adjacent sampling point. Represents the offset field. The predicted local offset increment in the Y direction for each channel at its corresponding spatial location. This represents the position index relative to the center sampling point, with a value range of 1. arrive , The index position of the center sampling point. Indicates the left side of the center sampling point The cumulative offset of each sampling point in the Y direction. Indicates the leftmost The cumulative offset of each sampling point in the Y direction, that is, the cumulative offset of the sampling point adjacent to the right of the current sampling point. Represents the offset field. The predicted local offset increment in the Y direction for each channel at its corresponding spatial location; For the serpentine convolution branch along the Y-axis, the convolution kernel has K sampling points arranged along the Y-axis. The offset of each sampling point in the X-axis is calculated using the same iterative accumulation strategy: with the center sampling point as the reference, its X-axis offset is set to zero; the cumulative X-axis offset of the sampling points on both sides of the center sampling point is iteratively calculated according to the following formula: in, This represents the cumulative offset of the center sampling point in the X direction. Setting it to zero means that the center sampling point is not offset. Indicates the number of sampling points above the center. The cumulative offset of each sampling point in the X direction. This represents the cumulative offset in the X direction of the previous adjacent sampling point. Represents the offset field. The predicted local offset increment in the X direction for each channel at its corresponding spatial location. Indicates the number below the center sampling point The cumulative offset of each sampling point in the X direction. Indicates the following number The cumulative offset of each sampling point in the X direction, that is, the cumulative offset of the adjacent sampling points above the current sampling point. Represents the displacement field. The local offset increment in the X direction predicted for each channel at its corresponding spatial location.

4. The method according to claim 3, characterized in that, In the serpentine convolution branch along the X-axis, the coordinates of each sampling point in the X-direction are uniformly distributed from -K / 2 to K / 2; after iterative accumulation of the offset, the th... The actual sampling coordinates of each sampling point are: in, This represents the reference coordinates of the current output position on the feature map. Indicates the first The actual sampling coordinates of each sampling point on the feature map This is the offset range control factor. Indicates the first Cumulative offset in the Y direction of each sampling point; The actual sampling coordinates of the kth sampling point are sampled using bilinear interpolation to obtain a deformed feature map; the deformed feature map is then aggregated using a strip convolution kernel along the X-axis to obtain a horizontal serpentine feature map.

5. The method according to claim 1, characterized in that, The lightweight self-attention module receives the input feature map. Perform self-attention calibration as follows: For position in channel C neurons at the location Calculate its energy function mapping value: in, This is the global mean of all pixels in this channel. To prevent division by zero constants, This represents the attention weight mapping value at the corresponding position after calculation using the energy function. , These represent the width and height of the feature map, respectively. , These represent the index variables for traversing along the width and height directions, respectively. After normalizing the energy function mapping value using a sigmoid activation function, it is multiplied element-wise with the input feature map to obtain the output feature map: in, This represents the Sigmoid activation function, which normalizes the energy function mapping values ​​to... Intervals are used to generate spatial attention weights.

6. The method according to claim 1, characterized in that, The dynamic sampling upsampling module adopts a linear prior style, with an upsampling ratio of s; The dynamic sampling upsampling module predicts a two-dimensional sampling offset field from the received input feature map through a convolutional layer; After the two-dimensional sampling offset field is suppressed by a scaling factor, it is compared with the preset initial sampling position. The samples are fused to obtain the fused sample coordinates; wherein, the initial sampling position... It consists of the normalized offset of each sampling point at the target resolution relative to the center of its corresponding low-resolution pixel; The fused sampling coordinates are added to the low-resolution standard grid coordinates and normalized. Then, the sampling point coordinates in the channel dimension are rearranged to the spatial dimension through a pixel rearrangement operation to obtain a high-resolution sampling coordinate grid. The input feature map is split into multiple groups according to the channel dimension. Each group of feature maps is sampled at the corresponding high-resolution sampling coordinates by bilinear interpolation. The sampling results of each group are then concatenated along the channel dimension to obtain the upsampled output feature map.

7. The method according to claim 1, characterized in that, The object detection network is trained using a two-stage training strategy: Freeze the network weights in the backbone network except for the dynamic serpentine convolution feature enhancement module and the lightweight self-attention module, and train only the parameters of the dynamic serpentine convolution feature enhancement module, the lightweight self-attention module, and the dynamic sampling upsampling module; Unfreeze all network weights and perform joint training on the entire object detection network.

8. The method according to claim 3, characterized in that, The deformation feature map obtained after deformation sampling in the serpentine convolution branch along the Y-axis direction. By aggregating features using strip convolution kernels along the Y-axis, a vertical serpentine feature map is obtained. : The output feature map of the dynamic serpentine convolution module is derived from the output feature map of the standard convolution branch. The horizontal serpentine feature map output by the serpentine convolution branch along the X-axis direction. and the vertical serpentine feature map After splicing along the channel dimension, through The number of channels is obtained by convolution compression.

9. The method according to claim 1, characterized in that, The lightweight self-attention module is configured after the last convolutional layer in each stage of the backbone network; The dynamic sampling upsampling module upsamples the feature maps at the P5 and P4 levels of the backbone network in the top-down path of the feature pyramid network, respectively, so as to fuse them with the feature maps at the P4 and P3 levels.

10. A multi-scale target detection device based on dynamic serpentine convolution feature enhancement and a parameter-free attention mechanism, characterized in that, include: Image acquisition unit, used to acquire infrared images to be detected; An initial feature extraction unit is used to extract an initial feature map from the infrared image; A backbone network module is used to extract features from the initial feature map and output multiple feature maps of different scales. The backbone network module embeds a dynamic serpentine convolutional feature enhancement unit and a lightweight self-attention unit. The dynamic serpentine convolutional feature enhancement unit uses learnable offsets to cause the convolution kernel to generate a serpentine sampling trajectory along the X and Y axes. The lightweight self-attention unit calculates attention weights for each neuron based on an energy function, and this calculation process does not introduce additional learnable parameters. The lightweight self-attention unit is configured after each level of the backbone network module's output and is used to perform self-attention calibration on the extracted features. The feature fusion module is used to fuse the features of the multiple feature maps at different scales and output a fused feature map for detection. The feature fusion module uses a dynamic sampling upsampling unit to construct a feature pyramid network. In the top-down path of the feature pyramid network, the dynamic sampling upsampling unit upsamples the feature maps from the deep layers of the backbone network module by predicting the sampling offset, in order to replace the fixed interpolation operation. The detection head module is used to output target detection results based on the fused feature map.

Images