Bird fine-grained detection algorithm of snake-shaped deformable convolution and dilated convolution

Abstract

The application discloses a snake-shaped deformable convolution and cavity convolution bird fine-grained detection algorithm, relates to bird identification, and solves the technical problems of insufficient coupling of fine-grained feature extraction and positioning, difficult target shape learning, limited modeling capability of local deformation, limited fine-grained perception, and huge deployment resource consumption of existing bird fine-grained identification technology. The algorithm comprises the following steps: S11: predicting a dynamic offset according to bird image information; S12: dynamically deforming according to the dynamic offset to generate target sampling coordinates; S13: obtaining bird features after dynamic deformation through the target sampling coordinates; S14: obtaining enhanced bird features according to the bird features after dynamic deformation; S15: fusing the enhanced bird features and basic bird features to generate bird fusion features; and S16: outputting target bird fine-grained features according to the bird fusion features. The application reduces the parameter quantity and the calculation complexity of the model, meets the lightweight demand, and effectively captures local deformation and fine features.

Description

Technical Field

[0001] This invention relates to bird identification, and more specifically, to a fine-grained bird detection algorithm based on serpentine deformable convolution and dilated convolution. Background Technology

[0002] Bird species identification, as an important technical means for biodiversity monitoring and ecological environmental protection, has received widespread attention in recent years. Studies have shown a significant correlation between bird population dynamics and habitat environmental changes, and real-time monitoring of bird community structure can effectively assess ecosystem health. Traditional bird identification mainly relies on manual observation and acoustic analysis, but faces limitations such as low efficiency and strong subjectivity in large-scale monitoring scenarios. With the development of computer vision technology, deep learning-based bird image classification has gradually become the mainstream solution.

[0003] Current deep learning-based fine-grained bird recognition technologies mainly focus on optimizing the target detection framework and can be divided into three main technical approaches:

[0004] Single-stage object detection methods in CNNs: Representative works such as YOLOv5 and RetinaNet directly predict bird location and species category through end-to-end regression.

[0005] CNN multi-stage object detection methods: Two-stage frameworks, represented by Faster R-CNN, generate candidate boxes through Region Proposal Networks (RPNs) and then perform fine-grained classification. Transformer object detection methods: DETR was the first to introduce the Transformer encoder-decoder architecture into the detection task, capturing global feature dependencies through a self-attention mechanism.

[0006] Defects and shortcomings of existing technology:

[0007] 1. Insufficient coupling between fine-grained feature extraction and localization: Existing single-stage methods (such as YOLOv5 and SSD) do not explicitly decouple fine-grained semantic information from spatial localization features during feature extraction, which makes key discriminative features such as feather texture and beak morphology easily interfered with by background noise.

[0008] 2. Difficulty in learning target shape and limited ability to model local deformation: The RPN network used in Faster R-CNN typically generates candidate regions using fixed-scale anchor boxes. However, for deformed or irregular targets, these fixed-shape anchor boxes often fail to effectively match the target shape, leading to missed detections. Because traditional R-CNN relies on RoIPooling or RoIAlign, its ability to model local deformation in target detection is limited.

[0009] 3. The Transformer architecture has limitations in fine-grained perception and consumes huge deployment resources: Traditional Transformers rely on global attention (Self-Attention), which can effectively learn long-distance dependencies, but its ability to capture local deformation features is weak, resulting in unstable performance of the detector for small and edge targets. The computational cost is too high, making it difficult to deploy on lightweight devices: The classic DETR requires calculating global attention across all pixels, resulting in a computational complexity of O(N^2). 2 However, with high-resolution input, the inference time is long, making it difficult to apply to real-time scenarios. Summary of the Invention

[0010] The technical problem to be solved by this invention is to address the shortcomings of existing technologies by providing a fine-grained bird detection algorithm based on snake-shaped deformable convolution and dilated convolution. This algorithm solves the technical problems of insufficient coupling between fine-grained feature extraction and localization, difficulty in target morphology learning, limited modeling ability of local deformation, limited fine-grained perception, and huge deployment resource consumption in existing bird fine-grained recognition technologies.

[0011] The present invention discloses a fine-grained bird detection algorithm based on snake-shaped deformable convolution and dilated convolution, which includes: acquiring bird image information; triggering a first convolution strategy based on the bird image information to output bird fusion features; and triggering a target convolution strategy based on the bird fusion features to output target bird fine-grained features.

[0012] As a further improvement, the first convolutional strategy includes:

[0013] S11: Extract basic bird features from the bird image information, and predict dynamic offset based on the basic bird features;

[0014] S12: Construct a basic sampling network, and dynamically deform the basic sampling coordinates of the basic sampling network according to the dynamic offset to generate target sampling coordinates;

[0015] S13: Perform bilinear interpolation sampling on the basic bird features using the target sampling coordinates to obtain dynamically deformed bird features;

[0016] S14: The dynamically deformed bird features are then subjected to subsequent convolution, normalization, and activation processing to obtain enhanced bird features;

[0017] S15: The enhanced bird features and the basic bird features are fused to generate a bird fusion feature.

[0018] Furthermore, in S11, the method for predicting the dynamic offset is to set a convolutional layer, predict the initial offset based on the convolutional layer and basic bird features, and map the initial offset value to the dynamic offset through the Batch Normalization data normalization function and the tanh activation function.

[0019] Furthermore, in S12, the method for generating the target sampling coordinates is to set the original center coordinates and convolution kernel template to form a basic sampling network, obtain the basic sampling coordinates of the basic sampling network, and apply dynamic offsets to the horizontal and vertical coordinates of the basic sampling coordinates according to the direction of the dynamic deformation to generate the target sampling coordinates.

[0020] Furthermore, when the direction of the dynamic deformation is perpendicular to the vertical direction, the expression for applying a dynamic offset to the vertical coordinate is:

[0021] y base =y center +0, x base =x center +x grid ;

[0022]

[0023] x new =x base ;

[0024] Among them, (x base y base (x) represents the base sampling coordinates. center y center () represents the original center coordinates, and the center position of the convolution kernel. When the dynamic deformation direction is set to vertical, the x-component of the convolution kernel template is in the interval Uniformly distributed inside This represents the dynamic offset of the vertical coordinate. pass Calculations show that extend_scope is used for scaling. The determined action offset, (x) new y new ) represents the target sampling coordinates.

[0025] Furthermore, when the direction of the dynamic deformation is horizontal, a dynamic offset is applied to the horizontal coordinate. The expression for applying the dynamic offset to the horizontal coordinate is:

[0026] y base =y center +0, x base =xcenter +x grid ;

[0027] y new =y base ;

[0028] Among them, (x base y base (x) represents the base sampling coordinates. center y center () represents the original center coordinates, and the center position of the convolution kernel. When the dynamic deformation direction is set to horizontal, the y-component of the convolution kernel template is in the interval... Uniformly distributed inside This represents the dynamic offset of the horizontal coordinate. pass Calculations show that extend_scope is used to scale the scope defined by... The obtained dynamic offset, (x new y new ) represents the target sampling coordinates.

[0029] Furthermore, in S13, the expression for the dynamically deformed bird characteristics is obtained as follows:

[0030] f(y new x new )≈f(y0,x0)·(y1-y)(x1-x)+f(y0,x1)·(y1-y)(x-X0)

[0031] +f(y1,x0)·(y-y0)(x1-x)+f(y1,x1)·(y-y0)(x-x0);

[0032] in, y1 = y0 + 1, x1 = x0 + 1, f(y new ,x new ) represents the bird's features after dynamic deformation.

[0033] Furthermore, in S15, the expression for generating the bird fusion feature is:

[0034] Output1=concat(conv_0(x), conv_x(x), conv_y(x))

[0035] Where Output1 is the bird fusion feature, conv_0(x) is the basic bird feature, conv_x(x) is the feature quantity of the enhanced bird feature in the horizontal direction, and conv_y(x) is the feature quantity of the enhanced bird feature in the vertical direction.

[0036] Furthermore, the target convolution strategy includes:

[0037] S161: Generate the main features of the main convolutional branch based on the bird fusion features;

[0038] S162: Generate inexpensive features for inexpensive convolutional branches based on the bird fusion features;

[0039] S163: The main features and the low-cost features are spliced ​​together to output the fine-grained features of the target bird.

[0040] Furthermore, in S163, the expression for concatenating and outputting the fine-grained features of the target bird is:

[0041] Output2=concat(y,z),Output2∈R B×c2×H′×W ′;

[0042] Where Output2 represents the fine-grained features of the target bird, y represents the main features, z represents the inexpensive features, and B×c2×H′×W′ represents the size of the fine-grained features of the target bird in the channel dimension.

[0043] Beneficial effects

[0044] The advantages of this invention are:

[0045] This invention acquires bird image information, triggers a first convolution strategy based on the bird image information to output bird fusion features, and triggers a target convolution strategy based on the bird fusion features to output fine-grained features of the target bird. Through deep group convolution and dynamic sampling, the number of model parameters and computational complexity are reduced, meeting the requirements for lightweight design. It effectively captures local deformation and subtle features, improves the classification and detection accuracy in complex scenes, and significantly reduces redundant computation while ensuring feature richness. It has high stability and robustness. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the overall neural network structure of the first convolution strategy and feature fusion of the present invention;

[0047] Figure 2 This is a schematic diagram of the standard convolutional branch neural network structure in the first convolutional strategy of the present invention;

[0048] Figure 3This is a diagram of the dynamic convolution branch X neural network structure in the first convolution strategy of the present invention;

[0049] Figure 4 This is a diagram of the dynamic convolution branch Y neural network structure in the first convolution strategy of the present invention;

[0050] Figure 5 This is a diagram of the neural network structure of the target convolution strategy of this invention;

[0051] Figure 6 This is a comparison chart of the training results of this invention with those of other neural networks. Detailed Implementation

[0052] The present invention will be further described below with reference to embodiments, but this does not constitute any limitation on the present invention. Any limited modifications made by any person within the scope of the claims of the present invention are still within the scope of the claims of the present invention.

[0053] See Figures 1-6 The present invention discloses a fine-grained bird detection algorithm based on snake-shaped deformable convolution and dilated convolution, comprising: acquiring bird image information; triggering a first convolution strategy based on the bird image information to output bird fusion features; and triggering a target convolution strategy based on the bird fusion features to output target bird fine-grained features.

[0054] The first objective of this invention is to provide a dynamic convolution method called DualAxis-Deform-Conv, which is used as the first convolution strategy. This enables the algorithm to obtain stronger feature representation capabilities when processing targets with deformation or irregular boundaries, and also achieves efficient capture of local deformation information in different directions.

[0055] Detailed process analysis of DualAxis-Deform-Conv (Dual-Axis Deformation-Convolution) design and feature fusion:

[0056] 1. Acquire bird image information, extract basic bird features from the bird image information, and predict dynamic offsets based on the basic bird features. Dynamic offset prediction (through 3x3 convolution + BN + tanh) limits the offset values ​​to [-1, 1].

[0057] For input features f∈R B×C×W×H ,

[0058] ① First, a 3x3 convolutional layer is used to predict the initial offset: Δ∈R B×(2K)×W×H , where K is the size of the convolution kernel.

[0059] ②Then the output is divided into two parts along the channel dimension: Δ=[Δ y Δ x The dimensions of each part are (B×K×W×H).

[0060] ③ Next, the initial offset values ​​are mapped to [-1, 1] using Batch Normalization and the tanh activation function to form dynamic offsets:

[0061] Δ←tanh(BN(Δ)).

[0062] 2. Construct a basic sampling network, such as Figure 1 As shown, the base sampling coordinates of the network are dynamically deformed based on the dynamic offset to generate the target sampling coordinates. This is done according to a preset convolutional kernel template (horizontal or vertical). The dynamic offset accumulates, adjusting the sampling coordinates.

[0063] For each pixel location (p, q), a basic sampling network is first constructed, which is based on the original center coordinates (x, q). center y center ) and preset convolution kernel templates:

[0064] ① Convolution kernel template

[0065] Depending on the direction of dynamic deformation:

[0066] (1) When morph = 1 (vertical direction):

[0067] The template fixes the y-component as 0 and the x-component as being in the interval [0, ... Uniform distribution within:

[0068] y grid＝0 ,

[0069] (2) When morph = 0 (horizontal direction):

[0070] The template fixes the x-component at 0, while the y-component is... Uniform distribution within:

[0071] x grid =0,

[0072] ② Dynamic offset (serpentine accumulation)

[0073] The base mesh is dynamically adjusted using dynamic offsets to generate target sampling coordinates. Taking the vertical direction (morph=1) as an example:

[0074] (1) For each convolution kernel point, its basic sampling coordinates are:

[0075] ybase =y center +0, x base =x center +x grid

[0076] (2) Apply a dynamic offset to the y-coordinate, such as Figure 4 As shown, with the center position of the convolution kernel Based on this standard, the following is stipulated:

[0077]

[0078] Simultaneously, the sampling coordinates are accumulated in the opposite direction (ensuring a fixed center position). Finally, the sampling coordinates are updated.

[0079]

[0080] This represents the dynamic offset of the vertical coordinate. pass Calculations show that extend_scope is used for scaling. The determined motion offset is used to adjust the base sampling coordinates in the vertical direction, thereby generating the target coordinates in the vertical direction, (x new y new ) represents the target sampling coordinates.

[0081] The x-coordinate remains unchanged:

[0082] x new =x base .

[0083] like Figure 3 As shown, for the horizontal direction (morph = 0), a similar process is used to dynamically offset the x-coordinate, and the formula is:

[0084]

[0085] The y-coordinate remains y new =y base .

[0086] in, This represents the dynamic offset of the horizontal coordinate. pass Calculations show that extend_scope is used to scale the scope defined by... The obtained dynamic offset is used to adjust the basic sampling coordinates in the horizontal direction, thereby generating the target coordinates in the horizontal direction, (x new y new ) represents the target sampling coordinates.

[0087] 3. Bilinear interpolation sampling is used to obtain enhanced bird features from the dynamically deformed bird feature sampling points.

[0088] Using target sampling coordinates (x) new y new Perform bilinear interpolation sampling on the input feature f:

[0089] in y1 = y0 + 1, x1 = x0 + 1, f(y new ,x new () represents the bird features after dynamic deformation. This sampling method ensures the continuity and smoothness of the feature map, making the dynamically deformed features more adaptable.

[0090] 4. Further convolution, normalization, and activation processing are performed on the dynamically deformed bird features to obtain enhanced bird features: After obtaining the dynamically deformed bird feature map:

[0091] (1) Call the convolution operation with kernel size (K,1) along the horizontal branch (morph=0).

[0092] (2) Call the convolution operation with kernel size (1,K) along the vertical branch (morph=1).

[0093] Next, the output is processed by Group Normalization (GN) and the ReLU activation function to enhance nonlinear representation and numerical stability.

[0094] 5. Fusion and Final Output

[0095] In the forward propagation of the DualAxis-Deform-Conv convolution method:

[0096] (1) Standard convolution branch: directly extract basic features.

[0097] (2) Dynamic snake convolution branch: The deformed features are generated by using the above dynamic offset and sampling mechanism in the horizontal (conv_x) and vertical (conv_y) directions respectively.

[0098] (3) Fusion: Fusion output, splicing three features to obtain a richer representation.

[0099] The general operating steps are as follows:

[0100] The constructor of DualAxisDeformConv defines three branches:

[0101] ① Standard convolution branch (conv_0): Extracts basic features using regular convolution (e.g., Conv(inc, ouc, k)) without involving dynamic offsets.

[0102] ② Dynamic convolution branches (two branches): conv_x and conv_y, whose internal mechanisms were originally implemented in DSConv, but are now directly embedded in DySnakeConv:

[0103] (1)conv_x: Performs dynamic sampling and convolution in the horizontal direction (along the x-axis), with parameter morph = 0 passed in;

[0104] (2)conv_y: Performs dynamic sampling and convolution in the vertical direction (along the y-axis), with parameter morph=1.

[0105] During forward propagation, the outputs of these three branches are concatenated along the channel dimension, as shown in the formula:

[0106] Output1=concat(conv_0(x), conv_x(x), conv_y(x));

[0107] This design retains the information from standard convolutions while incorporating adaptive dynamic features in both directions, giving the network a higher adaptability to local deformations and edge irregularities. The resulting fusion incorporates both the global information captured by standard convolutions and the sensitivity to local geometric deformations brought about by dynamic sampling.

[0108] This invention provides a Split-Efficient-Conv (SECV) convolution method that uses bird fusion features as the target convolution strategy to output fine-grained features of the target bird species. This method achieves both high computational efficiency and sufficient expressive power in feature representation at a low cost, while also fully utilizing redundant information, thus enabling efficient feature learning and optimization of computational resources.

[0109] A detailed breakdown of the Split-Efficient-Conv (SPE) convolution method: 1. Main convolution operation: generates initial features using a small convolution kernel; 2. Inexpensive convolution operation: generates additional features using a larger convolution kernel and depthwise convolution (group convolution); 3. Activation function: performs a non-linear transformation on the features after convolution; 4. Concatenate the features from the main convolution branch and the inexpensive convolution branch to obtain a comprehensive feature representation; 5. The final output is a feature representation with low computational cost, powerful functionality, low time complexity, and rich feature space.

[0110] Its general module structure is shown below:

[0111] The main features of the main convolution branch are generated based on the bird fusion features, and the inexpensive features of the inexpensive convolution branch are generated based on the bird fusion features. The main features and the inexpensive features are then concatenated to output the fine-grained features of the target bird.

[0112] The Split-Efficient-Conv module accepts the number of input channels c1, the number of output channels c2, the kernel size k, stride s, number of groups g, and activation flag act. The module primarily contains two convolution operations:

[0113] 1. Main convolution branch cV1

[0114] Used to generate the main features, its output channel number is:

[0115] When calling the function, the following parameters are used: input channel c1, output channel c′, kernel size k, stride s, number of groups g, and whether to activate act.

[0116] Mathematically, this can be described as: y = σ(Conv1(x; W1)), y ∈ R B×C′×H′×W′ ;

[0117] Where σ represents the activation function (if act = True) and is the convolution kernel parameter.

[0118] Calculate the main features

[0119] ① Input: Input features x∈R B×c1×H×W ;

[0120] ②Main convolution: Convolution operation is performed using cv1, y = Conv1(x).

[0121] Note: The parameters used here include the kernel size k and stride s, the number of groups g, and the number of output channels c′ = c2 / 2.

[0122] ③ Activation: If act is true, an activation function will be applied after convolution, giving the output non-linear expressive power.

[0123] 2. Inexpensive convolution branch cv2

[0124] To generate more ("ghost") features from the main feature y, it is designed with a large convolution kernel (5x5 in this case) and the idea of ​​depthwise convolution (the number of groups is set to the number of input channels, i.e., c_') to ensure the minimum computation.

[0125] The mathematical description is: z = σ(Conv2(y; W2)), z ∈ RB×C′×H′×W′ ;

[0126] Here, Conv2 uses a kernel size of 5, a stride of 1, and a group number of c′ to achieve an effect similar to depthwise convolution.

[0127] ① Inexpensive Convolution: After obtaining the main feature y, cv2 is used to process y, z = Conv2(y). Here, cv2 uses a large convolution kernel (5x5) and a stride of 1, plus a group number of 2 (i.e., depthwise convolution), thus greatly reducing computation. This part of the computation is mainly used to generate "ghost" features, i.e., inexpensive features, that are redundant with y but provide supplementary information.

[0128] ②Activation: Similarly, based on the act parameter, an activation function is applied after the convolution.

[0129] Finally, the module's output is formed by concatenating the features of the main convolutional branch and the cheap convolutional branch along the channel dimension:

[0130] Output2=concat(y,z),Output∈R B×(c′+c′)×H′×W′ =R B×c2×H′×W′ .

[0131] Finally, the main feature y and the cheap feature z are concatenated along the channel dimension, Output2 = concat(y, z). After this concatenation, the number of output channels is c′ + c′ = c2, which achieves the expected output dimension.

[0132] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention, and these will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.

Claims

1. A fine-grained bird detection algorithm based on snake-like deformable convolution and dilated convolution, characterized in that, The algorithm includes, S11: Acquire bird image information, extract basic bird features from the bird image information, and predict dynamic offset based on the basic bird features; S12: Construct a basic sampling network, and dynamically deform the basic sampling coordinates of the basic sampling network according to the dynamic offset to generate target sampling coordinates; S13: Sample the basic bird features using the target sampling coordinates to obtain the dynamically deformed bird features; S14: Enhanced bird features are obtained by processing the dynamically deformed bird features; S15: The enhanced bird features and basic bird features are fused to generate a bird fusion feature; S16: Trigger the target convolution strategy based on the bird fusion features to output fine-grained features of the target bird; In S12, the method for generating the target sampling coordinates is to set the original center coordinates and convolution kernel template to form a basic sampling network, obtain the basic sampling coordinates of the basic sampling network, and apply dynamic offsets to the horizontal and vertical coordinates of the basic sampling coordinates according to the direction of the dynamic deformation to generate the target sampling coordinates. In S13, bilinear interpolation sampling is performed on the basic bird features to obtain dynamically deformed bird features. The expression for the dynamically deformed bird features is as follows: ； in, , This represents the characteristics of birds after dynamic deformation; In S15, the expression for generating the bird fusion feature is: ； Output1 represents avian fusion features. Basic bird characteristics, To enhance the horizontal dimension of bird features, To enhance the vertical feature quantity of bird characteristics.

2. The fine-grained bird detection algorithm based on snake-like deformable convolution and dilated convolution according to claim 1, characterized in that, When the direction of the dynamic deformation is perpendicular to the vertical direction, a dynamic offset is applied to the vertical coordinate. The expression for applying the dynamic offset to the vertical coordinate is: ， ； ； ； ； Among them, (x base y base (x) represents the basic sampling coordinates. center y center () represents the original center coordinates, and the center position of the convolution kernel. When the dynamic deformation direction is set to vertical, the x-component of the convolution kernel template is in the interval Uniformly distributed inside , This represents the dynamic offset of the vertical coordinate. pass Calculations show that extend_scope Used for scaling The determined action offset, (x) new y new () represents the target sampling coordinates.

3. The fine-grained bird detection algorithm based on snake-like deformable convolution and dilated convolution according to claim 1, characterized in that, When the direction of the dynamic deformation is horizontal, a dynamic offset is applied to the horizontal coordinate. The expression for applying the dynamic offset to the horizontal coordinate is: ； ； ； Among them, (x base y base (x) represents the basic sampling coordinates. center y center () represents the original center coordinates, and the center position of the convolution kernel. When the dynamic deformation direction is set to horizontal, the y-component of the convolution kernel template is in the interval Uniformly distributed inside ; This represents the dynamic offset of the horizontal coordinate. pass Calculations show that extend_scope Used for scaling The obtained dynamic offset, (x) new y new () represents the target sampling coordinates.

4. The fine-grained bird detection algorithm based on snake-like deformable convolution and dilated convolution according to claim 1, characterized in that, In S11, the method for predicting the dynamic offset is to set a convolutional layer, predict the initial offset based on the convolutional layer and basic bird features, and map the initial offset value to the dynamic offset through the Batch Normalization data normalization function and the tanh activation function.

5. The fine-grained bird detection algorithm based on snake-like deformable convolution and dilated convolution according to claim 1, characterized in that, In S14, the dynamically deformed bird features are subjected to subsequent convolution, normalization, and activation processing to obtain enhanced bird features.

6. The fine-grained bird detection algorithm based on snake-like deformable convolution and dilated convolution according to claim 1, characterized in that, In S16, the target convolution strategy includes: S161: Generate the main features of the main convolutional branch based on the bird fusion features; S162: Generate inexpensive features for inexpensive convolutional branches based on the bird fusion features; S163: The main features and the low-cost features are spliced ​​together to output the fine-grained features of the target bird.

7. The fine-grained bird detection algorithm based on snake-like deformable convolution and dilated convolution according to claim 6, characterized in that, In S163, the expression for concatenating and outputting the fine-grained features of the target bird is: ； Where Output2 represents the fine-grained features of the target birds, y represents the main features, and z represents the inexpensive features. This represents the size of the fine-grained features of the target bird in the channel dimension.

Images