A flying bird detection method

Abstract

The application belongs to the technical field of target detection, and provides a flying bird detection method, which comprises the following steps: constructing a flying bird target detection data set and dividing the data set into a training set; the flying bird target detection data set comprises flying bird samples in different flight postures, sizes and backgrounds; a multi-scale real-time detection model is constructed, which is obtained by improving a YOLOv8 model; the multi-scale real-time detection model is trained based on the flying bird target detection data set to determine optimal parameters; and the to-be-tested data is input into the trained multi-scale real-time detection model to output the category and position information of the flying bird target. The application detects flying birds based on the trained multi-scale real-time detection model, significantly improves the accuracy of model detection and recognition while ensuring that the test rate reaches 178 FPS, has excellent average accuracy in complex flying bird scene detection applications, and can be applied to airport clearance area safety early warning to improve aviation safety.

Description

Technical Field

[0001] This invention relates to the field of target detection technology, and specifically to a method for detecting birds in flight. Background Technology

[0002] With the rapid development of intelligent ecological monitoring systems and aviation safety early warning technologies, real-time detection of flying birds has become a key technology for ensuring the safety of airport airspace, protecting biodiversity in wind farms, and monitoring agriculture and forestry. Unlike conventional static or slow-moving target detection, flying birds are characterized by high speed, variable flight attitude (non-rigid body deformation), complex background environments (such as cloud cover, forests, and sea surface interference), and a very small proportion in the field of view (mostly small targets). This poses a significant challenge to the real-time performance and accuracy of target detection algorithms.

[0003] Currently, mainstream real-time object detection algorithms are mainly divided into two categories: one is detectors based on convolutional neural networks, represented by the YOLO series; the other is end-to-end detectors based on the Transformer architecture, represented by RT-DETR (Real-Time Detection Transformer). CNN detectors, such as YOLOv5 and YOLOv8, are widely used in industry, and their excellent inference speed and multi-scale feature pyramid structure have solved the problem of detecting general objects to a certain extent. However, these algorithms usually rely heavily on non-maximum suppression (NMS) post-processing to remove redundant prediction boxes. In scenarios with dense flocks of birds, NMS operations are prone to missing overlapping targets, and their inference latency fluctuates with the number of targets, making it difficult to meet the stringent requirements of stable low latency for aviation early warning. In addition, CNNs are limited by their local receptive field, and when faced with small, distant targets like birds in cluttered backgrounds, they often struggle to capture global contextual information, resulting in a high false detection rate. To overcome the limitations of CNNs, RT-DETR achieves end-to-end NMS-free detection by introducing a Transformer architecture and an efficient hybrid encoder, maintaining high accuracy while also providing real-time inference capabilities. Although RT-DETR performs exceptionally well on general datasets, it still reveals three key technical bottlenecks when directly applied to complex dynamic bird detection scenarios: first, the backbone network's ability to extract subtle features from high-speed moving targets is insufficient; second, the static position encoding method struggles to adapt to the complex, non-rigid deformations of birds; and third, the problem of information dissipation from small targets is severe during deep feature fusion.

[0004] In summary, there is an urgent need to develop a real-time bird detection method optimized for complex dynamic scenes, which can effectively preserve the key features of small targets while overcoming motion blur and adapting to changes in non-rigid body posture, thereby achieving high-precision, low-latency end-to-end detection. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a bird detection method that improves detection accuracy while ensuring real-time detection speed, thus enabling the detection of birds in complex scenarios.

[0006] The present invention provides a method for detecting birds, comprising: Construct a bird target detection dataset and divide it into a training set and a dataset; the bird target detection dataset includes bird samples with different flight postures, sizes and backgrounds; A multi-scale real-time detection model was constructed. This model was obtained by improving upon the YOLOv8 model, including: The backbone network uses ResNet-18. The base block of ResNet-18 consists of four cascaded 3×3 convolutional layers. The first 3×3 convolutional layer in all base blocks is replaced with a REDConv module to extract multi-scale base features of flying bird targets. After the high-level features output by the backbone network, a DPA-Encoder module is built to replace the traditional static position embedding. The DPA-Encoder module is used to enhance the model's ability to perceive the dynamic position information of birds. The neck network is constructed with a C3Hyper module and a GateAdd structure. The C3Hyper module is used to achieve adaptive fusion of deep and shallow features, purify and retain key features, and the GateAdd structure guides and optimizes feature transfer through full-path distillation. The multi-scale real-time detection model is trained based on the aforementioned bird target detection dataset to determine the optimal parameters; The test data is input into the trained multi-scale real-time detection model, which outputs the category and location information of the flying bird target.

[0007] Optionally, the REDConv module includes two modes: training mode and inference mode. The training mode of the REDConv module includes: For the input feature map X red_in Execute branch operations separately: Branch 1 is derived from the input feature map X red_in With a parameter W std By concatenating standard convolution operators, we obtain the appearance semantic feature map X. std ; Branch 2 is based on the input feature map X red_in With a parameter W cd The central difference convolution operators are concatenated to obtain a fine-grained texture feature map X. cd ; Branch 3 is derived from the input feature map X red_in With a parameter Whd The horizontal difference convolution operators are concatenated to obtain the horizontal gradient feature map X. hd ; Branch 4 is derived from the input feature map X red_in With a parameter W vd The vertical difference convolution operators are concatenated to obtain the vertical gradient feature map X. vd ; Branch 5 is derived from the input feature map X red_in With a parameter W ad The adaptive angle difference convolution operators are concatenated to obtain multi-angle pose feature maps X. ad ; Feature fusion operation: from appearance semantic feature map X std Fine-grained texture feature map X cd Horizontal gradient feature map X hd Vertical gradient feature map X vd and multi-angle pose feature map X ad Performing a bitwise addition operation yields a multidimensional fused feature map X. sum ; Output: Multidimensional fused feature map X sum The REDConv module is sequentially concatenated with a batch normalization module and a ReLU activation function to obtain the final output feature map X. red_out ; Among them, the standard convolution operator W std For learnable regular convolution operations with a kernel size of 3×3; central difference convolution operator W cd The convolution operation is used to calculate the intensity difference between the center pixel and its surrounding neighboring pixels; the horizontal difference convolution operator W hd To perform a convolution operation for detecting vertical edges using a horizontal gradient template; the vertical difference convolution operator W vd To perform convolution operations for detecting horizontal edges using a vertical gradient template; an adaptive angle difference convolution operator W ad This is a convolution operation for detecting slanted edges using a gradient template along the diagonal direction.

[0008] Optionally, the REDConv module in the inference state includes a convolutional kernel with parameters W. rep A single-channel 3×3 convolution module; W rep By W std W cd W hd W vd and W ad Obtained by performing a bitwise addition operation.

[0009] As can be seen from the above technical solution, by setting standard convolution and differential convolution in parallel, feature maps are learned separately during the training phase, and then fused into a single convolution structure through reparameterization during the inference phase, while ensuring feature extraction capability and reducing inference latency.

[0010] Optionally, the DPA-Encoder module comprises N DPA-Encoder layers stacked in series, and the DPA-Encoder layer includes: The first residual connection structure is derived from the input feature map X. enc_in The output X of the position-aware self-attention module attn Perform a bitwise addition operation, then concatenate it with the first layer normalization module LayerNorm to obtain the intermediate feature map X. mid ; The second residual connection structure is composed of the intermediate feature map X. mid The output of the feedforward neural network (FFN) is bitwise added to the output of the second layer normalization module (LayerNorm) to obtain the final output feature map X of the DPA-Encoder layer. dpa-out ; The feedforward neural network (FFN) consists of two cascaded linear layers and a GeLU activation function located between the linear layers; the first and second layer normalization modules (LayerNorm) are used for feature value normalization.

[0011] Optionally, the position-aware self-attention module includes: Constructing the query projection layer W Q , key projection layer W K Sum projection layer W V All are linear fully connected layers; The query matrix Q, key matrix K, and value matrix V are constructed from the input feature map X. enc_in With query projection layer W Q , key projection layer W K and value projection layer W V Obtained by connecting in series; Perform matrix multiplication on the transpose of the query matrix Q and the key matrix K, and divide by the scaling factor. The basic attention score matrix Score is obtained. base ; Based on the basic attention score matrix Score base With dynamic position bias matrix P bias Performing a bitwise addition operation yields the corrected attention score matrix, Score. final ; The final attention output X is obtained by concatenating the corrected attention score matrix Score_final with the Softmax normalization function and the value matrix V. attn .

[0012] Optionally, the dynamic position bias matrix P bias The output of the dynamic location regression subnetwork is the dynamic location regression subnetwork, which includes: Construct a global context aggregation operator as a global average pooling operation to process feature map X. enc_in The spatial dimension is compressed to 1×1, resulting in the channel descriptor X. avg ; Define a location feature mapping network, which includes a first fully connected layer, a ReLU activation function, and a second fully connected layer, which are sequentially connected in series. Channel descriptor X avg The input is fed into the location feature mapping network, and the output is a dynamic location bias vector P corresponding to the number of attention heads. bias_vec Define the broadcast operation to dynamically adjust the position bias vector P. bias_vec The matrix is ​​replicated and expanded in the spatial dimension to generate a dynamic position bias matrix P with the same size as the self-attention matrix. bias .

[0013] As can be seen from the above technical solution, the DPA-Encoder module dynamically adjusts the encoding parameters based on the real-time position changes of the bird target, and improves the model's adaptability to changes in bird posture and displacement through adaptive association between position information and feature information.

[0014] Optionally, the C3Hyper module performs the following processes, including: The input feature map is X C3H_in ; From the input feature map X C3H_in It is concatenated with a 1×1 convolutional module for dimensionality reduction and then segmented along the channel dimension into the first component X. split1 Second component X split2 ; From the second component X split2 The global context feature map X is obtained by sequentially concatenating node projection, hyperedge convolution, and node reprojection operations. global The node projection operation maps pixel features to hypergraph nodes, while the hyperedge convolution operation aggregates node information using the correlation matrix. From the second component X split2 It is composed of N cascaded depthwise separable convolutional modules DSConv, used to extract local texture features, and obtain local texture feature map X. local ; From the first component X split1Global Context Feature Map X global and local texture feature map X local Perform channel concatenation, then concatenate with a 1×1 convolutional module to obtain the final enhanced feature map X of the C3Hyper module. enhanced .

[0015] As can be seen from the above technical solution, the C3Hyper module models the contextual information of the deep and shallow features output by the backbone network, and integrates the semantic and detailed information of features at different levels through an adaptive weight allocation mechanism, suppressing background interference features and strengthening the key features of the flying bird target.

[0016] Optionally, the GateAdd structure includes a first input and a second input, wherein the first input receives the enhanced feature map X. enhanced The second input receives the shallow, fine-grained feature map X from the backbone network. shallow ; Shallow fine-grained feature map X at the second input end shallow The spatial resolution is adjusted to match the enhanced feature map X at the first input. enhanced Consistency is achieved, resulting in the aligned feature map X. aligned ; A learnable scalar parameter Gate, initialized to 0, and an aligned feature map X aligned Performing a scalar multiplication operation yields a weighted fine-grained feature map X. weighted ; Enhanced feature map X from the first input terminal enhanced With weighted fine-grained feature map X weighted Performing bitwise addition enables adaptive fusion of C3Hyper enhanced features and shallow fine-grained features, resulting in a fused feature map X containing cross-scale information. gate_out .

[0017] As can be seen from the above technical solution, the GateAdd structure design is a distillation architecture for teacher and student networks. It guides the student network to learn the feature representation of the teacher network through full-path feature transfer, adaptively adjusts the distillation weights, and balances the effectiveness of feature distillation with the efficiency of model inference.

[0018] Optionally, the output of the backbone network includes Stage 3 feature map P3, Stage 4 feature map P4, and Stage 5 feature map P5; the neck network receives the output of the backbone network and performs the following operations: The Stage 5 feature map P5 is concatenated with the C3Hyper module to obtain the high-level semantic feature map P5. _enc ; Perform cross-scale adaptive distillation to enhance the high-level semantic feature map P5. encConnect the Stage3 feature map P3 to the first input of the GateAdd structure, connect Stage3 to the second input of the GateAdd structure, perform a distillation and fusion operation, and use a gating mechanism to inject the fine-grained texture information of P3 into P5. enc Output distillation feature map P5 containing cross-scale information. distilled ; Perform multi-scale fusion and output, define a top-down path, and integrate P5. distilled After performing an upsampling operation, it is concatenated with the Stage 4 feature map P4 through the execution channel, and then processed by a C3Hyper module to obtain the fused feature map N4. After performing an upsampling operation on N4, it is concatenated with the Stage 3 feature map P3 through the execution channel, and then processed by a C3Hyper module to obtain the fused feature map N3. The downsampling operation performed by N3 is concatenated with the channel operation performed by N4, and then cascaded with a C3Hyper module to obtain the small target detection output feature map Out. small , will Out small Continue performing downsampling operations and P5 distilled The process involves concatenating the execution channels and then linking them with a C3Hyper module to obtain the output feature map for medium to large target detection. large .

[0019] By adopting the above technical solution, this application has the following beneficial effects: The backbone network of the model was constructed using the built REDConv module, namely Reparameterized Differential Convolution. By extracting blurred edge features through multidimensional difference, an average accuracy improvement of 1.9% was achieved in complex bird detection applications. The model's encoder is built using the constructed DPA-Encoder module (i.e., Dynamic Anchor Boxes Encoder), and the model's neck structure is built in conjunction with the constructed C3Hyper module (i.e., C3 Hypergraph Context) and the GatedAdd structure (i.e., Gated Adaptive Distillation), which effectively solves the problems of non-rigid deformation and feature dilution of small targets. Using the REDConv module, DPA-Encoder module, and neck fusion structure, a multi-scale real-time detection model was obtained. The model parameters were determined using a constructed complex scene bird dataset, and finally, the trained multi-scale real-time detection model was obtained. Based on the trained multi-scale real-time detection model, the accuracy of model detection and recognition is significantly improved while ensuring a test rate of 178 FPS. In the application of complex bird detection, it achieved an excellent average accuracy (mAP@50) of 91.0%, which is 2.7% higher than the mainstream baseline model. It can be applied to improve aviation safety level in airport airspace safety early warning. Attached Figure Description

[0020] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0021] Figure 1 A flowchart of a bird detection method provided by an embodiment of the present invention is shown; Figure 2 A schematic diagram of the multi-scale real-time detection model provided in an embodiment of the present invention is shown; Figure 3 A schematic diagram of the REDConv module provided in an embodiment of the present invention is shown; Figure 4 A schematic diagram of the DPA-Encoder layer provided in an embodiment of the present invention is shown; Figure 5 This diagram illustrates the C3Hyper module and GateAdd structure provided in an embodiment of the present invention. Figure 6 A comparison chart of the loss values ​​of the multi-scale real-time detection model provided in this embodiment of the invention and other models is shown. Figure 7 The figure shows a comparison of the performance parameters of the multi-scale real-time detection model provided in this embodiment of the invention with other models; Figure 8 The diagram shows a comparison of the visualization effects of the multi-scale real-time detection model provided in this embodiment of the invention with other models. Detailed Implementation

[0022] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of the present invention and are therefore merely examples, and should not be construed as limiting the scope of protection of the present invention. It should be noted that, unless otherwise stated, the technical or scientific terms used in this application should have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0023] In one embodiment, such as Figure 1As shown, a bird detection method is provided, including: S1. Construct a bird target detection dataset and divide it into a training set and a dataset; the bird target detection dataset includes bird samples with different flight postures, sizes and backgrounds.

[0024] This embodiment first extracts images of birds from a public dataset. The resulting bird dataset, along with its labels, forms the initial dataset G1. Considering the complexity of the environment (such as fog, backlighting, and cluttered backgrounds) and the small sample size of birds, this invention performs image enhancement processing on G1 to obtain the enhanced dataset G2.

[0025] S110. The complex scene bird dataset G1 is augmented using image rotation and brightness enhancement methods. The enhancement methods include image rotation, brightness adjustment, scale scaling and flipping operations to expand the diversity of the dataset samples and obtain the complex scene bird augmentation dataset G2. S120. Define the feature maps of the complex scene bird dataset G2: Feature_maps={{X1 ,C1 ,Stage1},{X2,C2,Stage2} ,…,{Xs ,Cs ,Stages} ,…,{XQ ,CQ ,StageQ}} ,s∈[1 ,Q] ,Q is a natural number, where XQ is the feature map of the Qth stage of the network model, CQ is the number of channels of XQ, and StageQ is the Qth stage of the network model.

[0026] S2. Construct a multi-scale real-time detection model; such as... Figure 2 As shown, the multi-scale real-time detection model is obtained by improving upon the YOLOv8 model, including: S210. The backbone network uses ResNet-18. The basic block of ResNet-18 consists of four cascaded 3×3 convolutional layers. To address the issue of blurred bird textures, the first 3×3 convolutional layer in each basic block is replaced with a REDConv module to extract multi-scale basic features of the bird target. The REDConv module is as follows: Figure 3 As shown, the REDConv module includes two modes: training mode and inference mode. The training mode of the REDConv module includes: For the input feature map X red_in Execute branch operations separately: Branch 1 is derived from the input feature map X red_in With a parameter W std By concatenating standard convolution operators, we obtain the appearance semantic feature map X. std ; Branch 2 is based on the input feature map X red_in With a parameter W cdThe central difference convolution operators are concatenated to obtain a fine-grained texture feature map X. cd ; Branch 3 is derived from the input feature map X red_in With a parameter W hd The horizontal difference convolution operators are concatenated to obtain the horizontal gradient feature map X. hd ; Branch 4 is derived from the input feature map X red_in With a parameter W vd The vertical difference convolution operators are concatenated to obtain the vertical gradient feature map X. vd ; Branch 5 is derived from the input feature map X red_in With a parameter W ad The adaptive angle difference convolution operators are concatenated to obtain multi-angle pose feature maps X. ad ; Feature fusion operation: from appearance semantic feature map X std Fine-grained texture feature map X cd Horizontal gradient feature map X hd Vertical gradient feature map X vd and multi-angle pose feature map X ad Performing a bitwise addition operation yields a multidimensional fused feature map X. sum ; Output: Multidimensional fused feature map X sum The REDConv module is sequentially concatenated with a batch normalization module and a ReLU activation function to obtain the final output feature map X. red_out ; Among them, the standard convolution operator W std For learnable regular convolution operations with a kernel size of 3×3; central difference convolution operator W cd The convolution operation is used to calculate the intensity difference between the center pixel and its surrounding neighboring pixels; the horizontal difference convolution operator W hd To perform a convolution operation for detecting vertical edges using a horizontal gradient template; the vertical difference convolution operator W vd To perform convolution operations for detecting horizontal edges using a vertical gradient template; an adaptive angle difference convolution operator W ad This is a convolution operation for detecting slanted edges using a gradient template along the diagonal direction.

[0027] The inference state REDConv module includes a convolution kernel with parameters W. rep A single-channel 3×3 convolution module; W rep By W std W cd W hd W vd and W adThe result is obtained by performing a bitwise addition operation, calculated as follows: .

[0028] Then the backbone network is as follows Figure 2 As shown, the input image is processed sequentially through four stages: Layer1, Layer2, Layer3, and Layer4, which include the REDConv module. Multi-scale feature maps with scales of 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image are extracted and used as inputs for the subsequent encoder and neck network.

[0029] The mathematical model for the input image feature extraction process can be represented as a cascaded function mapping: , , , , In the formula: This represents the input image, where H and W are the height and width of the image, respectively. Indicates the initial downsampling operation; This represents the residual stage processing function containing the REDConv module; F1, F2, F3, and F4 are the multi-scale feature maps output from the four stages, with dimensions respectively. , , (where C) i (where is the number of channels), corresponding to 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the input image size, respectively, providing rich multi-scale anti-interference features for the subsequent encoder and neck network.

[0030] S220. After the high-level features output by the backbone network, a DPA-Encoder module is constructed to replace the traditional static position embedding. The DPA-Encoder module is used to enhance the model's ability to perceive the dynamic position information of birds.

[0031] S221. Construct a dynamic location regression subnetwork: The input feature map is X enc_in ; Construct a global context aggregation operator as a global average pooling operation to process feature map X. enc_in The spatial dimension is compressed to 1×1, resulting in the channel descriptor X. avg ; ; Define a location feature mapping network, which includes a first fully connected layer, a ReLU activation function, and a second fully connected layer, which are sequentially connected in series. Channel descriptor Xavg The input is fed into a location feature mapping network, and the output is a dynamic position bias vector P corresponding to the number of attention heads. bias_vec ; ; The dynamic position bias vector P bias_vec The matrix is ​​replicated and expanded in the spatial dimension to generate a dynamic position bias matrix P with the same size as the self-attention matrix. bias ; .

[0032] S222. Construct a position-aware self-attention module: Constructing the query projection layer W Q , key projection layer W K Sum projection layer W V All are linear fully connected layers; The query matrix Q, key matrix K, and value matrix V are constructed from the input feature map X. enc_in With query projection layer W Q , key projection layer W K and value projection layer W V Obtained by connecting in series; ; Perform matrix multiplication on the transpose of the query matrix Q and the key matrix K, and divide by the scaling factor. The basic attention score matrix Score is obtained. base ;in, ; Based on the basic attention score matrix Score base With dynamic position bias matrix P bias Performing a bitwise addition operation yields the corrected attention score matrix, Score. final , ; The final attention output X is obtained by concatenating the corrected attention score matrix Score_final with the Softmax normalization function and the value matrix V. attn ; .

[0033] S223. For example Figure 4 As shown, construct the DPA-Encoder layer: The first residual connection structure is derived from the input feature map X. enc_in The output X of the position-aware self-attention module attn Perform a bitwise addition operation, then concatenate it with the first layer normalization module LayerNorm to obtain the intermediate feature map X. mid ; ; The second residual connection structure is composed of the intermediate feature map X. midThe output of the feedforward neural network (FFN) is bitwise added to the output of the second layer normalization module (LayerNorm) to obtain the final output feature map X of the DPA-Encoder layer. dpa-out ; ; The feedforward neural network (FFN) consists of two linear layers connected in series with an intermediate GeLU activation function. W1 represents the weight matrix of the first fully connected layer (expansion layer) in the feedforward neural network, W2 represents the weight matrix of the second fully connected layer (projection layer), and b1 and b2 are the corresponding bias vectors; the first and second layer normalization modules LayerNorm are used for feature value standardization.

[0034] The S224.DPA-Encoder module consists of N DPA-Encoder layers stacked in series. The feature encoding process involves the multi-scale feature map output from the backbone network being processed sequentially through these N DPA-Encoder layers. The dynamic position bias generated by each layer continuously corrects the model's sensitivity to changes in bird posture, ultimately outputting an encoded feature map X containing rich global semantics and dynamic position information. enc This serves as the input for the subsequent neck network.

[0035] In this embodiment, N is set to 6.

[0036] S230. For example Figure 5 As shown, the neck network constructs a C3Hyper module and a GateAdd structure. The C3Hyper module is used to achieve adaptive fusion of features from deep and shallow layers, purifying and retaining key features. The GateAdd structure guides and optimizes feature propagation through full-path distillation, such as... Figure 5 As shown, it includes: S231. Constructing the Hybrid Adaptive Context Coding C3Hyper Module: The C3Hyper module performs the following procedures, including: The input feature map is X C3H_in ; From the input feature map X C3H_in It is concatenated with a 1×1 convolutional module for dimensionality reduction and then segmented along the channel dimension into the first component X. split1 Second component X split2 ; From the second component X split2 The global context feature map X is obtained by sequentially concatenating node projection, hyperedge convolution, and node reprojection operations. global The node projection operation maps pixel features to hypergraph nodes, while the hyperedge convolution operation aggregates node information using the correlation matrix. From the second component X split2It is composed of N cascaded depthwise separable convolutional modules DSConv, used to extract local texture features, and obtain local texture feature map X. local ; From the first component X split1 Global Context Feature Map X global and local texture feature map X local Perform channel concatenation, then concatenate with a 1×1 convolutional module to obtain the final enhanced feature map X of the C3Hyper module. enhanced .

[0037] S232. Constructing the full-path adaptive distillation GateAdd structure: The implementation process of the full-path adaptive distillation GateAdd structure is as follows: design the distillation architecture of the teacher network and the student network, guide the student network to learn the feature representation of the teacher network through full-path feature transfer, adaptively adjust the distillation weights, and balance the effectiveness of feature distillation with the model inference efficiency.

[0038] Specifically, the GateAdd structure includes a first input and a second input, wherein the first input receives the enhanced feature map X. enhanced The second input receives the shallow, fine-grained feature map X from the backbone network. shallow ; Shallow fine-grained feature map X at the second input end shallow The spatial resolution is adjusted to match the enhanced feature map X at the first input. enhanced Consistency is achieved, resulting in the aligned feature map X. aligned ; A learnable scalar parameter Gate, initialized to 0, and an aligned feature map X aligned Performing a scalar multiplication operation yields a weighted fine-grained feature map X. weighted ; Enhanced feature map X from the first input terminal enhanced With weighted fine-grained feature map X weighted Performing bitwise addition enables adaptive fusion of C3Hyper enhanced features and shallow fine-grained features, resulting in a fused feature map X containing cross-scale information. gate_out .

[0039] S233. Construct a neck feature fusion network: The backbone network outputs include Stage 3 feature maps P3 (Layer 5, 80x80), Stage 4 feature maps P4 (Layer 6, 40x40), and Stage 5 feature maps P5 (Layer 7, 20x20). These feature maps preserve the enhanced bird feather texture and contour details. The neck network receives the backbone network outputs and performs the following operations: The Stage 5 feature map P5 is concatenated with the C3Hyper module to obtain the high-level semantic feature map P5. _enc ; Perform cross-scale adaptive distillation to enhance the high-level semantic feature map P5. enc Connect the Stage3 feature map P3 to the first input of the GateAdd structure, connect Stage3 to the second input of the GateAdd structure, perform a distillation and fusion operation, and use a gating mechanism to inject the fine-grained texture information of P3 into P5. enc Output distillation feature map P5 containing cross-scale information. distilled ; Perform multi-scale fusion and output, define a top-down path, and integrate P5. distilled After performing an upsampling operation, it is concatenated with the Stage 4 feature map P4 through the execution channel, and then processed by a C3Hyper module to obtain the fused feature map N4. After performing an upsampling operation on N4, it is concatenated with the Stage 3 feature map P3 through the execution channel, and then processed by a C3Hyper module to obtain the fused feature map N3. The downsampling operation performed by N3 is concatenated with the channel operation performed by N4, and then cascaded with a C3Hyper module to obtain the small target detection output feature map Out. small , will Out small Continue performing downsampling operations and P5 distilled The process involves concatenating the execution channels and then linking them with a C3Hyper module to obtain the output feature map for medium to large target detection. large .

[0040] In summary, the model construction is complete, and the network structure parameters are shown in Table 1.

[0041] First, the constructed REDConv module is used to replace the standard convolutional module in the ResNet-18 backbone network to construct the backbone network of the new model, i.e., layers 0 to 8, which are used to suppress environmental noise and enhance texture response in the early stage of feature extraction. Secondly, using the constructed DPA-Encoder module, namely Dynamic Anchor Boxes Encoder, the encoder structure of the model is constructed, namely, 9 layers, to generate dynamic position biases to adapt to the non-rigid deformation of the bird. Finally, using the constructed C3Hyper module, namely C3 Hypergraph Context (hybrid adaptive context encoding), and the constructed GateAdd structure, namely Gated Adaptive Distillation (full path adaptive distillation), a neck feature fusion network of the model is constructed, namely, 10 to 36 layers, to achieve global context aggregation and lossless transmission across layers of multi-scale features.

[0042] Table 1

[0043] S3. Train the multi-scale real-time detection model based on the bird target detection dataset to determine the optimal parameters.

[0044] On a multi-scale real-time detection model, bird detection and bounding box regression are performed on a bird target detection dataset in complex scenes, and the predicted dataset R1 of bird detection results in complex scenes is output. The input to the multi-scale real-time detection model is a keyframe from a video stream or a still image in a complex scene. The input is then normalized to obtain a standard input tensor X. in_test .

[0045] The entire network cascading reasoning process is as follows: Feature extraction stage: from standard input tensor X in_test The input is fed into the S2 backbone network, and the REDConv module is used to extract multi-scale anti-interference feature maps. In the encoding enhancement stage, the features output from the backbone network are processed by the DPA-Encoder module described in S3, and the dynamic position bias injection mechanism is used to output an encoded feature map containing bird posture information. The feature fusion stage starts by inputting the encoded feature map into the neck network, which then undergoes global context aggregation via the C3Hyper module and full-path adaptive distillation via the GateAdd structure, outputting a final predicted feature map P containing multi-scale semantics. out .

[0046] Decouple the detection head prediction process: Define a detection head that receives fused features from the neck network and initializes a fixed number (e.g., 300) of learnable object query vectors. The classification prediction process involves the interaction between the target query vector and the fused features, which are then input into the classification branch. The output is the confidence score S for the bird category corresponding to each query vector. core_cls ; The regression prediction process involves the interaction between the target query vector and the fused features, which are then input into the regression branch. The output is the predicted bounding box coordinates Box_pred corresponding to each query vector. Note: Thanks to the One-to-One Bipartite Matching mechanism used in the training phase, each real target corresponds to only one high-confidence prediction query, so the output results naturally do not have overlap or redundancy.

[0047] End-to-end filtering process: No non-maximum suppression (NMS) operation is required. All prediction results output by S5.3 are directly filtered by confidence threshold (e.g., retaining prediction boxes with Score_cls greater than 0.5) to obtain the final prediction dataset R1. Coordinate mapping process: Map the normalized predicted bounding box coordinates in R1 back to the pixel coordinate system of the original video stream keyframes; Visualization output: Draw bounding boxes and category labels on the original image based on the mapped coordinates to complete end-to-end real-time detection of birds in complex scenes.

[0048] Model training process: S310. Initialize the network parameters of the multi-scale real-time detection model, and set training hyperparameters such as the number of training rounds, batch size, initial learning rate, and weight decay coefficient.

[0049] The model parameters of the multi-scale real-time detection model and the benchmark model RT-DETR-R18 obtained by training in this embodiment are adjusted. According to the prior parameters, the initial settings of the model parameters are: 200 training rounds, batch size of 16, initial learning rate of 0.01, minimum learning rate of 0.001, and weight decay coefficient of 0.005.

[0050] S320. Input the training set into the model for iterative training, calculate the error between the prediction result and the true label using a preset loss function, and update the model parameters through the backpropagation algorithm.

[0051] S330. After each preset number of iterations, the model performance is evaluated using the validation set, with average accuracy (mAP@50) as the core evaluation metric, and the weight file of the model with the best performance is saved.

[0052] The constructed enhanced training and validation sets are input into the multi-scale real-time detection model and the baseline model RT-DETR-R18 for training and testing. During the training process, the loss function descent curve and the change in validation set accuracy are monitored in real time. The optimal weight files of the multi-scale real-time detection model and RT-DETR-R18 model parameters are extracted during the training process. Specifically, the optimal weight file of the multi-scale real-time detection model (MS-RT-DETR model) is P1, and the optimal weight file of the YOLOv8 model is P2, which are saved separately.

[0053] S340. Use the test set to perform performance testing on the optimal model, verify the model's real-time detection accuracy and inference speed in the airport scenario, ensure that the average accuracy mAP@50 reaches 91%, and that the inference speed meets the requirement of 30FPS for real-time monitoring.

[0054] Comparing the training and validation results revealed that an excessively large initial learning rate caused model convergence oscillations, and the number of training epochs was insufficient to fully fit the deep features. After multiple experiments to fine-tune the model parameters, the optimal parameters were determined to be: 400 training epochs, batch size of 16, initial learning rate of 0.0001, minimum learning rate of 0.0001, weight decay coefficient of 0.0001, and a cosine annealing strategy was used to dynamically adjust the learning rate. Figure 6 As shown, the MS-RT-DETR model, which is the multi-scale real-time detection model trained in this embodiment, is compared with the loss of other models.

[0055] The method described in this embodiment was verified and tested on the NVIDIA RTX 3080 hardware computing platform. The experimental results show that, thanks to the reparameterization design of REDConv and the cross-scale lossless transmission of GateAdd, the method provided in this embodiment achieves an average accuracy (mAP@50) of 91.0% for bird detection in complex scenes, which is 2.7% higher than the benchmark model RT-DETR. At the same time, the model inference speed reaches 178 FPS (frames per second), which is better than the existing mainstream algorithms and meets the engineering requirements for real-time monitoring and early warning.

[0056] S4. Input the data to be tested into the trained multi-scale real-time detection model, and output the category and location information of the flying bird target.

[0057] Extract the data to be tested to obtain continuous data images to be detected; input the data images to be detected into the trained multi-scale real-time detection model, and the model output layer generates the position coordinates and category confidence of the bird target through bounding box regression and category prediction. Prediction results with confidence scores higher than a preset threshold are retained, and the final bird detection result is output.

[0058] To better illustrate the application effect of the method provided in this embodiment, a comprehensive quantitative evaluation of the multi-scale real-time detection model obtained through training is presented below.

[0059] The performance evaluation metrics for the model include two aspects: detection speed and detection accuracy. This embodiment uses internationally recognized evaluation metrics to comprehensively assess performance, including precision (P), recall (R), mean average precision (mAP@50), mean high-threshold average precision (mAP@50-95), and frames per second (FPS). Each metric has a different emphasis: Precision P: Represents the proportion of targets predicted as positive that meet the conditions of true positive samples, mainly measuring the degree of false positives of the model; Recall R: Represents the proportion of true positive class targets that are accurately predicted by the model, mainly measuring the degree of false negatives in the model; Average Precision (AP): This is the average precision at different confidence thresholds. It is measured by calculating the degree of overlap between detected targets and real targets (using IoU, Intersection over Union). The AP value is numerically equal to the area under the precision-recall (PR) curve. Mean Accuracy (mAP): This is the average of AP across all categories and is the most important indicator for evaluating the overall performance of an object detection system. Specifically, mAP@50 represents the mAP value at an IoU threshold of 0.5, reflecting the model's basic recognition capability; mAP@50-95 represents the average mAP calculated within the range of IoU thresholds from 0.5 to 0.95 (step size 0.05), used to evaluate the model's robustness and accuracy under high-precision positioning requirements.

[0060] Frames per second (FPS): This refers to the number of image frames that the model can process per unit of time (1 second), and is a core indicator for measuring system throughput. When the FPS value is greater than 30, the system is generally considered to meet the requirements for real-time smooth detection.

[0061] Precision P:

[0062] Recall rate R:

[0063] Average accuracy mAP50:

[0064] Frames per second (FPS):

[0065]

[0066] Where TP refers to the number of samples that are positive and predicted as positive, FP refers to the number of samples that are negative and predicted as positive, FN refers to the number of samples that are positive and predicted as negative, u is the number of detected target classes, and APs is the AP of the s-th target class. T total This represents the total processing time for a single frame of image (in milliseconds, ms). Under the test conditions of this embodiment (NVIDIA RTX 3080), the average total processing time of the multi-scale real-time detection model is approximately 5.6ms, which translates to approximately 178 FPS, meeting the real-time processing requirements of high frame rate video streams for video and surveillance.

[0067] This embodiment also included comparative experiments to verify the optimization effects of the constructed REDConv module, DPA-Encoder module, and C3Hyper module & GateAdd structure, further evaluating the performance of the multi-scale real-time detection model. Compared with the benchmark model RT-DETR-R18, the multi-scale real-time detection model significantly improved the accuracy of the algorithm in detecting distant small flying birds in complex scenes while ensuring that the testing rate meets real-time requirements.

[0068] like Figure 7 and Figure 8 As shown in the experimental data, the multi-scale real-time detection model provided in this embodiment effectively suppresses complex background noise such as clouds and trees by introducing the REDConv reparameterized differential convolution module; it solves the positioning deviation caused by the non-rigid deformation of birds by introducing the DPA-Encoder dynamic position-aware encoder; and it overcomes the problem of feature dilution of small targets at multiple scales by constructing a neck feature fusion network containing the C3Hyper module and the GateAdd structure. Finally, on the self-built bird target detection dataset, it achieved an excellent performance with an average accuracy (mAP@50) of 91.0%, which is 2.7% higher than the benchmark model, while maintaining an inference speed of 178 FPS, achieving the best balance between accuracy and speed. It can be widely used in the construction of smart civil aviation, such as airport bird strike prevention and low-altitude airspace safety early warning, effectively improving the level of aviation operation safety.

[0069] The above embodiments are only used to provide a detailed description of the technical solutions of this application. However, the descriptions of the above embodiments are only for the purpose of helping to understand the methods of the embodiments of the present invention and should not be construed as limiting the embodiments of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art should be covered within the protection scope of the embodiments of the present invention.

Claims

1. A method for detecting birds in flight, characterized in that, include: Construct a bird target detection dataset and divide it into a training set and a dataset; The bird target detection dataset includes bird samples with different flight postures, sizes, and backgrounds; Construct a multi-scale real-time detection model; The multi-scale real-time detection model is obtained by improving upon the YOLOv8 model, including: The backbone network uses ResNet-18. The base block of ResNet-18 consists of four cascaded 3×3 convolutional layers. The first 3×3 convolutional layer in all base blocks is replaced with a REDConv module to extract multi-scale base features of flying bird targets. After the high-level features output by the backbone network, a DPA-Encoder module is built to replace the traditional static position embedding. The DPA-Encoder module is used to enhance the model's ability to perceive the dynamic position information of birds. The neck network is constructed with a C3Hyper module and a GateAdd structure. The C3Hyper module is used to achieve adaptive fusion of deep and shallow features, purify and retain key features, and the GateAdd structure guides and optimizes feature transfer through full-path distillation. The multi-scale real-time detection model is trained based on the aforementioned bird target detection dataset to determine the optimal parameters; The test data is input into the trained multi-scale real-time detection model, which outputs the category and location information of the flying bird target.

2. The method according to claim 1, characterized in that, The REDConv module includes two modes: training mode and inference mode. The training mode of the REDConv module includes: For the input feature map X red_in Execute branch operations separately: Branch 1 is derived from the input feature map X red_in With a parameter W std By concatenating standard convolution operators, we obtain the appearance semantic feature map X. std ; Branch 2 is based on the input feature map X red_in With a parameter W cd The central difference convolution operators are concatenated to obtain a fine-grained texture feature map X. cd ; Branch 3 is derived from the input feature map X red_in With a parameter W hd The horizontal difference convolution operators are concatenated to obtain the horizontal gradient feature map X. hd ; Branch 4 is derived from the input feature map X red_in With a parameter W vd The vertical difference convolution operators are concatenated to obtain the vertical gradient feature map X. vd ; Branch 5 is derived from the input feature map X red_in With a parameter W ad The adaptive angle difference convolution operators are concatenated to obtain multi-angle pose feature maps X. ad ; Feature fusion operation: from appearance semantic feature map X std Fine-grained texture feature map X cd Horizontal gradient feature map X hd Vertical gradient feature map X vd and multi-angle pose feature map X ad Performing a bitwise addition operation yields a multidimensional fused feature map X. sum ; Output: Multidimensional fused feature map X sum The REDConv module is sequentially concatenated with a batch normalization module and a ReLU activation function to obtain the final output feature map X. red_out ; Among them, the standard convolution operator W std For learnable regular convolution operations with a kernel size of 3×3; central difference convolution operator W cd The convolution operation is used to calculate the intensity difference between the center pixel and its surrounding neighboring pixels; the horizontal difference convolution operator W hd To perform a convolution operation for detecting vertical edges using a horizontal gradient template; the vertical difference convolution operator W vd To perform convolution operations for detecting horizontal edges using a vertical gradient template; an adaptive angle difference convolution operator W ad This is a convolution operation for detecting slanted edges using a gradient template along the diagonal direction.

3. The method according to claim 2, characterized in that, The inference state REDConv module includes a convolution kernel with parameters W. rep A single-channel 3×3 convolution module; W rep By W std W cd W hd W vd and W ad Obtained by performing a bitwise addition operation.

4. The method according to claim 1 or 3, characterized in that, The DPA-Encoder module consists of N DPA-Encoder layers stacked in series. Each DPA-Encoder layer includes: The first residual connection structure is derived from the input feature map X. enc_in The output X of the position-aware self-attention module attn Perform a bitwise addition operation, then concatenate it with the first layer normalization module LayerNorm to obtain the intermediate feature map X. mid ; The second residual connection structure is composed of the intermediate feature map X. mid The output of the feedforward neural network (FFN) is bitwise added to the output of the second layer normalization module (LayerNorm) to obtain the final output feature map X of the DPA-Encoder layer. dpa-out ; The feedforward neural network (FFN) consists of two cascaded linear layers and a GeLU activation function located between the linear layers; the first and second layer normalization modules (LayerNorm) are used for feature value normalization.

5. The method according to claim 4, characterized in that, The location-aware self-attention module includes: Constructing the query projection layer W Q , key projection layer W K Sum projection layer W V All are linear fully connected layers; The query matrix Q, key matrix K, and value matrix V are constructed from the input feature map X. enc_in With query projection layer W Q , key projection layer W K and value projection layer W V Obtained by connecting in series; Perform matrix multiplication on the transpose of the query matrix Q and the key matrix K, and divide by a scaling factor to obtain the basic attention score matrix Score. base ; Based on the basic attention score matrix Score base With dynamic position bias matrix P bias Performing a bitwise addition operation yields the corrected attention score matrix, Score. final ; The final attention output X is obtained by concatenating the corrected attention score matrix Score_final with the Softmax normalization function and the value matrix V. attn .

6. The method according to claim 5, characterized in that, Dynamic position bias matrix P bias The output of the dynamic location regression subnetwork is the dynamic location regression subnetwork, which includes: Construct a global context aggregation operator as a global average pooling operation to process feature map X. enc_in The spatial dimension is compressed to 1×1, resulting in the channel descriptor X. avg ; Define a location feature mapping network, which includes a first fully connected layer, a ReLU activation function, and a second fully connected layer, which are sequentially connected in series. Channel descriptor X avg The input is fed into the location feature mapping network, and the output is a dynamic location bias vector P corresponding to the number of attention heads. bias_vec ; The dynamic position bias vector P bias_vec The matrix is ​​replicated and expanded in the spatial dimension to generate a dynamic position bias matrix P with the same size as the self-attention matrix. bias .

7. The method according to claim 6, characterized in that, The C3Hyper module executes the following processes, including: The input feature map is X C3H_in ; From the input feature map X C3H_in It is concatenated with a 1×1 convolutional module for dimensionality reduction and then segmented along the channel dimension into the first component X. split1 Second component X split2 ; From the second component X split2 The global context feature map X is obtained by sequentially concatenating node projection, hyperedge convolution, and node reprojection operations. global The node projection operation maps pixel features to hypergraph nodes, while the hyperedge convolution operation aggregates node information using the correlation matrix. From the second component X split2 It is composed of N cascaded depthwise separable convolutional modules DSConv, used to extract local texture features, and obtain local texture feature map X. local ; From the first component X split1 Global Context Feature Map X global and local texture feature map X local Perform channel concatenation, then concatenate with a 1×1 convolutional module to obtain the final enhanced feature map X of the C3Hyper module. enhanced .

8. The method according to claim 7, characterized in that, The GateAdd structure includes a first input and a second input, wherein the first input receives the enhanced feature map X. enhanced The second input receives the shallow, fine-grained feature map X output from the backbone network. shallow ; Shallow fine-grained feature map X at the second input end shallow The spatial resolution is adjusted to match the enhanced feature map X at the first input. enhanced Consistency is achieved, resulting in the aligned feature map X. aligned ; A learnable scalar parameter Gate, initialized to 0, and an aligned feature map X aligned Performing a scalar multiplication operation yields a weighted fine-grained feature map X. weighted ; Enhanced feature map X from the first input terminal enhanced With weighted fine-grained feature map X weighted Performing bitwise addition enables adaptive fusion of C3Hyper enhanced features and shallow fine-grained features, resulting in a fused feature map X containing cross-scale information. gate_out .

9. The method according to claim 8, characterized in that, The backbone network outputs feature maps P3 (Stage 3), P4 (Stage 4), and P5 (Stage 5). The neck network receives the backbone network outputs and performs the following operations: The Stage 5 feature map P5 is concatenated with the C3Hyper module to obtain the high-level semantic feature map P5. _enc ; Perform cross-scale adaptive distillation to enhance the high-level semantic feature map P5. enc Connect the Stage3 feature map P3 to the first input of the GateAdd structure, connect Stage3 to the second input of the GateAdd structure, perform a distillation and fusion operation, and use a gating mechanism to inject the fine-grained texture information of P3 into P5. enc Output distillation feature map P5 containing cross-scale information. distilled ; Perform multi-scale fusion and output, define a top-down path, and integrate P5. distilled After performing an upsampling operation, it is concatenated with the Stage 4 feature map P4 through the execution channel, and then processed by a C3Hyper module to obtain the fused feature map N4. After performing an upsampling operation on N4, it is concatenated with the Stage 3 feature map P3 through the execution channel, and then processed by a C3Hyper module to obtain the fused feature map N3. The downsampling operation performed by N3 is concatenated with the channel operation performed by N4, and then cascaded with a C3Hyper module to obtain the small target detection output feature map Out. small , will Out small Continue performing downsampling operations and P5 distilled The process involves concatenating the execution channels and then linking them with a C3Hyper module to obtain the output feature map for medium to large target detection. large .

Images