Flying bird and unmanned aerial vehicle high-precision identification system and method

Abstract

According to the bird and unmanned aerial vehicle high-precision identification system and method provided by the invention, the output result of the target detection module is depended on, a plurality of algorithm modules including the target detection module, the sequence classification module, the brightness distribution feature classification module and the track classification module are fused, and a precise decision rule is combined, so that the accuracy of the bird and unmanned aerial vehicle identification is improved. And a high-precision final identification result is generated. Wherein the sequence classification module captures the states of birds at different moments through a group of target image sequences, so that the classification accuracy is remarkably improved; the brightness distribution feature classification module extracts brightness distribution features caused by wing flapping of the birds in the flying process through mathematical analysis, and the classification effect is further optimized; the trajectory classification module analyzes the motion trajectory features of the bird and the unmanned aerial vehicle in the image, breaks through the dependence on the detail features of the image, and guarantees the recognition accuracy under the condition of a minimum target. Through fusion of results of various modules, the classification error rate of the birds and the unmanned aerial vehicle is effectively reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of low-altitude anti-UAV and bird-repelling technology, and more particularly to a high-precision identification system and method for flying birds and UAVs. Background Art

[0002] In the field of low-altitude anti-UAV and bird-repelling technology, radar equipment is usually used to guide optoelectronic equipment to lock and track flying birds and drone targets and assess their threat level. In this process, high-precision target identification is the key to accurately assess the threat level. At the same time, in order to reduce system costs and simplify deployment, it is generally recommended to integrate the recognition algorithm directly into the optoelectronic equipment, which places higher requirements on the recognition accuracy and operating efficiency of the algorithm. In optoelectronic equipment, in order to reduce the interference of environmental factors such as light and weather, thermal imaging technology can be used to collect image data. The existing methods for bird and drone recognition have shown high accuracy in target positioning, but there are still certain challenges in category classification.

[0003] In the field of low-altitude anti-UAV and bird-repelling technology, the target to be identified is usually far away from the optoelectronic equipment. Due to the influence of distance, optoelectronic equipment has difficulty focusing at long distances, resulting in a certain degree of blur and noise in the thermal imaging image, and unclear target details, which affects the accuracy of category classification. In addition, the posture of birds changes during flight, and in certain postures, there is a great similarity between the image features and those of drones, which makes it difficult to accurately distinguish their categories based on a single frame of image. Under extreme conditions, such as when the target is too far away from the optoelectronic equipment or the environment is too harsh, birds and drones usually appear as blurred bright spots in thermal imaging images, further increasing the difficulty of classification.

[0004] There are usually two approaches to existing methods: one is to rely solely on the target detection algorithm to locate and classify the target; the other is to combine the track information returned by the radar for multi-source fusion detection. However, both methods have certain limitations. For the target detection algorithm, the loss function used in its training process is usually the weighted sum of classification loss, confidence loss and positioning loss. This weighting method will lead to a negative impact on the classification accuracy. In the method that combines radar track information, the data update frequency of different models of radars varies greatly, especially for low-cost radars, whose track data update frequency is usually in seconds, resulting in sparse track points, which is not conducive to accurately distinguishing birds from drones. Although radars with high update frequencies can provide more intensive data, they are expensive and difficult to deploy on a large scale. In addition, when birds glide, their tracks are highly similar to those of drones, which further weakens the effect of track classification methods. Therefore, how to balance cost and actual deployment needs while ensuring high accuracy is still a major challenge currently faced. Summary of the invention

[0005] In view of the above problems, the purpose of the present invention is to provide a high-precision identification system and method for birds and drones. By integrating multiple algorithm modules and sophisticated decision-making rules, high-precision identification of birds and drones is achieved, which has the advantages of low cost and high efficiency.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions: In a first aspect, the present invention discloses a high-precision bird and drone identification system, comprising: Target detection module, sequence classification module, brightness distribution feature classification module, trajectory classification module and fusion decision module; The target detection module is used to read the detection data set generated by the thermal imaging video, identify the target position and target type in the detection data set based on the YOLOv5 target detection model, and generate first recognition result information; the target types include flying birds and drones; The sequence classification module is used to read the classification data set generated by the thermal imaging video, extract a set of image frame sequences from the classification data set, extract features through the Resnet34 feature extraction network, perform denoising, and then use a bidirectional long short-term memory network to capture the timing information in the image frame sequence, and output the category score of the target through the classification head to achieve the classification of flying birds and drones, and generate second recognition result information; The brightness distribution feature classification module is used to read the classification data set generated by the thermal imaging video, distinguish between a bird flapping its wings frequently and a relatively stable drone by analyzing the brightness distribution features of the target image in the classification data set, and generate third recognition result information; The trajectory classification module is used to read the target frame annotation information of the detection data set generated by the thermal imaging video, calculate the motion trajectory characteristics of the target in the image, analyze the miss distance between the target frame and the image center and the stability of the trajectory to determine the target type, and generate fourth recognition result information; The fusion decision module is used to generate a target recognition result based on the first recognition result, the second recognition result, the third recognition result and the fourth recognition result according to a preset decision rule.

[0007] Furthermore, the target detection module includes: a feature extraction network, a denoising module, a bidirectional feature pyramid network, a detection head 1, a detection head 2 and a detection head 3; The feature extraction network includes a CSP-Darknet53 backbone network and an SPPF spatial pyramid pooling module, wherein the CSP-Darknet53 backbone network is used to obtain video frames of the detection data set and generate first feature maps, second feature maps, and third feature maps of different scales; and the SPPF spatial pyramid pooling module is used to extract key features from the third feature map and generate a key feature map. The denoising module is used to perform denoising on the key feature map to generate a target area feature map; The bidirectional feature pyramid network is used to fuse the first feature map, the second feature map and the target area feature map to generate a first pyramid feature map, a second pyramid feature map and a third pyramid feature map of different scales; The detection head one, the detection head two and the detection head three are used to perform target detection on the first pyramid feature map, the second pyramid feature map and the third pyramid feature map respectively, and generate target detection results; the target detection results include target categories, target category scores and target location information.

[0008] Furthermore, the sequence classification module includes: a Resnet34 feature extraction network, a denoising module, a bidirectional long short-term memory network and a classification head; The Resnet34 feature extraction network is used to obtain an image frame sequence and generate a feature map sequence; The denoising module is used to perform denoising processing on the feature map sequence to generate a target area feature map sequence; The bidirectional long short-term memory network is used to generate a representation feature vector sequence through the bidirectional long short-term memory network after the target area feature map sequence is globally averaged pooled to generate a feature vector sequence; The classification head adopts a softmax activation function to convert the feature vector sequence into a probability distribution, generate a category score of the target, and determine the target category.

[0009] Furthermore, the denoising module includes: a Non-Local attention module, a channel attention module and a spatial attention module; The Non-Local attention module is used to obtain a feature map and generate a global context-aware feature map as an enhanced feature map by calculating the similarity between all positions in the feature map; The channel attention module is used to generate a channel attention score vector by extracting and weighting the global feature information of each channel according to the channel dimension of the enhanced feature map; recalibrate the channel weight of the enhanced feature map according to the attention score vector to generate a feature map after weight adjustment; The spatial attention module is used to generate a spatial attention score map by extracting and weighting feature information at different spatial positions according to the spatial dimension of the feature map after weight adjustment; and perform spatial weight adjustment on the feature map after weight adjustment according to the spatial attention score map to generate a feature map of the target area of ​​interest.

[0010] In a second aspect, the present invention further discloses a method for high-precision identification of flying birds and drones, comprising the following steps: S1: Collect thermal imaging videos of flying birds and drones to generate a thermal imaging video frame image dataset , for thermal imaging video frame image dataset Perform preprocessing to obtain the preprocessed detection data set , and classification datasets ; S2: Detection dataset Input to the target detection module and train the target detection module; S3: Classification dataset Input to the sequence classification module and train the sequence classification module; S4: Classification dataset Input to the brightness distribution feature classification module, and design the brightness distribution feature classification module; S5: Detection dataset The target box annotation is input into the trajectory classification module to train the trajectory classification module; S6: deploy the target detection module, sequence classification module, brightness distribution feature classification module, trajectory classification module and fusion decision module to the optoelectronic device, start the optoelectronic device, collect thermal imaging video, and generate four recognition results through the target detection module, sequence classification module, brightness distribution feature classification module and trajectory classification module respectively. Based on the four recognition results, according to the preset decision rules, the recognition results of flying birds and drones are generated and output.

[0011] Further, the step S1 comprises: Collect thermal imaging videos of flying birds and drones to generate a thermal imaging video frame image dataset ;in, , , is the number of video frame images in the dataset, Refers to the data set video frames; Will The target box is marked as ;in, is the marked target category, For the purpose The upper left corner horizontal coordinate of For the purpose The upper left corner ordinate in For the purpose The size width in , For the purpose Dimension height in; Thermal imaging video frame image dataset The size of the video frame images contained in the image is uniformly scaled to , and perform data enhancement operations to obtain the preprocessed detection data set ;in, , , Refers to the detection dataset Middle video frames; At the same time, mark the target box In according to and The size ratio is scaled to obtain the scaled target box annotation ; Labeled according to the target box , in the thermal imaging video frame image dataset Crop the marked target area and scale it to the same size , get the classification data set ; in, , Pointing at The cropped and scaled target image.

[0012] Further, the step S2 comprises: Video Frame Input into the CSP-Darknet53 backbone network to obtain feature maps of different scales , , ; Among them, the feature map The size is , feature map The size is , feature map The size is ; The feature map Input to the SPPF spatial pyramid pooling module to obtain the feature map ; The feature map Input to the Non-Local attention module to get the feature map ; The feature map Input to the channel attention module and obtain the feature map through the convolution layer , the feature map Perform global average pooling to obtain a feature vector with a vector dimension of 1024 , the feature vector Input to the fully connected layer with sigmoid activation to obtain the attention score vector with a vector dimension of 1024 , the attention score vector With feature map Multiply the corresponding positions to obtain the feature map; The feature map Input to the spatial attention module and obtain the feature map through the dilated convolution layer ; The feature map Input to the sigmoid activated point-by-point convolution layer to get the attention score map , size is ; The attention score map With feature map Multiply the corresponding positions to get the feature map ; The feature map With feature map Add them together to get the feature map output by the noise feature suppression module , size is ; The feature map , , Input into the bidirectional feature pyramid network to obtain pyramid feature maps of different scales , , ; Among them, the pyramid feature map The size is , pyramid feature map The size is , pyramid feature map The size is ; The pyramid feature map , , Input to detection head 1, detection head 2 and detection head 3 respectively to get the detection results ;in, is the target category classified by the target detection module, is the target category score, is the horizontal coordinate of the upper left corner of the target in the image, is the upper left corner ordinate of the target in the image, is the width of the target in the image, The height of the target in the image.

[0013] Test results Provide supervision to train the improved YOLOv5 object detection model; The improved YOLOv5 target detection model after training is subjected to Network Slimming, and the network branches with weight parameters close to zero are removed, where the branch slimming rate is set to .

[0014] Further, the step S3 comprises: In the classification dataset Extract image sequence from ; in, , represents a classification data set Any continuous The image sequence consists of target images. , express Middle target image; The image sequence Input to the sequence classification module, and obtain the feature map sequence through the Resnet34 feature extraction network ; Among them, the size of each feature map is ; The feature map sequence Input to the denoising module, for the Feature map , firstly transform the feature map Input to the Non-Local attention module to get the feature map ; The feature map Input to the convolution layer of the channel attention module to obtain the feature map ; The feature map Perform global average pooling to obtain a feature vector with a vector dimension of 512 ; The feature vector Input to the fully connected layer with sigmoid activation to obtain the attention score vector with a vector dimension of 512 ; The attention score vector With feature map Multiply the corresponding positions to get the feature map ; The feature map The spatial attention module is input to the denoising module, and the feature map is obtained through the dilated convolution layer. ; The feature map Input to the sigmoid activated point-by-point convolution layer to get the attention score map , size is , the attention score map With feature map Multiply the corresponding positions to get the feature map , size is ; The feature map With feature map Add them together to get the final output of the noise feature suppression module , size is ; The feature map sequence After all feature maps of are denoised by the denoising module, a feature map sequence is generated. ; in, .

[0015] The feature map sequence Each feature map in is globally averaged and pooled to obtain a feature vector sequence ; where the dimension of each feature vector is ; Will Input into the bidirectional long short-term memory network to obtain the representative feature vector of the image sequence , the vector dimension is 512; will represent the feature vector Input to the classification head activated by the softmax function to get the classification result ;in, is the target category classified by the sequence classification module, is the target category score.

[0016] Sequence classification results Provide supervision to train the sequence classification module.

[0017] Further, the step S4 comprises: In the classification dataset Extract image sequence from ; in, , which means any continuous The image sequence consists of target images. , express Middle target image.

[0018] The target image Convert to height direction and divide it into upper and lower parts equally, and calculate the average brightness value of the upper and lower parts respectively and , and calculate the difference in average brightness between the upper and lower parts .by is the threshold value, Binarize it and get ; In the image sequence If two adjacent frames With Computational and If they are inconsistent, it is recorded as one jump, and the statistics The number of transitions that occurred ; According to the number of jumps Compute image sequence The jump rate in ; According to the switching rate Get the category output by the brightness distribution feature classification module ; in, , is the brightness distribution feature threshold.

[0019] Further, the step S5 comprises: In the dataset Target box annotation Extracting annotation sequences ; in, , which represents the data set Any continuous A sequence of annotations for each image. , express Middle target box annotation.

[0020] Calculate target box The horizontal coordinate of the center of in the image is , the vertical axis is ; Calculate target box The horizontal miss distance from the image center is ; Calculate target box The vertical miss distance from the image center is ; Calculate target box The Euclidean miss distance from the image center is ; calculate The mean of the average level of misses for all target boxes in , the average vertical miss distance , the mean of the average European off-target amount ; Will Input to a multi-layer perceptron, through the hidden layer, to get the hidden vector , the vector dimension is 128; Hide the vector Input to the output layer of the multi-layer perceptron to obtain the classification result ;in is the target category classified by the trajectory classification module, is the target category score; Trajectory classification results Provide supervision to train the trajectory classification module.

[0021] Compared with the prior art, the beneficial effect of the present invention is that: the present invention discloses a high-precision identification system and method for birds and drones, which not only relies on the output results of the target detection module, but also generates a high-precision final identification result by fusing multiple algorithm modules, including the target detection module, the sequence classification module, the brightness distribution feature classification module and the trajectory classification module, combined with sophisticated decision rules. As the target tracking time increases, the accuracy of category recognition is gradually improved. Among them, the sequence classification module captures the state of the bird at different times through a set of target image sequences, which significantly improves the classification accuracy; the brightness distribution feature classification module extracts the brightness distribution characteristics caused by the flapping of the wings of the bird in flight through mathematical analysis, and further optimizes the classification effect; the trajectory classification module breaks through the dependence on the image detail features by analyzing the motion trajectory characteristics of the bird and the drone in the image, especially under the condition of extremely small targets. The accuracy of recognition is guaranteed. Through the fusion of the results of multiple modules, the present invention greatly reduces the classification error rate of birds and drones.

[0022] It can be seen that compared with the prior art, the present invention has outstanding substantive features and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0024] Figure 1 It is a system principle diagram of a specific implementation mode of the present invention.

[0025] Figure 2 It is a structural principle diagram of a target detection module according to a specific implementation mode of the present invention.

[0026] Figure 3 It is a structural principle diagram of a sequence classification module according to a specific embodiment of the present invention.

[0027] Figure 4 It is a structural principle diagram of a denoising module according to a specific implementation mode of the present invention.

[0028] Figure 5 It is a method flow chart of a specific implementation mode of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0030] See also Figure 1 As shown, this embodiment provides a high-precision bird and drone identification system, including: Target detection module, sequence classification module, brightness distribution feature classification module, trajectory classification module and fusion decision module.

[0031] The target detection module is used to read the detection data set generated by the thermal imaging video, identify the target position and target type in the detection data set based on the YOLOv5 target detection model, and generate first recognition result information; the target types include flying birds and drones.

[0032] As an example, the role of the target detection module is to detect and obtain the position and type of the target while the optoelectronic device is tracking the target to be identified. The target detection module in this system has the advantages of both detection speed and accuracy.

[0033] The sequence classification module is used to read the classification data set generated by the thermal imaging video, extract a set of image frame sequences from the classification data set, extract features through the Resnet34 feature extraction network, perform denoising, and then use the bidirectional long short-term memory network to capture the timing information in the image frame sequence. The classification head outputs the category score of the target to achieve the classification of flying birds and drones, and generate the second recognition result information.

[0034] It should be noted that the reason for designing a sequence classification module in this system is that when birds are gliding or flapping their wings, their morphology is similar to that of fixed-wing aircraft. At this time, using a single-frame image for classification is prone to misclassification. However, if a set of image frame sequences is used, birds and drones are easier to distinguish.

[0035] The brightness distribution feature classification module is used to read the classification data set generated by the thermal imaging video, distinguish between birds that flap their wings frequently and relatively stable drones by analyzing the brightness distribution features of the target image in the classification data set, and generate third recognition result information.

[0036] The reason why this system designs a brightness distribution feature classification module is that for small targets such as birds that flap their wings frequently, their brightness distribution characteristics show a pattern of frequent changes in time sequence, so a brightness distribution feature classification module is designed.

[0037] The trajectory classification module is used to read the target frame annotation information of the detection data set generated by the thermal imaging video, calculate the motion trajectory characteristics of the target in the image, analyze the off-target amount between the target frame and the image center and the stability of the trajectory to determine the target type, and generate fourth recognition result information.

[0038] The reason for designing the trajectory classification module in this system is that when the target to be identified is too far away or the environment is too harsh, it is no longer possible to distinguish between birds and drones through image features. However, for birds with strong maneuverability, when the optoelectronic device tracks them, the birds will repeatedly jump around the center of the image field of view, but the maneuverability of drones is not as good as that of birds, so they will be relatively stable and basically stay in the center of the image field of view. Taking advantage of this, this system constructs a trajectory classification module based on the trajectory distribution characteristics of birds and drones in the images of optoelectronic devices.

[0039] The fusion decision module is used to generate a target recognition result based on the first recognition result, the second recognition result, the third recognition result and the fourth recognition result according to a preset decision rule.

[0040] In the present invention, high-precision recognition of the target to be identified is a process of continuous confirmation. Different from previous methods, this system not only relies on the category identified by the target detection module, but also utilizes the fusion decision module to fuse multiple algorithm modules and obtain high-precision recognition results through the set decision rules. Moreover, as the tracking time increases, the category recognition becomes more accurate.

[0041] In a specific embodiment, Figure 2 As shown, the target detection module includes: a feature extraction network, a denoising module, a bidirectional feature pyramid network, a detection head one, a detection head two and a detection head three.

[0042] The feature extraction network includes a CSP-Darknet53 backbone network and an SPPF spatial pyramid pooling module. The CSP-Darknet53 backbone network is used to obtain video frames of the detection data set and generate first feature maps, second feature maps, and third feature maps of different scales; the SPPF spatial pyramid pooling module is used to extract key features of the third feature map and generate a key feature map.

[0043] The denoising module is used to denoise the key feature map and generate a target area feature map.

[0044] The bidirectional feature pyramid network is used to fuse the first feature map, the second feature map and the target area feature map to generate first pyramid feature maps, second pyramid feature maps and third pyramid feature maps of different scales.

[0045] Detection head one, detection head two and detection head three are used to perform target detection on the first pyramid feature map, the second pyramid feature map and the third pyramid feature map respectively, and generate target detection results; the target detection results include target categories, target category scores and target location information.

[0046] In a specific embodiment, Figure 3 As shown in the figure, the sequence classification module includes: Resnet34 feature extraction network, denoising module, bidirectional long short-term memory network and classification head.

[0047] Resnet34 feature extraction network is used to obtain image frame sequences and generate feature map sequences.

[0048] The denoising module is used to perform denoising on the feature map sequence to generate a target area feature map sequence.

[0049] The bidirectional long short-term memory network is used to generate a feature vector sequence after the target area feature map sequence is globally averaged pooled to generate a feature vector sequence. Then, the bidirectional long short-term memory network is used to generate a feature vector sequence.

[0050] The classification head uses a softmax activation function to convert the feature vector sequence into a probability distribution, generate the target category score, and determine the target category.

[0051] In a specific implementation, the target detection module and the sequence classification module are provided with denoising modules with the same functions. Figure 4 As shown, the denoising module includes: Non-Local attention module, channel attention module and spatial attention module.

[0052] The Non-Local Attention Module is used to obtain the feature map and generate a global context-aware feature map as an enhanced feature map by calculating the similarity between all locations in the feature map.

[0053] The channel attention module is used to generate a channel attention score vector by extracting and weighting the global feature information of each channel according to the channel dimension of the enhanced feature map; and recalibrate the channel weight of the enhanced feature map according to the attention score vector to generate a feature map with adjusted weights.

[0054] The spatial attention module is used to generate a spatial attention score map by extracting and weighting the feature information of different spatial positions according to the spatial dimension of the feature map after weight adjustment; the spatial weight of the feature map after weight adjustment is adjusted according to the spatial attention score map to generate a feature map of the target area of ​​attention.

[0055] As an example, the denoising module is a module designed by this system to suppress the image noise features in the network. It consists of a Non-Local attention module, a channel attention module, and a spatial attention module. The Non-Local attention module can effectively aggregate similar features in distant feature areas while retaining the details of the target area while reducing noise. The core of this mechanism is to introduce global contextual relationships, so that the model can interactively calculate all pixel positions in the input feature map to generate a feature map with global perception capabilities. Since non-local operations only consider global spatial information interactions and lack explicit modeling of channel and spatial characteristics, the distribution of weights is strengthened through the channel attention module and the spatial attention module.

[0056] When the denoising module is used in the target detection module, the channel attention module is used to enhance the model's ability to represent small targets and improve target positioning. The spatial attention module is used to enhance the ability to perceive target areas in complex scenes, especially to distinguish overlapping targets or complex backgrounds.

[0057] When the denoising module is applied to the sequence classification module, the Non-Local module can effectively aggregate similar features in distant feature areas during classification, reduce noise features, and retain target details. The channel attention module can highlight the features of category-related channels and improve classification accuracy. When it is necessary to distinguish detailed differences, the spatial attention module can accurately capture the spatial features of key parts and improve the ability of the sequence classification model to distinguish subtle differences.

[0058] See also Figure 5 As shown, based on the above system architecture, the present invention also discloses a method for high-precision identification of flying birds and drones, comprising the following steps: S1: Collect thermal imaging videos of flying birds and drones to generate a thermal imaging video frame image dataset , for thermal imaging video frame image dataset Perform preprocessing to obtain the preprocessed detection data set , and classification datasets .

[0059] S2: Detection dataset Input to the target detection module and train the target detection module.

[0060] S3: Classification dataset Input to the sequence classification module and train the sequence classification module.

[0061] S4: Classification dataset Input into the brightness distribution feature classification module and design the brightness distribution feature classification module.

[0062] S5: Detection dataset The target box annotation is input into the trajectory classification module to train the trajectory classification module.

[0063] S6: Deploy the target detection module, sequence classification module, brightness distribution feature classification module, trajectory classification module and fusion decision module to the optoelectronic device, start the optoelectronic device, collect thermal imaging video, and generate four recognition results through the target detection module, sequence classification module, feature classification module and trajectory classification module respectively. Based on the four recognition results, according to the preset decision rules, the recognition results of flying birds and drones are generated and output.

[0064] In an embodiment of the present invention, based on step S1, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0065] S101: Collect thermal imaging videos of flying birds and drones to generate a thermal imaging video frame image dataset , Thermal imaging video frame image dataset The expression is as follows: ,in, , is the number of video frame images in the dataset, Refers to the data set video frames, the video frames are sequential in time, and each frame has a unique target box annotation. The target box is marked as ,in is the marked target category, For the purpose The upper left corner horizontal coordinate of For the purpose The upper left corner ordinate in For the purpose The size width in , For the purpose The height of the dimension in .

[0066] S102: Dataset The size of the video frame images contained in the image is uniformly scaled to , and then perform data enhancement operations to obtain the preprocessed detection data set , , Refers to the first video frames. In according to and Scale the size ratio to get .

[0067] S103: Labeling based on target box , in the dataset Crop the marked target area and scale it to the same size , get the classification data set , Pointing at The cropped and scaled target image.

[0068] In an embodiment of the present invention, based on step S2, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0069] S201: preprocessed video frames Input into the CSP-Darknet53 backbone network to obtain feature maps of different scales , , , where the feature map The size is , feature map The size is , feature map The size is . The feature map Input to the SPPF spatial pyramid pooling module to obtain the feature map , size is .

[0070] S202: First, the feature map Input to the Non-Local attention module to get the feature map , size is . The feature map Input to the channel attention module, the channel attention module consists of a convolution kernel size of The convolutional layer and the fully connected layer with sigmoid activation are composed of Input to the convolution layer to get the feature map , size is , the feature map Perform global average pooling to obtain the feature vector , the vector dimension is , the feature vector Input to the fully connected layer with sigmoid activation to get the attention score vector , the vector dimension is , the attention score vector With feature map Multiply the corresponding positions to get the feature map , size is . The feature map Input to the spatial attention module, the spatial attention module consists of a convolution kernel size of , expansion rate For the expanded convolution layer, the convolution kernel size of the sigmoid activation is The point-by-point convolution layer is composed of Input to the dilated convolution layer to get the feature map , size is , the feature map Input to the sigmoid activated point-by-point convolution layer to get the attention score map , size is , the attention score map With feature map Multiply the corresponding positions to get the feature map , size is . The feature map With feature map Add them together to get the output of the noise feature suppression module , size is .

[0071] S203: Feature map , , Input into the bidirectional feature pyramid network to obtain pyramid features of different scales , , . The feature map The size is , feature map The size is , feature map The size is .

[0072] S204: Pyramid feature map , , Input to detection head 1, detection head 2 and detection head 3 respectively to get the detection results ,in is the target category classified by the target detection module, is the target category score, is the horizontal coordinate of the upper left corner of the target in the image, is the upper left corner ordinate of the target in the image, is the width of the target in the image, The height of the target in the image.

[0073] S205: The preprocessed data set According to the process described in steps S201 to S204, the improved YOLOv5 target detection model is input in sequence, and the detection results are Provide supervision to train a modified YOLOv5 object detection model.

[0074] S206: Perform Network Slimming on the trained improved YOLOv5 target detection model, and remove the network branches whose weight parameters are close to zero, where the branch slimming rate is set to In the process of pruning, L1 regularization is first used to perform sparse training on the normalization coefficients of the BN layer in the improved YOLOv5 target detection model, ultimately achieving the goal of making the redundant weights close to zero, and then the pruning rate is calculated. The weights whose weight parameters are close to zero are deleted to obtain the pruned improved YOLOv5 target detection model. Finally, the pruned model is fine-tuned to solve the problem of decreased accuracy caused by pruning.

[0075] In an embodiment of the present invention, based on step S3, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0076] S301: In the classification data set Extract image sequence from .

[0077] in, , which means any continuous The image sequence consists of target images. , express Middle target image.

[0078] S302: Image sequence Input to the sequence classification module, which consists of a Resnet34 feature extraction network, a denoising module, a bidirectional long short-term memory network and a classification head.

[0079] S302: Input into the Resnet34 feature extraction network to obtain a feature map sequence , where the size of each feature map is .

[0080] S304: Input to the denoising module to obtain a feature map sequence , where the size of each feature map is .

[0081] Specifically, firstly, the feature map Input to the Non-Local attention module to get the feature map , size is . The feature map Input to the channel attention module, the channel attention module consists of a convolution kernel size of The convolutional layer and the fully connected layer with sigmoid activation are composed of Input to the convolution layer to get the feature map , size is , the feature map Perform global average pooling to obtain the feature vector , the vector dimension is , the feature vector Input to the fully connected layer with sigmoid activation to get the attention score vector , the vector dimension is , the attention score vector With feature map Multiply the corresponding positions to get the feature map , size is . The feature map Input to the spatial attention module, the spatial attention module consists of a convolution kernel size of , expansion rate For the expanded convolution layer, the convolution kernel size of the sigmoid activation is The point-by-point convolution layer is composed of Input to the dilated convolution layer to get the feature map , size is , the feature map Input to the sigmoid activated point-by-point convolution layer to get the attention score map , size is , the attention score map With feature map Multiply the corresponding positions to get the feature map , size is . The feature map With feature map Add them together to get the final output of the noise feature suppression module , size is .

[0082] S305: Each feature map in is globally averaged and pooled to obtain a feature vector sequence , where the dimension of each feature vector is .Will Input into the bidirectional long short-term memory network to obtain the representative feature vector of the image sequence , the vector dimension is 512.

[0083] S306: Input to the classification head activated by the softmax function to get the classification result .in, is the target category classified by the sequence classification module, is the target category score.

[0084] S307: Classify the dataset The sequence classification module is input in sequence according to the process described in step S303 to step S306 in a sliding window manner, and the sequence classification results are Provide supervision to train the sequence classification module.

[0085] In an embodiment of the present invention, based on step S4, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0086] S401: In the classification data set Extract image sequence from .

[0087] in, , which means any continuous The image sequence consists of target images. , express Middle target image.

[0088] S402: It is divided into two parts in the height direction, and the average brightness values ​​of the upper and lower parts are calculated respectively. and . Calculate the difference in average brightness between the upper and lower parts .by is the threshold value, Binarize it and get .

[0089] S403: If two adjacent frames With Computational and If they are inconsistent, it is recorded as one jump, and the statistics The number of transitions that occurred .calculate The jump rate in Finally, according to the jump rate Get the category output by the brightness distribution feature classification module ,in is the brightness distribution feature threshold.

[0090] In an embodiment of the present invention, based on step S5, a possible embodiment is given below to illustrate its specific implementation in a non-limiting manner.

[0091] S501: In the dataset Target box annotation Extracting annotation sequences .

[0092] in, , which represents the data set Any continuous A sequence of annotations for each image. , express Middle target box annotation.

[0093] S502: Calculate target frame The horizontal coordinate of the center of in the image is , the vertical axis is . Calculate the target box The horizontal miss distance from the image center is , calculate the target box The vertical miss distance from the image center is , calculate the target box The Euclidean miss distance from the image center is .

[0094] S503: Calculation The mean of the average level of off-target for all target boxes in is The average vertical miss distance is The mean of the average Euclidean off-target amount is

[0095] S504: Input to a multi-layer perceptron, which consists of one hidden layer and one softmax activated output layer. Input to the hidden layer to get the hidden vector , the vector dimension is 128, Input to the output layer to get the classification result ,in is the target category classified by the trajectory classification module, is the target category score.

[0096] S505: Data set The target box annotation is input into the trajectory classification module in sequence according to the process described in step S502 to step S504 in a sliding window manner, and the trajectory classification result is Provide supervision to train the trajectory classification module.

[0097] In an embodiment of the present invention, based on step S6, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0098] S601: deploying a target detection module, a sequence classification module, a brightness distribution feature classification module and a trajectory classification module in an optoelectronic device, and starting the optoelectronic device to identify the target after the radar device detects the target.

[0099] S602: Collect thermal imaging video frames through optoelectronic equipment, pre-process and scale the video frames , input to the target detection module, and obtain the target frame detection result according to the process described in steps S201 to S204 , Indicates the detection result of any frame detected when the optoelectronic device is running.

[0100] S603: The optoelectronic device is used to track and lock the target to be identified through the KCF target tracking algorithm. After obtaining the next frame of thermal imaging video, a video with a size of 10000 is cropped with the center point of the target tracked last time as the center. The cropped area is input into the target detection module for local field of view target detection.

[0101] By implementing the local field of view target detection strategy in this step, the present invention further improves recognition efficiency and reduces deployment costs.

[0102] At the same time, in the process of target detection, according to the detected target frame Crop and cache the detected image and target frame of the target to be identified .

[0103] S604: After continuous detection After the target is retrieved, the cached target image is input into the sequence classification module, and the classification result of the sequence classification module is obtained according to the process described in step S303 to step S306. , Indicates that any length recognized during the operation of the optoelectronic device is Sequence classification results for the target image sequence.

[0104] S605: After continuous detection After the target is detected, the cached target image is input into the brightness distribution feature classification module, and the classification result of the brightness distribution feature classification module is obtained according to the process described in step S402 to step S403. , Indicates that any length recognized during the operation of the optoelectronic device is The brightness distribution feature classification results of the target image sequence.

[0105] S606: After continuous detection After the target is found, the cached target frame is input into the trajectory classification module, and the classification result of the trajectory classification module is obtained according to the process described in step S502 to step S504. , Indicates that any length recognized during the operation of the optoelectronic device is Trajectory classification results for target image sequences.

[0106] S607: During the first 5 seconds of the optoelectronic device tracking and locking the target to be identified, the identified classification category is ignored and the category is output as unknown. After 5 seconds, the category information identified by the target detection module in all frames Perform statistics and calculate the ratio of flying birds to drones. The category with the larger ratio is the output category of the target detection module. .

[0107] S608: If other classification modules do not have category output, the current output type is the category output by the target detection module , otherwise, the output category is the result of other classification modules The most common type.

[0108] In actual use, the target detection module, sequence classification module, brightness distribution feature classification module and trajectory classification module described in the present invention can be deployed in the optoelectronic device through the built-in Atlas acceleration module. Each module works together to perform real-time analysis on the thermal imaging video collected by the optoelectronic device. Through the preset decision rules, the system can integrate the analysis results of multiple modules to generate high-precision target recognition results. The use of the Atlas acceleration module not only provides powerful computing performance support for optoelectronic equipment, but also has a high cost-effectiveness and energy efficiency ratio. This modular design can give full play to the advantages of the combination of software and hardware. Compared with traditional high-performance computing platforms, its hardware cost is significantly reduced, while also greatly reducing energy consumption and equipment maintenance costs.

[0109] In addition, the high-precision identification method for flying birds and drones provided by the present invention and other conventional identification methods are simultaneously applied to experiments for performing low-altitude anti-drone and bird-repelling tasks. The experimental results are shown in Table 1 below: Table 1 Comparison of experimental indicators of low-altitude anti-UAV and bird-repelling missions

[0110] As shown in Table 1, the experimental results of the method of the present invention in low-altitude anti-UAV and bird-repelling tasks can significantly reduce the missed detection rate, false alarm rate and classification error rate compared with other methods. The experimental results show that the method of the present invention has obvious advantages in low-altitude anti-UAV and bird-repelling tasks. Its innovative design not only improves the accuracy and reliability of the task, but also has broad application prospects and commercial value, providing an efficient and reliable solution for the safe development of the low-altitude economy.

[0111] In terms of specific implementation, the present invention proposes an improved YOLOv5 model as a target detection module, which integrates a denoising module and effectively reduces the interference of noise features by introducing a Non-Local attention module, a channel attention module and a spatial attention module. At the same time, it improves the feature representation ability of the target area and the target perception ability in complex scenes, thereby achieving significant optimization in terms of missed detection rate and false alarm rate, and further reducing the classification error rate.

[0112] In summary, the present invention not only relies on the output results of the target detection module, but also generates a high-precision final recognition result by fusing multiple algorithm modules, including the target detection module, the sequence classification module, the brightness distribution feature classification module and the trajectory classification module, combined with sophisticated decision rules. As the target tracking time increases, the accuracy of category recognition is gradually improved. Among them, the sequence classification module captures the state of the bird at different times through a set of target image sequences, which significantly improves the classification accuracy; the brightness distribution feature classification module extracts the brightness distribution characteristics caused by the flapping of the wings of the bird in flight through mathematical analysis, and further optimizes the classification effect; the trajectory classification module breaks through the dependence on the image detail features by analyzing the motion trajectory characteristics of the bird and the drone in the image, especially under the condition of extremely small targets. The accuracy of recognition is guaranteed. Through the fusion of the results of multiple modules, the present invention greatly reduces the classification error rate of birds and drones.

[0113] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. As for the method disclosed in the embodiment, since it corresponds to the system disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0114] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0115] The above is a detailed introduction to the high-precision bird and drone identification system and method provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A high-precision bird and drone identification system, characterized in that: include: Target detection module, sequence classification module, brightness distribution feature classification module, trajectory classification module and fusion decision module; The target detection module is used to read the detection data set generated by the thermal imaging video, identify the target position and target type in the detection data set based on the YOLOv5 target detection model, and generate first recognition result information; The target types include birds and drones; The sequence classification module is used to read the classification data set generated by the thermal imaging video, extract a set of image frame sequences from the classification data set, extract features through the Resnet34 feature extraction network, perform denoising, and then use a bidirectional long short-term memory network to capture the timing information in the image frame sequence, and output the category score of the target through the classification head to achieve the classification of flying birds and drones, and generate second recognition result information; The brightness distribution feature classification module is used to read the classification data set generated by the thermal imaging video, distinguish between a bird flapping its wings frequently and a relatively stable drone by analyzing the brightness distribution features of the target image in the classification data set, and generate third recognition result information; The trajectory classification module is used to read the target frame annotation information of the detection data set generated by the thermal imaging video, calculate the motion trajectory characteristics of the target in the image, analyze the miss distance between the target frame and the image center and the stability of the trajectory to determine the target type, and generate fourth recognition result information; The fusion decision module is used to generate a target recognition result based on the first recognition result, the second recognition result, the third recognition result and the fourth recognition result according to a preset decision rule.

2. The high-precision bird and drone identification system according to claim 1, characterized in that: The target detection module includes: a feature extraction network, a denoising module, a bidirectional feature pyramid network, a detection head 1, a detection head 2 and a detection head 3; The feature extraction network includes a CSP-Darknet53 backbone network and an SPPF spatial pyramid pooling module, wherein the CSP-Darknet53 backbone network is used to obtain video frames of the detection data set and generate first feature maps, second feature maps, and third feature maps of different scales; and the SPPF spatial pyramid pooling module is used to extract key features from the third feature map and generate a key feature map. The denoising module is used to perform denoising on the key feature map to generate a target area feature map; The bidirectional feature pyramid network is used to fuse the first feature map, the second feature map and the target area feature map to generate a first pyramid feature map, a second pyramid feature map and a third pyramid feature map of different scales; The detection head one, the detection head two and the detection head three are used to perform target detection on the first pyramid feature map, the second pyramid feature map and the third pyramid feature map respectively, and generate target detection results; the target detection results include target categories, target category scores and target location information.

3. The high-precision bird and drone identification system according to claim 2, characterized in that: The sequence classification module includes: a Resnet34 feature extraction network, a denoising module, a bidirectional long short-term memory network and a classification head; The Resnet34 feature extraction network is used to obtain an image frame sequence and generate a feature map sequence; The denoising module is used to perform denoising processing on the feature map sequence to generate a target area feature map sequence; The bidirectional long short-term memory network is used to generate a representation feature vector sequence through the bidirectional long short-term memory network after the target area feature map sequence is globally averaged pooled to generate a feature vector sequence; The classification head adopts a softmax activation function to convert the feature vector sequence into a probability distribution, generate a category score of the target, and determine the target category.

4. The high-precision bird and drone identification system according to claim 3, characterized in that: The denoising module includes: a Non-Local attention module, a channel attention module and a spatial attention module; The Non-Local attention module is used to obtain a feature map and generate a global context-aware feature map as an enhanced feature map by calculating the similarity between all positions in the feature map; The channel attention module is used to generate a channel attention score vector by extracting and weighting the global feature information of each channel according to the channel dimension of the enhanced feature map; recalibrate the channel weight of the enhanced feature map according to the attention score vector to generate a feature map after weight adjustment; The spatial attention module is used to generate a spatial attention score map by extracting and weighting feature information at different spatial positions according to the spatial dimension of the feature map after weight adjustment; and perform spatial weight adjustment on the feature map after weight adjustment according to the spatial attention score map to generate a feature map of the target area of ​​interest.

5. A high-precision method for identifying flying birds and drones, characterized in that: The method adopts the high-precision bird and drone identification system as described in any one of claims 1 to 4; The method comprises the following steps: S1: Collect thermal imaging videos of flying birds and drones to generate a thermal imaging video frame image dataset , for thermal imaging video frame image dataset Perform preprocessing to obtain the preprocessed detection data set , and classification datasets ; S2: Detection dataset Input to the target detection module and train the target detection module; S3: Classification dataset Input to the sequence classification module and train the sequence classification module; S4: Classification dataset Input to the brightness distribution feature classification module, and design the brightness distribution feature classification module; S5: Detection dataset The target box annotation is input into the trajectory classification module to train the trajectory classification module; S6: Deploy the target detection module, sequence classification module, brightness distribution feature classification module, trajectory classification module and fusion decision module to the optoelectronic device, start the optoelectronic device, collect thermal imaging video, and generate four recognition results through the target detection module, sequence classification module, brightness distribution feature classification module and trajectory classification module respectively. Based on the four recognition results, according to the preset decision rules, the recognition results of flying birds and drones are generated and output.

6. The method for high-precision identification of flying birds and drones according to claim 5, characterized in that: The step S1 comprises: Collect thermal imaging videos of flying birds and drones to generate a thermal imaging video frame image dataset ;in, , , is the number of video frame images in the dataset, Refers to the data set video frames; Will The target box is marked as ;in, is the marked target category, For the purpose The upper left corner horizontal coordinate of For the purpose The upper left corner ordinate in For the purpose The size width in , For the purpose Dimension height in; Thermal imaging video frame image dataset The size of the video frame images contained in the image is uniformly scaled to , and perform data enhancement operations to obtain the preprocessed detection data set ;in, , , Refers to the detection dataset Middle video frames; At the same time, mark the target box In according to and The size ratio is scaled to obtain the scaled target box annotation ; Labeled according to the target box , in the thermal imaging video frame image dataset Crop the marked target area and scale it to the same size , get the classification data set ; in, , Pointing at The cropped and scaled target image.

7. The method for high-precision identification of flying birds and drones according to claim 6, characterized in that: The step S2 comprises: Video Frame Input into the CSP-Darknet53 backbone network to obtain feature maps of different scales , , ; Among them, the feature map The size is , feature map The size is , feature map The size is ; The feature map Input to the SPPF spatial pyramid pooling module to obtain the feature map ; The feature map Input to the Non-Local attention module to get the feature map ; The feature map Input to the channel attention module and obtain the feature map through the convolution layer , the feature map Perform global average pooling to obtain a feature vector with a vector dimension of 1024 , the feature vector Input to the fully connected layer with sigmoid activation to obtain the attention score vector with a vector dimension of 1024 , the attention score vector With feature map Multiply the corresponding positions to obtain the feature map; The feature map Input to the spatial attention module and obtain the feature map through the dilated convolution layer ; The feature map Input to the sigmoid activated point-by-point convolution layer to get the attention score map , size is ; The attention score map With feature map Multiply the corresponding positions to get the feature map ; The feature map With feature map Add them together to get the feature map output by the noise feature suppression module , size is ; The feature map , , Input into the bidirectional feature pyramid network to obtain pyramid feature maps of different scales , , ; Among them, the pyramid feature map The size is , pyramid feature map The size is , pyramid feature map The size is ; The pyramid feature map , , Input to detection head 1, detection head 2 and detection head 3 respectively to get the detection results ;in, is the target category classified by the target detection module, is the target category score, is the horizontal coordinate of the upper left corner of the target in the image, is the upper left corner ordinate of the target in the image, is the width of the target in the image, is the size height of the target in the image; Test results Provide supervision to train the improved YOLOv5 object detection model; The improved YOLOv5 target detection model after training is subjected to Network Slimming, and the network branches with weight parameters close to zero are removed, where the branch slimming rate is set to .

8. The method for high-precision identification of flying birds and drones according to claim 7, characterized in that: The step S3 comprises: In the classification dataset Extract image sequence from ; in, , represents a classification data set Any continuous The image sequence consists of target images. , express Middle target image; The image sequence Input to the sequence classification module, and obtain the feature map sequence through the Resnet34 feature extraction network ; Among them, the size of each feature map is ; The feature map sequence Input to the denoising module, for the Zhang feature map , firstly transform the feature map Input to the Non-Local attention module to get the feature map ; The feature map Input to the convolution layer of the channel attention module to obtain the feature map ; The feature map Perform global average pooling to obtain a feature vector with a vector dimension of 512 ; The feature vector Input to the fully connected layer with sigmoid activation to obtain the attention score vector with a vector dimension of 512 ; The attention score vector With feature map Multiply the corresponding positions to get the feature map ; The feature map The spatial attention module is input to the denoising module, and the feature map is obtained through the dilated convolution layer. ; The feature map Input to the sigmoid activated point-by-point convolution layer to get the attention score map , size is , the attention score map With feature map Multiply the corresponding positions to get the feature map , size is ; The feature map With feature map Add them together to get the final output of the noise feature suppression module , size is ; The feature map sequence After all feature maps of are denoised by the denoising module, a feature map sequence is generated. ; in, ; The feature map sequence Each feature map in is globally averaged and pooled to obtain a feature vector sequence ; where the dimension of each feature vector is ; Will Input into the bidirectional long short-term memory network to obtain the representative feature vector of the image sequence , the vector dimension is 512; will represent the feature vector Input to the classification head activated by the softmax function to get the classification result ;in, is the target category classified by the sequence classification module, is the target category score; Sequence classification results Provide supervision to train the sequence classification module.

9. The method for high-precision identification of flying birds and drones according to claim 8, characterized in that: The step S4 comprises: In the classification dataset Extract image sequence from ; in, , which means any continuous The image sequence consists of target images. , express Middle target image; The target image Convert to height direction and divide it into upper and lower parts equally, and calculate the average brightness value of the upper and lower parts respectively and , and calculate the difference in average brightness between the upper and lower parts ;by is the threshold value, Binarize it and get ; In the image sequence If two adjacent frames With Computational and If they are inconsistent, it is recorded as one jump, and the statistics The number of transitions that occurred ; According to the number of jumps Compute image sequence The jump rate in ; According to the switching rate Get the category output by the brightness distribution feature classification module ; in, , is the brightness distribution feature threshold.

10. The method for high-precision identification of flying birds and drones according to claim 9, characterized in that: The step S5 comprises: In the dataset Target box annotation Extracting annotation sequences ; in, , which represents the data set Any continuous A sequence of annotations for each image. , express Middle target box annotation; Calculate target box The horizontal coordinate of the center of in the image is , the vertical axis is ; Calculate target box The horizontal miss distance from the image center is ; Calculate target box The vertical miss distance from the image center is ; Calculate target box The Euclidean miss distance from the image center is ； calculate The mean of the average level of misses for all target boxes in , the average vertical miss , the mean of the average European off-target amount ; Will Input into a multi-layer perceptron, through the hidden layer, to get the hidden vector , the vector dimension is 128; Hide the vector Input to the output layer of the multi-layer perceptron to obtain the classification result ;in is the target category classified by the trajectory classification module, is the target category score; Trajectory classification results Provide supervision to train the trajectory classification module.

Images