Infrared bird recognition and detection method based on attention mechanism and improved YOLOv5

Abstract

The invention provides an infrared bird recognition and detection method based on an attention mechanism and improved YOLOv5, and relates to the technical field of computer vision target detection. Comprising the following steps: constructing an infrared bird image data set; a YOLOv5 network model is constructed; a CA attention mechanism module is newly added in a backbone network of the YOLOv5; newly adding a small target detection layer in the neck network of the YOLOv5; replacing the loss function of the YOLOv5 with an NWD loss function; the improved YOLOv5 detection network is trained; and inputting an infrared bird image for identification, and evaluating the test set. Compared with the prior art, the learning difficulty is reduced, the recognition accuracy is improved, and deployment can be quickly carried out while the precision is kept.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision target detection, and in particular to an infrared bird recognition and detection method based on an attention mechanism and improved YOLOv5. Background Art

[0002] When birds appear in critical scenarios such as airports, they pose a certain threat to the safety and operation of the area. Therefore, timely detection and identification of birds is key to reducing potential risks. Due to the high flying altitude and small targets, the accuracy of traditional bird detection methods is limited in complex environments.

[0003] Infrared imaging systems are less affected in harsh environments, and birds have certain characteristics in infrared images. The use of infrared images by the recognition system can further improve the reliability of the detection system. In actual application scenarios, noise and complex backgrounds will interfere with infrared images, and the resolution is lower than that of visible light images, making the recognition cost higher; traditional infrared detection methods have slow recognition speeds and low generalization capabilities, and cannot meet the increasing recognition needs.

[0004] Therefore, there is an urgent need for an infrared bird recognition and detection method based on attention mechanism and improved YOLOv5 to solve the above-mentioned problems. Summary of the invention

[0005] The purpose of the present invention is to provide an infrared bird recognition and detection method based on attention mechanism and improved YOLOv5, which reduces the learning difficulty, improves the recognition accuracy, and can be deployed quickly while maintaining the accuracy.

[0006] The embodiment of the present invention is achieved as follows:

[0007] The embodiment of the present application provides an infrared bird recognition and detection method based on an attention mechanism and an improved YOLOv5, which includes the following steps:

[0008] Step S1: construct infrared bird image dataset;

[0009] Step S2: Build a YOLOv5 network model;

[0010] Step S3: Add a CA attention mechanism module to the YOLOv5 backbone network;

[0011] Step S4: Add a small target detection layer to the neck network of YOLOv5;

[0012] Step S5: Replace the loss function of YOLOv5 with the NWD loss function;

[0013] Step S6: training the improved YOLOv5 detection network;

[0014] Step S7: Input infrared bird images for recognition and evaluate the test set.

[0015] In some embodiments of the present invention, in the above step S1, the infrared bird images are classified to construct a training set, a validation set and a test set of the infrared bird images.

[0016] In some embodiments of the present invention, the above step S2 specifically includes the following steps:

[0017] Step S2.1: preprocessing the infrared bird image, wherein the preprocessing includes mosaic data enhancement, adaptive anchor frame calculation and adaptive image scaling;

[0018] Step S2.2: The backbone feature extraction network extracts features from the input preprocessed image. The network consists of CSPDarknet, which includes the Focus layer, CSP layer, ConvBNSiLU layer and SPP layer.

[0019] Step S2.3: The feature fusion network composed of FPN+PAN feature pyramid fuses the extracted features, and then the network of the Head part detects the image after feature fusion.

[0020] In some embodiments of the present invention, the above step S3 specifically includes the following steps:

[0021] Step S3.1: In the backbone network, the input feature map of the shape of C×H×W is average pooled, and the pooling kernels of (H, 1) and (1, W) are used to perform pooling in the X and Y axis directions respectively to generate feature maps of the shape of C×H×1 and C×1×W, and the extracted feature maps are spliced ​​into the shape of C / r×1×(W+H);

[0022] Step S3.2: The feature map is passed through the F1 convolution transformation function and the nonlinear activation function to generate an f intermediate feature map;

[0023] Step S3.3: Split f into two tensors fh and fw according to the spatial dimension, perform convolution transformation function and Sigmoid activation function on Fh and Fw respectively, obtain gh and gw coordinate attention, and then multiply them to get the output of the same shape as the input.

[0024] In some embodiments of the present invention, the above step S4 specifically includes the following steps:

[0025] Step S4.1: Upsample the 80×80 feature map in the neck network once to a 160×160 feature map;

[0026] Step S4.2: Concat the 160×160 feature map of the second layer in the backbone network to obtain a 160×160 feature map, then downsample it to obtain an 80×80 feature map and pass it to the detection head through C3.

[0027] In some embodiments of the present invention, the above step S5 specifically includes the following steps:

[0028] Step S5.1: Assuming the bounding box R = (cx, cy, w, h), the definition formula of the second-order Wasserstein distance is as follows:

[0029] Step S5.2: The normalized index obtains the metric. The normalized Wasserstein distance formula is as follows:

[0030] Step S5.3: NWD-based loss function Replace the original loss function of YOLOv5.

[0031] In some embodiments of the present invention, the above step S6 specifically includes the following steps:

[0032] Step S6.1: Set training parameters, input the infrared bird image training set and its verification set constructed in step 1 into the improved detection network, where the training set is used for network training and learning, and the verification set is used to verify the effect of training. The size of the input image is 640×640;

[0033] Step S6.2: Determine the optimal learning rate and number of iterations, and load the trained optimal weight file into the network model.

[0034] In some embodiments of the present invention, the above step S7 specifically includes the following steps:

[0035] Step S7.1: Input the test set obtained in step S1 into the trained improved YOLOv5 network to test its performance;

[0036] Step S7.2: Use average precision, accuracy, and recall to evaluate the detection effect of the improved YOLOv5 infrared bird detection network. The average precision is used to measure the overall detection effect of the network. The calculation formula is as follows:

[0037] AP=∫P(R)dR

[0038] Where P is the precision, R is the recall, and this formula is used to find the area of ​​the PR curve;

[0039] The accuracy is used to measure the accuracy of the network, and its calculation formula is as follows:

[0040]

[0041] The recall rate is used to measure the completeness of the network, and its calculation formula is as follows:

[0042] Where TP means predicting the correct target as correct, FP means predicting the wrong target as correct, and FN means predicting the correct target as wrong.

[0043] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0044] Compared with the traditional infrared detection method, the present invention provides an infrared bird recognition and detection method based on the attention mechanism and improved YOLOv5. The CA attention mechanism is added to ensure the flexibility and lightweight of the model, so as to quickly capture effective information and perform detection; the recall rate and detection effect of identifying small bird targets are improved by adding a small target layer; the intersection-and-union ratio metric in the loss function of YOLOv5 is replaced by the NWD metric to reduce the difficulty of model learning and improve the accuracy of network recognition. This method can effectively prevent the loss of spatial position information of weak targets such as birds, improve the accuracy of identifying infrared bird small targets, and can be deployed quickly while maintaining accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 is a flowchart of steps of an embodiment of the present invention;

[0047] Figure 2 is a network structure diagram of an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the CA attention mechanism of an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0050] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Example:

[0052] Please refer to Figure 1-3 , this embodiment provides an infrared bird recognition and detection method based on attention mechanism and improved YOLOv5, which includes the following steps:

[0053] Step S1: construct infrared bird image dataset;

[0054] Step S1.1: Create an infrared bird image dataset and label the birds in the dataset;

[0055] Step S1.2: Construct the training set, validation set, and test set of infrared bird images. The dataset constructed in step 1.1 is divided into 8:2 ratios, where the training set accounts for 80% of the dataset, the test set accounts for 20% of the dataset, and the validation set accounts for 10% of the dataset.

[0056] Step S2: Build a YOLOv5 network model;

[0057] Step S2.1: preprocessing the infrared bird image, wherein the preprocessing includes mosaic data enhancement, adaptive anchor frame calculation and adaptive image scaling;

[0058] Step S2.2: The backbone feature extraction network extracts features from the input preprocessed image. The network consists of CSPDarknet, which includes the Focus layer, CSP layer, ConvBNSiLU layer and SPP layer.

[0059] Step S2.3: The feature fusion network composed of FPN+PAN feature pyramid fuses the extracted features, and then the network of the Head part detects the image after feature fusion.

[0060] Step S3: Add a CA attention mechanism module to the YOLOv5 backbone network, and decompose the channel attention mechanism SENet into two parallel one-dimensional feature encoding processes. This behavior can effectively reduce the impact of position information loss caused by two-dimensional pooling, so that its information can be more efficiently integrated into the attention map. The CA mechanism has the following features:

[0061] (1) Flexible and lightweight, it can insert a lightweight backbone network to enhance feature expression and location information expression.

[0062] (2) Improve model accuracy by capturing additional direction-aware and position-aware information for more accurate positioning.

[0063] (3) It can effectively reduce the computational complexity for obtaining information in a large area.

[0064] After adding the CA attention mechanism, useful features are enhanced, useless information is shielded, and the performance and deployment capabilities of the model are significantly improved.

[0065] Step S3.1: In the backbone network, the input feature map of the shape of C×H×W is average pooled, and the pooling kernels of (H, 1) and (1, W) are used to perform pooling in the X and Y axis directions respectively to generate feature maps of the shape of C×H×1 and C×1×W, and the extracted feature maps are spliced ​​into the shape of C / r×1×(W+H);

[0066] Step S3.2: The feature map is passed through the F1 convolution transformation function and the nonlinear activation function to generate an f intermediate feature map;

[0067] Step S3.3: Split f into two tensors fh and fw according to the spatial dimension, perform convolution transformation function and Sigmoid activation function on Fh and Fw respectively, obtain gh and gw coordinate attention, and then multiply them to get the output of the same shape as the input.

[0068] Step S4: Add a small target detection layer to the neck network of YOLOv5; the size of small target samples is small, and the downsampling multiple of YOLOv5 is large, which makes it difficult for deeper feature maps to obtain feature information of small targets, increasing the difficulty of learning. Therefore, a small target detection layer is added to the neck network of YOLOv5 to make the network pay more attention to small target detection, thereby improving the recall rate and detection effect of small bird targets;

[0069] Step S4.1: Upsample the 80×80 feature map in the neck network once to a 160×160 feature map;

[0070] Step S4.2: Concat the 160×160 feature map of the second layer in the backbone network to obtain a 160×160 feature map, then downsample it to obtain an 80×80 feature map and pass it to the detection head through C3.

[0071] Step S5: Replace the loss function of YOLOv5 with the NWD loss function; Replace the loss function of YOLOv5 with the NWD loss function. The loss function of YOLOv5 is calculated by the intersection over union (IoU). This calculation method is too sensitive to the position deviation of small targets such as birds, resulting in poor detection results and reduced accuracy. Therefore, the normalized Wasserstein distance (NWD) is used to replace the original loss function. Compared with IoU, NWD has the following advantages:

[0072] (1) The position deviation changes smoothly and is insensitive to targets of different scales;

[0073] (2) It has the ability to identify the similarity of non-overlapping boxes and can measure the similarity between small objects regardless of whether they overlap or not.

[0074] The NWD metric can be easily embedded into the assignment, NMS, and loss function of any anchor-based detector. Using NWD to replace the original loss function of YOLOv5 significantly improves the model detection performance.

[0075] Step S5.1: Assuming the bounding box R = (cx, cy, w, h), the definition formula of the second-order Wasserstein distance is as follows:

[0076] Step S5.2: The normalized index obtains the metric. The normalized Wasserstein distance formula is as follows:

[0077] Step S5.3: NWD-based loss function Replace the original loss function of YOLOv5.

[0078] Step S6: training the improved YOLOv5 detection network;

[0079] Step S6.1: Set training parameters, input the infrared bird image training set and its verification set constructed in step 1 into the improved detection network, where the training set is used for network training and learning, and the verification set is used to verify the effect of training. The size of the input image is 640×640;

[0080] Step S6.2: Determine the optimal learning rate and number of iterations, and load the trained optimal weight file into the network model.

[0081] Step S7: Input infrared bird images for recognition and evaluate the test set.

[0082] Step S7.1: Input the test set obtained in step S1 into the trained improved YOLOv5 network to test its performance;

[0083] Step S7.2: Use average precision, accuracy, and recall to evaluate the detection effect of the improved YOLOv5 infrared bird detection network. The average precision is used to measure the overall detection effect of the network. The calculation formula is as follows:

[0084] AP=∫P(R)dR

[0085] Where P is the precision, R is the recall, and this formula is used to find the area of ​​the PR curve;

[0086] The accuracy is used to measure the accuracy of the network, and its calculation formula is as follows:

[0087]

[0088] The recall rate is used to measure the completeness of the network, and its calculation formula is as follows:

[0089] Where TP means predicting the correct target as correct, FP means predicting the wrong target as correct, and FN means predicting the correct target as wrong.

[0090] In some embodiments of the present invention, in step S1, the infrared bird images are classified according to a certain ratio to construct a training set, a validation set and a test set of the infrared bird images.

[0091] In summary, the present invention provides an infrared bird recognition and detection method based on the attention mechanism and improved YOLOv5. The CA attention mechanism is added to ensure the flexibility and lightweight of the model so as to quickly capture effective information and perform detection; the recall rate and detection effect of identifying small bird targets are improved by adding a small target layer; the intersection-and-union ratio metric in the loss function of YOLOv5 is replaced by the NWD metric to reduce the difficulty of model learning and improve the accuracy of network recognition. This method can effectively prevent the loss of spatial position information of weak targets such as birds, improve the accuracy of identifying infrared bird small targets, and can be deployed quickly while maintaining accuracy.

[0092] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An infrared bird recognition and detection method based on attention mechanism and improved YOLOv5, characterized in that: The following steps are involved: Step S1: construct infrared bird image dataset; Step S2: Build a YOLOv5 network model; Step S3: Add a CA attention mechanism module to the YOLOv5 backbone network; Step S4: Add a small target detection layer to the neck network of YOLOv5; Step S5: Replace the loss function of YOLOv5 with the NWD loss function; Step S6: training the improved YOLOv5 detection network; Step S7: Input infrared bird images for recognition and evaluate the test set.

2. The infrared bird recognition and detection method based on attention mechanism and improved YOLOv5 according to claim 1, characterized in that: In step S1, the infrared bird images are classified to construct a training set, a validation set and a test set of the infrared bird images.

3. The infrared bird recognition and detection method based on attention mechanism and improved YOLOv5 according to claim 2 is characterized in that: The step S2 specifically includes the following steps: Step S2.1: preprocessing the infrared bird image, wherein the preprocessing includes mosaic data enhancement, adaptive anchor frame calculation and adaptive image scaling; Step S2.2: The backbone feature extraction network extracts features from the input preprocessed image. The network consists of CSPDarknet, which includes the Focus layer, CSP layer, ConvBNSiLU layer and SPP layer. Step S2.3: The feature fusion network composed of FPN+PAN feature pyramid fuses the extracted features, and then the network of the Head part detects the image after feature fusion.

4. The infrared bird recognition and detection method based on attention mechanism and improved YOLOv5 according to claim 3 is characterized in that: The step S3 specifically comprises the following steps: Step S3.1: In the backbone network, the input feature map of the shape of C×H×W is average pooled, and the pooling kernels of (H, 1) and (1, W) are used to perform pooling in the X and Y axis directions respectively to generate feature maps of the shape of C×H×1 and C×1×W, and the extracted feature maps are spliced ​​into the shape of C / r×1×(W+H); Step S3.2: The feature map is passed through the F1 convolution transformation function and the nonlinear activation function to generate an f intermediate feature map; Step S3.3: Split f into two tensors fh and fw according to the spatial dimension, perform convolution transformation function and Sigmoid activation function on Fh and Fw respectively, obtain gh and gw coordinate attention, and then multiply them to get the output of the same shape as the input.

5. The infrared bird recognition and detection method based on attention mechanism and improved YOLOv5 according to claim 4 is characterized in that: The step S4 specifically comprises the following steps: Step S4.1: Upsample the 80×80 feature map in the neck network once to a 160×160 feature map; Step S4.2: Concat the 160×160 feature map of the second layer in the backbone network to obtain a 160×160 feature map, then downsample it to obtain an 80×80 feature map and pass it to the detection head through C3.

6. The infrared bird recognition and detection method based on attention mechanism and improved YOLOv5 according to claim 5, characterized in that: The step S5 specifically includes the following steps: Step S5.1: Assuming the bounding box R = (cx, cy, w, h), the definition formula of the second-order Wasserstein distance is as follows: Step S5.2: The normalized index obtains the metric. The normalized Wasserstein distance formula is as follows: Step S5.3: NWD-based loss function Replace the original loss function of YOLOv5.

7. The infrared bird recognition and detection method based on attention mechanism and improved YOLOv5 according to claim 6 is characterized in that: The step S6 specifically includes the following steps: Step S6.1: Set training parameters, input the infrared bird image training set and its verification set constructed in step 1 into the improved detection network, where the training set is used for network training and learning, and the verification set is used to verify the effect of training. The size of the input image is 640×640; Step S6.2: Determine the optimal learning rate and number of iterations, and load the trained optimal weight file into the network model.

8. The infrared bird recognition and detection method based on attention mechanism and improved YOLOv5 according to claim 7, characterized in that: The step S7 specifically includes the following steps: Step S7.1: Input the test set obtained in step S1 into the trained improved YOLOv5 network to test its performance; Step S7.2: Use average precision, accuracy, and recall to evaluate the detection effect of the improved YOLOv5 infrared bird detection network. The average precision is used to measure the overall detection effect of the network. The calculation formula is as follows: AP=∫P(R)dR Where P is the precision, R is the recall, and this formula is used to find the area of ​​the PR curve; The accuracy is used to measure the accuracy of the network, and its calculation formula is as follows: The recall rate is used to measure the completeness of the network, and its calculation formula is as follows: Where TP means predicting the correct target as correct, FP means predicting the wrong target as correct, and FN means predicting the correct target as wrong.

Images