Northeast tiger leopard identification method based on transfer learning and double attention mechanism

By introducing deep learning methods of transfer learning and double attention mechanisms in Siberian tiger and leopard recognition, combined with YOLO v5m network and feature extraction structure, the problems of Siberian tiger and leopard recognition accuracy and training time are solved, and efficient and accurate recognition and Improved identification efficiency.

CN118918606APending Publication Date: 2024-11-08CHANGCHUN NORMAL UNIV

Patent Information

Application Number
CN202410970571.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The prior art in wild animal recognition, especially Siberian tigers and leopards, has low recognition accuracy and long training time, and faces the problem of insufficient feature extraction under complex background, resulting in insufficient recognition efficiency and accuracy.

Method used

Using a deep learning recognition method based on transfer learning and dual attention mechanism, the YOLO v5m network combines Mosaic data augmentation, Focus and CSP structures to introduce spatial pyramid pooling and path aggregation network, and pay attention to the space of the feature layer through the CA attention mechanism. and channel information to improve feature extraction capabilities.

Benefits of technology

It realizes efficient and accurate identification of Siberian tigers and leopards in complex contexts, taking into account recognition accuracy and training speed, significantly improving model training efficiency, reducing computing resource requirements, and providing a more scientific basis for wildlife monitoring decision-making.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918606A_ABST
    Figure CN118918606A_ABST
Patent Text Reader

Abstract

The invention discloses a Northeast tiger identification method based on transfer learning and a double attention mechanism. The method comprises the following steps: constructing a Northeast tiger deep learning identification model by using a programming language python and a deep learning framework pytorch; acquiring image data by using an infrared wild animal camera to construct sample data, and dividing a sample data set into a training sample, a test sample and a verification sample; using the training sample to train a deep learning recognition model of the Calistoma northern; selecting an optimal parameter for the whole network according to a loss function convergence condition and a precision improvement condition of the verification sample to obtain an optimal model; and carrying out precision evaluation on a prediction result by adopting four indexes including the precision rate, the recall rate, the average precision and the F1 score, and analyzing the overall performance of the model. The method can achieve the efficient and accurate recognition of the wild northeast tiger and the northeast leopard, provides a scientific basis for the decision management of the northeast tiger and other wild animals, and provides a reference for the intelligent protection of the wild animals.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a method for identifying wild animals, in particular to a deep learning method for identifying Siberian tigers and leopards based on the fusion of transfer learning and double attention mechanism, and belongs to the field of artificial intelligence technology. Background Art

[0002] The Siberian tiger and the Amur leopard (Panthera tigris altaica & Amur Leopard) are located at the top of the food chain of the forest ecosystem. They are national first-class protected wild animals and are classified as endangered species and critically endangered species by the International Union for Conservation of Nature (IUCN). They have very important conservation and scientific research value. Using infrared camera technology to carry out wildlife monitoring is currently the most direct and effective means of wildlife protection. Infrared cameras have the advantages of little interference to animals, real background, and all-weather, and can obtain wildlife activity trajectories and spatiotemporal information. With the widespread application of infrared cameras, the amount of data obtained from monitoring has increased rapidly. Faced with the increasingly large amount of data, the traditional manual screening and identification methods are insufficient, take a long time, and have a high misjudgment rate. It is urgent to use new methods to achieve accurate analysis of massive data sets.

[0003] The continuous development of deep learning technology has improved the real-time and practicality of target detection, making animal identification more convenient and quick. The deep learning model can effectively complete the image recognition task. However, the activity trajectories of wild animals are complex and there is complex background information. Deep learning loses features seriously when extracting infrared image information, resulting in poor recognition accuracy. A deeper network structure can effectively enhance feature extraction and improve the recognition of wild animals in complex backgrounds, but the complex network structure also increases the training and detection time, and has higher requirements on the computing power and performance of the computer. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method for identifying Siberian tigers and leopards based on transfer learning and double attention mechanism, taking into account both recognition accuracy and detection speed, so as to achieve efficient and accurate recognition of Siberian tigers and leopards in the wild environment.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0006] A method for identifying Siberian tigers and leopards based on transfer learning and attention mechanism, comprising the following steps:

[0007] (1) Use the programming language python and the deep learning framework pytorch to build a deep learning recognition model for Siberian tigers and leopards; the deep learning model uses the YOLO v5m network as the basic framework. The input layer adopts the Mosaic data enhancement method to enrich the data set to be detected and enhance the robustness of the model; the backbone network Backbone uses the Focus and CSP structures to extract feature maps; the Neck module uses the spatial pyramid pooling SPP-NET and the path aggregation network PANet structure to aggregate network features, enhancing the module's ability to adapt to the detection of targets of different sizes;

[0008] (2) Use the image data of Siberian tigers and leopards taken by infrared cameras and public datasets to construct sample data;

[0009] (3) Perform noise processing, image expansion, image enhancement, positive and negative sample balance, and image calibration on the sample data to construct a Siberian tiger and leopard sample dataset.

[0010] (4) Divide the obtained Siberian tiger and leopard sample data set into training samples, test samples, and validation samples;

[0011] (5) Train the deep learning model with training samples, set appropriate hyperparameters before training, and select appropriate gradient descent optimizer and loss function;

[0012] (6) According to the convergence of the loss function and the improvement of accuracy of the verification sample, the optimal parameters are selected for the entire network to obtain the optimal model;

[0013] (7) The accuracy of the prediction results is evaluated using four indicators: precision, recall, average precision, and F1-Score, and the overall performance of the model is analyzed.

[0014] As a preferred solution, the image data in step (2) includes RGB three-band image data and grayscale image data.

[0015] As a preferred solution, the image enhancement and image expansion in step (3) are written in the programming language Python, and the image calibration is performed by annotating with the aid of the target detection tool Labelimg.

[0016] As a preferred solution, the step (5) of training the Siberian tiger and leopard deep learning model comprises the following steps:

[0017] (1) Initialize model parameters;

[0018] (2) Input training samples and validation samples;

[0019] (3) Forward propagation;

[0020] (4) Calculate the error based on the output probability;

[0021] (5) Back propagation; select an appropriate gradient descent optimization algorithm;

[0022] (6) Traverse all training data sets and train the data according to the training generations;

[0023] (7) When the number of learning times reaches the threshold, the optimal model is obtained.

[0024] Beneficial effects: Compared with the prior art, the present invention makes full use of deep learning and image recognition technology, and uses the advantages of deep learning models, such as fast speed, high efficiency and automation, to achieve effective identification of Siberian tigers and leopards. The present invention uses the YOLO V5m deep learning network structure, introduces transfer learning and attention mechanism technology, and achieves efficient identification of Siberian tigers and leopards under complex backgrounds. Transfer learning is a machine learning technology that is used for a new model after fine-tuning a pre-trained model. Simply put, it is to use the model developed for task A as the starting point and reuse it in the process of developing a model for task B. The present invention uses transfer learning to retrain the existing deep learning model, which solves the problems of long training time, complex process, large data sample size, and high iteration requirements of the learning model, improves the model training efficiency and adapts to unpredictable conditions more flexibly. The attention mechanism in deep learning imitates human vision, selectively focuses on more important information, and selects more critical information for the current task objectives from a large amount of image information. The present invention integrates transfer learning and double attention mechanism methods, combines the attention of the two dimensions of feature channel and feature space in the Backbone layer, and strengthens the feature extraction capability; introduces the CA attention mechanism in the two effective feature layers extracted by the backbone network and the upsampling results, and pays full attention to the spatial information and channel information of the feature layer. The present invention takes into account both recognition accuracy and training speed, and can achieve efficient and accurate recognition of wild Siberian tigers and leopards, providing a new idea for solving the problem of unclear local features of the target in the image and real-time monitoring by infrared cameras, and at the same time provides a more scientific basis for decision-making and management of Siberian tigers, leopards and other wild animals. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a technical flow chart of the Northeast tiger and leopard deep learning recognition model of the present invention;

[0026] Figure 2 This is a schematic diagram of the network structure of the Northeast Tiger and Leopard deep learning model used in the present invention;

[0027] Figure 3 It is a display diagram of some samples in the sample data set constructed by the present invention;

[0028] Figure 4 It is a graph showing the total loss value and Epoch change trend of the model of the present invention;

[0029] Figure 5 It is a heat map of the attention area of ​​the positive sample by the attention mechanism model of the present invention;

[0030] Figure 6 It is the test output of the present invention that the correct recognition result is obtained. DETAILED DESCRIPTION

[0031] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0032] Embodiment 1: The deep learning recognition method of Northeast tiger and leopard based on transfer learning and double attention mechanism of the present invention comprises the following steps: first, a deep learning recognition model of Northeast tiger and leopard is constructed using programming language python and deep learning framework pytorch, the deep learning recognition model of Northeast tiger and leopard adopts YOLO v5m network as the basic framework, and the input layer adopts Mosaic data enhancement method to enrich the data set to be detected and enhance the robustness of the model; the backbone network Backbone uses Focus and CSP structures to extract feature maps; the Neck module uses spatial pyramid pooling SPP-NET and path aggregation network PANet structure to aggregate network features, and enhances the detection ability of the module to adapt to targets of different sizes. Then, the image data of Northeast tiger and leopard taken by infrared camera and the public data set are used to construct sample data, and the sample data is subjected to noise processing, image expansion, image enhancement, positive and negative sample balance and image calibration to construct a sample data set of Northeast tiger and leopard. The obtained sample data set of Northeast tiger and leopard is divided into training sample, test sample and verification sample. The deep learning recognition model of Northeast tiger and leopard is trained with the training sample, and appropriate hyperparameters are set before training, and appropriate gradient descent optimizer and loss function are selected. According to the convergence of the loss function and the improvement of accuracy of the validation sample, the optimal parameters of the entire network are selected to obtain the optimal model. The four indicators of precision, recall, average precision, and F1-Score are used to evaluate the accuracy of the prediction results and analyze the overall performance of the model.

[0033] The Siberian tiger and leopard deep learning model used in the present invention is an improved YOLOv5 deep learning network: a Siberian tiger and leopard deep learning network based on transfer learning and dual attention mechanism fusion. This network introduces a transfer learning mechanism to solve the problems of long training time, complex process, large data sample size, high iteration requirements, etc. of the learning model, improve the model training efficiency and adapt to unpredictable conditions more flexibly. This network considers the feature extraction capabilities of the two dimensions of feature channel and feature space at the Backbone layer, and introduces the CBAM dual-channel attention mechanism; considering that the channel attention mechanism will lose the object space information during the global maximum pooling, the model introduces the CA attention mechanism to embed the position information into the channel attention, and applies it to the two effective feature layers extracted by the backbone network and the upsampling results, and fully pays attention to the spatial information and channel information of the feature layer. The present invention takes into account both recognition accuracy and training speed, and can achieve efficient and accurate recognition of wild Siberian tigers and Siberian leopards.

[0034] To verify the effect of the present invention, the applicant conducted a field test in the Northeast Tiger and Leopard National Park, and the results are as follows:

[0035] ①Data source

[0036] This example uses infrared wildlife protection cameras deployed in the Northeast Tiger and Leopard National Park to obtain real wildlife video images, realize multi-angle and multi-directional species monitoring, and obtain rich terrestrial vertebrate resources such as Northeast tigers, Northeast leopards, wild boars, sika deer, and roe deer in the decade from 2013 to 2023. After preprocessing, the video data is used as image data, including RGB three-band image data and grayscale image data.

[0037] ②Build a sample data set

[0038] The image data of Siberian tigers, Siberian leopards and other representative endangered wild animals in the Siberian Tiger and Leopard National Park were selected to make samples. Matlab2020a was used to extract key frames of video data, and invalid feature images were screened by manual recognition. In order to prevent the classification model from overfitting, the image enhancement method can be used to increase the number of samples. Commonly used image enhancement methods include adding impulse noise, scaling rotation, grid segmentation, mirror processing and color jitter. Labelimg was used to annotate and draw the prior frames required for deep learning training, and the sample data was calibrated at the image level and pixel level.

[0039] ③Model training and prediction

[0040] Python 3.8 is used as the development language, and the simple and efficient open source deep learning framework Pytorch 1.10 is selected for the experiment. The CUDA version is 11.0. The hardware platform is Intel (R) Xeon (R) CPU E5-2696 v3, an NVIDIA GeForce RTX 4060Ti workstation, the video memory size is 64G, and the software used is the Windows 11 operating system. In the hyperparameter setting of training the target detection network, the batch size is set to 8, the learning rate (lr) is selected to be 0.001, the confidence is (confidence) 0.001, and the threshold value is 0.5. When the training epoch is 100 times, the loss function tends to be stable and the model converges. The SGD optimizer is used to optimize the parameter training process. When training the model, the ratio of the training set to the test set is 8:2.

[0041] ④ Accuracy evaluation

[0042] The AP (map average precision) value is used to evaluate the accuracy of multi-category detection tasks; the loss function (Loss) value is used as a measure of whether the model has converged; the F1 score (F1Score), recall rate (Recall), and precision (Precision) are used as verification indicators. Among them: TP is a positive sample predicted as positive by the model, FP is a negative sample predicted as positive by the model, and FN is a positive sample predicted as negative by the model.

[0043] ⑤Results

[0044] In order to verify the superiority of YOLO_v5m algorithm combined with transfer learning technology in the recognition of Siberian tiger and leopard targets under day and night background, the present invention conducted three groups of comparative experiments. The first group of experiments only trained the night RGB images (a), the second group of experiments put the daytime and nighttime images into the same training batch for training (b), and the third group of experiments used the pre-trained weights of the second group of experiments to fine-tune the training images to reduce the time required for model training (c).

[0045] Table 1 Three groups of experimental test results

[0046]

[0047] The results show that when the model stops training and the training is completed, the Loss has reached the condition of model convergence, so the model can be considered effective. According to the comparison results of three groups of experiments, the mAP value of the Siberian tiger and Siberian leopard in the natural state at night based on the YOLO_v5m model is 83.15%, and the mAP value in the natural state during the day and night is 86.67%. The recognition effect is poor when the target features are not obvious. The results of the third group of experiments show that the transfer learning mechanism can further improve the recognition accuracy and efficiency on a small sample data set. Its mAP value is 91.16%, the training time is short, and the model converges well. The above results show that the introduction of transfer learning technology in the present invention has certain advantages, taking into account both recognition accuracy and training efficiency.

[0048] In order to explore the model of the present invention that combines transfer learning with double attention mechanism, it has better recognition effect for complex backgrounds. SE, ECA, CBAM, and CA attention mechanisms are introduced for comparison, and F1, Recall, Precision, and AP values ​​are used as evaluation indicators. In order to control the effective comparison of different attention mechanism models, the training parameters, training generations, and attention positions used in the experiment are kept consistent. The model can correctly distinguish the background and foreground under complex day and night conditions, and also has a good degree of attention to the species category of Siberian tigers and leopards.

[0049] Table 2 Three groups of experimental test results

[0050]

[0051] The results show that the overall accuracy of the method adopted by the present invention reaches 94.42%, the F1 value is 0.90, the recall rate is 94.88%, and the accuracy rate is 95.48%, which is better than the five deep learning methods, and the training time is short. Compared with the original YOLO v5 model, the AP value is increased by 7.75%, which greatly improves the detection effect of infrared targets in the wild. The present invention takes a total of 2.35h (141 minutes), which saves a lot of labor costs and time costs compared with traditional methods. It is verified on the wild Siberian tiger and leopard data set, and the recognition effect shows a higher advantage.

[0052] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for identifying Siberian tigers and leopards based on transfer learning and double attention mechanism. It is characterized in that The following steps are involved: (1) The programming language Python and the deep learning framework PyTorch were used to build a deep learning recognition model for Siberian tigers and leopards. The Siberian tiger and leopard deep learning model used the YOLO v5m network as the basic framework, the input layer adopted the Mosaic data enhancement method, the backbone network Backbone used the Focus and CSP structures to extract feature maps, and the Neck module used the spatial pyramid pooling SPP-NET and the path aggregation network PANet structure to aggregate network features. (2) Use the image data of Siberian tigers and leopards taken by infrared cameras and public datasets to construct sample data; (3) Perform noise processing, image expansion, image enhancement, positive and negative sample balance, and image calibration on the sample data to construct a Northeast tiger and leopard sample dataset; (4) Divide the obtained Siberian tiger and leopard sample data set into training samples, test samples, and validation samples; (5) Use the training samples to train the Siberian tiger and leopard deep learning model, set appropriate hyperparameters before training, and select the appropriate gradient descent optimizer and loss function; (6) According to the convergence of the loss function and the improvement of accuracy of the verification sample, the optimal parameters are selected for the entire network to obtain the optimal model; (7) The accuracy of the prediction results is evaluated using four indicators: precision, recall, average precision, and F1 score, and the overall performance of the model is analyzed.

2. The method for identifying Siberian tigers and leopards based on transfer learning and double attention mechanism according to claim 1, Features: The image data in step (2) includes RGB three-band image data and grayscale image data.

3. The Northeast tiger and leopard identification method based on transfer learning and double attention mechanism according to claim 1, Features: The image enhancement and image expansion in step (3) are written in the programming language Python, and the image calibration is performed by annotating with the aid of the target detection tool Labelimg.

4. The method for identifying Siberian tigers and leopards based on transfer learning and double attention mechanism according to claim 1, Features: The step (5) of training the Siberian tiger and leopard deep learning model comprises the following steps: (1) Initialize model parameters; (2) Input training samples and validation samples; (3) Forward propagation; (4) Calculate the error based on the output probability; (5) Back propagation: update model parameters and select an appropriate gradient descent optimization algorithm; (6) Traverse all training data sets and train data according to training generations; (7) When the number of learning times reaches the threshold, the optimal model is obtained.

Citation Information

Patent Citations

  • A wild animal monitoring and analyzing system and method based on a deep convolutional neural network

    CN109829888A

  • Small sample wild animal detection method based on improved YOLOv5

    CN115393618A

  • Arctic cyclone identification method and device based on traditional method and deep learning, and storage medium

    CN116010812A

  • Field crested ibis real-time detection method based on YOLOv5-CBAM

    CN116416490A

Cited By

  • Navigation language instruction automatic generation and evaluation method and device and storage medium

    CN119860761A