A method for detecting wild animals based on multiple encoding and decoding

Abstract

The application discloses a kind of based on multiple encoding and decoding wild animal detection method, belong to computer vision, deep learning field.Train a wild animal detection model comprising multiple encoding and decoding module, multiple encoding and decoding module is constituted by several single encoding and decoding structure, single encoding and decoding structure is equipped with internal skip connection and top-down down-sampling feedback connection, and external skip connection and bottom-up up-sampling feedback connection are equipped between adjacent two single encoding structure;The trained wild animal detection model is deployed to server, according to the confidence of detection result, the video segment containing wild animal is saved to corresponding local folder.The multiple encoding and decoding module in the model of the application can deeply mine the information difference between low layer and high layer, utilize feedback connection to carry out deep-level fusion to high-level information and low-level information, improve feature extraction capability, can accurately detect small target animals, significantly reduce the false detection rate of animals at night.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning, specifically to a wildlife detection method based on multiple encoding and decoding. Background Technology

[0002] Wild animals are an important component of ecosystems and play a vital role in maintaining ecological balance. Wildlife detection technology can help people better understand the distribution and numbers of wild animals, providing a scientific basis for ecosystem management and protection. Wildlife tourism is an emerging tourism industry that is significant for boosting local economies and tourism development. Wildlife detection technology can help people better understand the activity and distribution of wild animals, providing a scientific basis for wildlife tourism. Therefore, collecting and analyzing wildlife information is crucial for better understanding the survival status of wild animals. Currently, the main method of wildlife protection is to deploy a large number of cameras in nature reserves for fixed-point recording. However, due to various factors such as falling leaves, rain, and wind, the cameras constantly trigger recording functions, resulting in a large amount of video footage containing no wild animals being saved. Therefore, a large amount of manual screening is required to select segments in the videos that actually contain wild animals. However, video screening is a stress-intensive task, requiring long-term high-pressure and high-intensity work environments, which can easily lead to false positives and false negatives. Moreover, frequently hiring a large number of people for video screening consumes a significant amount of human resources. Therefore, utilizing deep learning technology to detect the presence of wild animals in videos and improve work efficiency is of great significance.

[0003] In recent years, researchers both domestically and internationally have conducted a series of studies on the classification, detection, and identification of wild animals. With the continuous advancement of computer vision and deep learning algorithms, object detection algorithms have begun to be applied in various fields due to their high accuracy and fast detection speed. Deep learning-based object detection networks are generally divided into two types: one is two-stage detection algorithms, typical methods including the R-CNN series algorithms and various improved algorithms based on R-CNN; the other is one-stage detection algorithms, such as SSD and its derivative networks, and the YOLO series algorithms. Among the YOLO series, YOLOv5 is a fast, high-accuracy, versatile, and easy-to-use object detection algorithm, suitable for various application scenarios and with broad application prospects. YOLOv5 achieves faster detection speed through a series of optimization measures, such as SPP convolution, PAN convolution, CSP convolution, and model pruning. YOLOv5 also employs a series of improvements, such as the GIoU loss function and class balancing strategies, enabling higher detection accuracy.

[0004] However, in real-world applications within nature reserves, wild animals are often small targets, and their fur color blends in with the surrounding background, such as trees. Furthermore, some wild animals are nocturnal, and these factors all affect the YOLOv5 network's ability to extract animal features, thus reducing the model's detection accuracy. Therefore, improving the accuracy of wild animal target detection remains a key challenge for researchers in this field. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a wildlife detection method based on multiple encoding and decoding. Compared to other target detection algorithms, this method utilizes multiple encoding and decoding modules to deeply mine the information differences between low-level and high-level layers, and employs feedback connections to deeply fuse high-level and low-level information, thereby enhancing the network's feature extraction capabilities.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A wildlife detection method based on multiple encoding and decoding includes the following steps:

[0008] Step 1: Extract frames from the video captured by cameras in the nature reserve, use the images as a sample dataset, and classify and label each wild animal in the sample images;

[0009] Step 2: Establish a wildlife detection model consisting of a backbone network, a multi-encoder / decoder module, and a prediction module. The multi-encoder / decoder module consists of several single encoder / decoder structures. Each single encoder / decoder structure has n layers, with the number of layers increasing sequentially from top to bottom. It includes n encoders and n-1 decoders connected in series. Each single encoder / decoder structure has internal skip connections and top-down downsampling feedback connections. There are external skip connections and bottom-up upsampling feedback connections between two adjacent single encoder / decoder structures.

[0010] Step 3: Train the wildlife detection model using the sample dataset from Step 1;

[0011] Step 4: Deploy the trained wildlife detection model to the server;

[0012] Step 5: Use the trained wildlife detection model to detect wildlife in the videos captured by cameras in the nature reserve;

[0013] Step 6: Based on the confidence level of the detection results, save the video clips containing wild animals to the corresponding local folder.

[0014] Furthermore, the multi-encoding / decoding module is located between the backbone network and the prediction module, and is used to perform deep-level fusion of the features extracted by the backbone network, and use the fused features for prediction.

[0015] Furthermore, in the aforementioned single encoder-decoder structure, the internal skip connection specifically refers to: the input of the first-layer encoder F1 is the external input feature map, and any layer F1 of the second to nth layers of the encoder... i The input is the (i-1)th layer encoder F. i-1 The output of any layer F' in the decoder from layer 1 to n-2. i The input is the i-th layer encoder F. i The output of the (i+1)th layer decoder F' i+1 The output of the (n-1)th layer decoder F' n-1 The input is the (n-1)th layer encoder F. n-1 The output of the nth layer encoder F n The output;

[0016] Furthermore, in the aforementioned single codec structure, the top-down downsampling feedback connection specifically refers to: any layer F in the encoder from layer 2 to layer n-1 of the single codec structure. i Next, we set up the downsampling feedback connection layer, named M sequentially from top to bottom. i , 2≤i≤n-1; each downsampling feedback connection layer M i The output is used as decoder F' i The input, in addition, the (n-1)th downsampled feedback connection layer M n-1 The output is also used as encoder F n Input;

[0017] The input and calculation formula for each downsampling feedback connection layer are as follows:

[0018]

[0019] Among them, f 1×1 F represents a 1×1 convolution operation. n-1 M represents the (n-1)th layer encoder. n-1 This indicates the (n-1)th downsampling feedback connection layer.

[0020] Furthermore, the external skip connection between two adjacent single coding structures specifically refers to the decoder F' of the k-th single coding / decoding structure. k,i The encoder F with the (k+1)th single encoder-decoder structure k+1,i Connect them, 2≤i≤n, and pass the data through decoder F' k,i Transmitted to encoder F k+1,i .

[0021] Furthermore, the bottom-up upsampling feedback connection between two adjacent single coding structures specifically refers to: any layer F' of the decoder from the 2nd to the (n-1th)th layers of the kth single coding structure.k,i The corresponding layer F of the encoder from layer 2 to layer (n-1) of the (k+1)th single coding structure k+1,i An upsampling feedback connection layer is set up between them, named P sequentially from bottom to top. i , n-1≥i≥2; each layer upsampled feedback connection layer P i The output of the encoder F k+1,i In addition to the input, the output of the second-layer upsampled feedback connection layer P2 also serves as the decoder F'. k,1 enter;

[0022] The input and calculation formula for each upsampling feedback connection layer are as follows:

[0023]

[0024] Among them, f 1×1 F' represents a 1×1 convolution operation. k,n-1 F represents the (n-1)th layer decoder with the kth single coding structure. k,n P represents the nth layer encoder with the kth single coding structure. n-1 This indicates the (n-1)th upsampling feedback connection layer.

[0025] Furthermore, the decoder F' of the kth single codec structure k,1 The encoder F with the (k+1)th single encoder-decoder structure k+1,1 shared.

[0026] The wildlife detection method based on multiple encoding and decoding of the present invention has the following advantages:

[0027] This invention addresses the challenges of wildlife detection tasks, such as small target objects, target fur color similar to surrounding trees and other background colors, and the fact that wildlife is mostly active at night. It employs a multi-encoder / decoder module that integrates the feedback of encoding and decoding to the backbone network. This allows the features obtained from retraining the backbone network to better adapt to wildlife detection tasks, achieving deep fusion of high-level and low-level information, improving the network's feature extraction capabilities, and reducing the interference of environmental factors on the model's detection results. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating a wildlife detection method based on multiple encoding and decoding, as shown in an embodiment of the present invention;

[0029] Figure 2 This is a structural diagram of the wildlife detection model shown in an embodiment of the present invention;

[0030] Figure 3 This is a structural diagram of the multiple codec module shown in an embodiment of the present invention;

[0031] Figure 4 This is the loss convergence graph during the training phase of this invention;

[0032] Figure 5 These are the test results for individual wild animals. Detailed Implementation

[0033] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the invention.

[0034] A wildlife detection method based on multiple encoding and decoding, such as Figure 1 As shown, it includes the following steps:

[0035] Step 1: Extract frames from the video captured by cameras in the nature reserve, use the images as a sample dataset, and classify and label each wild animal in the sample images;

[0036] In this step, nature reserve cameras typically record automatically based on a trigger mechanism (e.g., wild animals, fallen leaves, etc. entering the recording field of view), and automatically save a video clip every 2-10 seconds. From the video samples, sample images containing wild animals are extracted, and the LabelMe tool is used to label all animals in the wild animal sample images. Considering that some wild animals prefer nocturnal activity, the dataset includes data collected under different lighting conditions. To increase the diversity of the data samples, random brightness processing is applied to the sample images. Furthermore, to simulate the shooting effect of different camera angles, the sample images need to be rotated at different angles. To simulate occlusion, random mosaic occlusion processing is applied to the sample images. These processing methods effectively improve the model's adaptability to different lighting and occlusion conditions.

[0037] Step 2: Establish a wildlife detection model that integrates multiple codec modules (MCM).

[0038] like Figure 2 As shown, the wildlife detection model consists of a backbone network, a multi-encoder / decoder module, and a prediction module (classification and regression).

[0039] The backbone network and prediction module mentioned above adopt the YOLOv5 model, which will not be described in detail here.

[0040] The aforementioned multi-encoder / decoder module consists of several single encoder / decoder structures, each comprising two main parts: an encoder and a decoder. The encoder consists of a series of convolutional and pooling layers, used to extract high-level feature representations from the input image. The decoder consists of a series of upsampling and convolutional layers, used to reconstruct the feature representations extracted by the encoder into feature information of the original image.

[0041] In this embodiment, Figure 3 In (a), F1, F2, F3, F4, and F5 are encoder structures, while F'1, F'2, F'3, and F'4 are decoder structures. Five encoders and four decoders are connected in series. The subscript i indicates the layer number of the encoder / decoder, increasing sequentially from top to bottom. The encoder F in the i-th layer... i and the i-th layer decoder F' i There are also internal skip connections, meaning that the input of the first-layer encoder F1 is the external input image, and the input of the i-th layer encoder F... i The input is the (i-1)th layer encoder F. i-1 The output of the i-th layer decoder F' (2≤i≤5) i The input is the i-th layer encoder F. i The output of the (i+1)th layer decoder F' i+1 The output of the encoder (1≤i≤3) is used, and the input of the fourth layer decoder F'4 is the output of the fourth layer encoder F4 and the output of the fifth layer encoder F5. The aforementioned encoder-decoder structure can combine low-level features with high-level features, and connect feature maps from different levels in the encoder with corresponding feature maps in the decoder, thereby passing richer contextual information to the decoder to generate more accurate results.

[0042] exist Figure 3 Based on the encoding / decoding structure described in (a), this invention adds a top-down downsampling feedback connection layer, located after the intermediate encoder of each layer (layers 2-4 encoders), and named M2, M3, and M4 sequentially from top to bottom. The downsampling feedback connection layer M in the i-th layer is located at the bottom. i It integrates the encoded information from layer (i-1) and layer 1 (2≤i≤4), fusing high-frequency and low-frequency information, with the structure as follows: Figure 3 As shown in (b), M2, M3, and M4 form a top-down downsampling feedback connection layer. M2 integrates information from F1 and F2, M3 integrates information from F3 and M2, and M4 integrates information from F4 and M3. The information integrated by M4 is input to F5. In addition, M... i The integrated information is also input into F 'i .

[0043] In summary, the formula for the downsampling feedback connection layer is shown in (1):

[0044]

[0045] Among them, f 1×1 This represents a 1×1 convolution operation.

[0046] Multiple codec modules (MCMs) are stacked from single codec structures, further fusing shallow low-frequency information with deep high-frequency information, such as... Figure 3 As shown in (c), where F k,i F' represents the i-th layer encoder with the k-th single coding structure. k,i M represents the i-th layer decoder of the k-th single coding structure. k,i Let F' represent the i-th feedback connection layer of the k-th single coding structure. For two adjacent single coding structures, the decoder F' of the previous single coding structure... k,1 The encoder F with the subsequent single codec structure k+1,1 In addition, external skip connections and bottom-up upsampling feedback connection layers are added between two adjacent single coding structures.

[0047] Taking a multicodec module (MCM) obtained by stacking two single codec structures as an example, the external jump connection is: the decoder F' of the first single codec structure k,i encoder F with a second single codec structure k+1,i Connected (2≤i≤5).

[0048] The upsampling feedback connection layer is located between the intermediate layer decoder (layers 2-4 decoder) of the preceding single coding structure and the intermediate layer encoder (layers 2-4 encoder) of the following single coding structure. Named P4, P3, and P2 from bottom to top, it integrates the information differences between the two single coding structures. The upsampling feedback connection layer P is located at layer i. i It integrates information from layer i+1 and layer i; P4 integrates information from F. 1,5 With F' 1,4 The information in P3 integrates P4 and F' 1,3 The information in P2 integrates P3 and F' 1,2 The information, and the information integrated by P2, is input into F'. 1,1 In addition, P i The integrated information is also input into F 2,i .

[0049] In summary, the formula for the upsampling feedback connection layer is shown in (2):

[0050]

[0051] The encoding / decoding structure in the middle acts as both an encoder and decoder compared to the structures on the left and right. Through continuous encoding and decoding, the differences between shallow and deep layers are explored, thereby improving the network's feature extraction capabilities.

[0052] Step 3: Train the wildlife detection model using the sample dataset from Step 1, specifically as follows:

[0053] Step 3.1: Use the labeled training set images as input data to train the wildlife detection model, and calculate the loss value during training to determine whether the model training has converged. This invention uses the GIoU loss function for training, which has a smaller loss value than the original loss function. The change in loss value during training is as follows: Figure 4 As shown.

[0054] Step 3.2: After repeated iterative calculations, the loss value of the wildlife detection model gradually converges, and finally a stable and reliable model is obtained.

[0055] Step 4: Deploy the trained model to the server.

[0056] Step 5: Use the trained wildlife detection model to detect wildlife in the videos captured by cameras in the nature reserve. Specifically:

[0057] Step 5.1: Acquire real-time video, extract frames from the video according to the preset frame rate, and use them as input to the model.

[0058] Step 5.2: The backbone network of the YOLOv5 model extracts features from the input image. The feature maps extracted by the backbone network are then processed by multiple encoding and decoding modules to deeply mine the information differences between low-level and high-level layers. Feedback connections are then used to fuse high-level and low-level information at a deeper level, generating fused feature maps. After passing through the YOLOv5 prediction layer, a series of prediction boxes, along with the coordinates and confidence scores of each prediction box, are obtained on each fused feature map. Figure 2 The classification regression section yields the predicted bounding box coordinates B and the confidence class score C.

[0059] Step 5.3: Filter all predicted boxes according to the threshold, and use the coordinates, categories and confidence scores of the filtered predicted boxes as the output of the network.

[0060] Step 6: Based on the confidence level of the detection results, save the video clips containing wild animals to the corresponding local folder.

[0061] If the input video contains wild animal targets, and the proportion of detected target frames with a confidence level greater than or equal to 0.75 exceeds 85% of all sampled frames in the video, then the video segment is saved to the corresponding animal category folder. If the proportion of detected target frames is less than 85%, the video is saved to the folder requiring manual judgment. If no wild animal targets are detected in the input video, then the video is not saved.

[0062] The effects of the present invention will be illustrated below with reference to specific embodiments.

[0063] Dataset: To evaluate the performance of this invention, this embodiment collected 3915 images from surveillance cameras in a nature reserve under different scenes, climates, and brightness conditions, containing 4010 labeled objects. The dataset comprises 10 categories, with a large number of small object samples; for example, there are 864 samples of objects classified as mice and 570 samples of objects classified as squirrels. Finally, the wildlife dataset was divided into training and testing sets in an 8:2 ratio.

[0064] Experimental environment: The implementation environment of this invention is PyTorch 1.2, CUDA 10.1, the server operating system is Linux 16.04, and the GPU model is Tesla V100-SXM2.

[0065] Experimental parameters: The training dataset was in VOC format, with 8 training data points per batch. The momentum size was set to 0.9. When using non-maximum suppression, the confidence threshold was set to 0.5, and the epoch count was 300. The learning rate was adjusted on demand, with an initial learning rate of 0.005, decreasing at the 100th and 200th epochs with a decay coefficient of 0.001. Stochastic gradient descent was used to optimize the network parameters, with a weight decay coefficient of 0.0001. Evaluation criteria: The mean accuracy (mAP) metric was used to evaluate the detection performance of the wildlife detection model.

[0066] Experimental results: mAP: 93.5%, and the detection effect is shown in the figure below. Figure 5 As shown, the model of the present invention has good recognition effect and high accuracy.

[0067] The above examples are merely specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments and many variations are possible. All variations that can be directly derived or conceived by those skilled in the art from the disclosure of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A wildlife detection method based on multiple encoding and decoding, characterized in that, Includes the following steps: Step 1: Extract frames from the video captured by cameras in the nature reserve, use the images as a sample dataset, and classify and label each wild animal in the sample images; Step 2: Establish a wildlife detection model consisting of a backbone network, a multi-encoder / decoder module, and a prediction module. The multi-encoder / decoder module consists of several single encoder / decoder structures. Each single encoder / decoder structure has n layers, with the number of layers increasing sequentially from top to bottom. It includes n encoders and n-1 decoders connected in series. Each single encoder / decoder structure has internal skip connections and top-down downsampling feedback connections. There are external skip connections and bottom-up upsampling feedback connections between two adjacent single encoder / decoder structures. In the aforementioned single encoder-decoder structure, the internal skip connections are specifically as follows: the input to the first-layer encoder F1 is the external input feature map, and any layer F1 in the second to nth layers of the encoder... i The input is the (i-1)th layer encoder F. i-1 The output of any layer F' in the decoder from layer 1 to n-2. i The input is the i-th layer encoder F. i The output of the (i+1)th layer decoder F' i+1 The output of the (n-1)th layer decoder F' n-1 The input is the (n-1)th layer encoder F. n-1 The output of the nth layer encoder F n The output; In the aforementioned single codec architecture, the top-down downsampling feedback connection specifically refers to: any layer F in the encoder from layer 2 to layer n-1 of the single codec architecture. i Next, we set up the downsampling feedback connection layer, named M sequentially from top to bottom. i , 2≤i≤n-1; each downsampling feedback connection layer M i The output is used as decoder F' i The input, in addition, the (n-1)th downsampled feedback connection layer M n-1 The output is also used as encoder F n Input; The input and calculation formula for each downsampling feedback connection layer are as follows: Among them, f 1×1 F represents a 1×1 convolution operation. n-1 M represents the (n-1)th layer encoder. n-1 This indicates the (n-1)th downsampling feedback connection layer; The bottom-up upsampling feedback connection between two adjacent single coding structures is specifically defined as follows: in any layer F' of the decoder from layer 2 to layer (n-1) of the k-th single coding structure. k,i The corresponding layer F of the encoder from layer 2 to layer (n-1) of the (k+1)th single coding structure k+1,i An upsampling feedback connection layer is set up between them, named P sequentially from bottom to top. i , n-1≥i≥2; each layer upsampled feedback connection layer P i The output of the encoder F k+1,i In addition to the input, the output of the second-layer upsampled feedback connection layer P2 also serves as the decoder F'. k,1 enter; The input and calculation formula for each upsampling feedback connection layer are as follows: Among them, f 1×1 F' represents a 1×1 convolution operation. k,n-1 F represents the (n-1)th layer decoder with the kth single coding structure. k,n P represents the nth layer encoder with the kth single coding structure. n-1 This indicates the (n-1)th upsampling feedback connection layer; Step 3: Train the wildlife detection model using the sample dataset from Step 1; Step 4: Deploy the trained wildlife detection model to the server; Step 5: Use the trained wildlife detection model to detect wildlife in the videos captured by cameras in the nature reserve; Step 6: Based on the confidence level of the detection results, save the video clips containing wild animals to the corresponding local folder.

2. The wildlife detection method based on multiple encoding and decoding according to claim 1, characterized in that, The cameras in the nature reserve automatically record video based on a trigger mechanism, and automatically save a video clip every 2 to 10 seconds.

3. The wildlife detection method based on multiple encoding and decoding according to claim 1, characterized in that, The backbone network and prediction module in the wildlife detection model adopt the backbone network and prediction module of the YOLOv5 model.

4. The wildlife detection method based on multiple encoding / decoding according to claim 1 or 3, characterized in that, The aforementioned multi-encoder / decoder module is located between the backbone network and the prediction module. It is used to perform deep fusion of the features extracted by the backbone network and to use the fused features for prediction.

5. The wildlife detection method based on multiple encoding and decoding according to claim 1, characterized in that, The external skip connection between two adjacent single coding structures is specifically: the decoder F' of the kth single coding-decoding structure. k,i The encoder F with the (k+1)th single encoder-decoder structure k+1,i Connect them, 2≤i≤n, and pass the data through decoder F' k,i Transmitted to encoder F k+1,i .

6. The wildlife detection method based on multiple encoding and decoding according to claim 1, characterized in that, The decoder F' with the k-th single codec structure k,1 The encoder F with the (k+1)th single encoder-decoder structure k+1,1 shared.

7. The wildlife detection method based on multiple encoding and decoding according to claim 1, characterized in that, In step 6, if the input video contains wild animal targets, and the proportion of detected target frames with a confidence level greater than or equal to 0.75 exceeds 85% in all sampled frames of the video, then the video segment is saved to the corresponding animal category folder. If the detected target frames account for no more than 85%, the video will be saved to a folder that requires manual judgment. If no wildlife targets are detected in the input video, the video will not be saved.

Images