Deep learning-based infrared camera animal image intelligent identification method along railway

Abstract

The invention discloses a deep learning-based intelligent identification method for animal images of infrared cameras along a railway. A trunk module, a neck module and a detection head module which are connected in sequence are included; the trunk module comprises an input layer, a spatial pyramid pooling layer, a position space attention layer, two spatial channel decoupling down-sampling layers, three Conv layers and C2f layers, a C2fCIB layer and an output layer, and the trunk module outputs feature maps of five scales; the neck module is composed of a path from bottom to top and a path from top to bottom and comprises an input layer, a space channel decoupling lower sampling layer, a Conv layer, a C2f layer, two dynamic upper sampling layers, three C2fCIB layers and an output layer. The detection head module includes a pair of heads and a pair of multiple heads, each head including regression and classification branches. According to the identification method, a dynamic up-sampling layer DySample based on point sampling is adopted, the characterization capability of the feature map is improved, the model is helped to recover the feature resolution, the classification and positioning accuracy is improved, and therefore the identification precision is improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence recognition technology, and specifically relates to an intelligent recognition method for animal images using infrared cameras along railways based on deep learning, in particular a wild animal recognition method based on a Dy-YOLO network with a dynamic upsampling mechanism. Background Art

[0002] Wildlife identification refers to the process of identifying and classifying different species by observing, recording and analyzing the characteristics, behaviors and habitats of wild animals. Wildlife identification is extremely important in protecting biodiversity, maintaining ecological balance and promoting scientific research. Generally speaking, when railways are built near animal survival and reproduction areas such as mountains, plains or forests, they will restrict the survival and reproduction of wild animals, and even pose a great threat to the survival of wild animals passing through them. The presence of wild animals also poses a great threat to the driving safety of vehicles. Infrared cameras are the key technology for capturing animal images, but the quality of infrared images is easily affected by environmental factors, making it difficult for wild animals to be identified. With the rapid development of artificial intelligence algorithm technology, wildlife identification technology derived from image recognition aims to identify and classify different species, help assess the health of ecosystems, and promote ecological balance and sustainable development.

[0003] Kernel-based feature upsamplers are widely used in wildlife recognition models to gradually restore feature resolution. In order to increase flexibility, learnable upsamplers or dynamic upsampling layers are usually introduced to upsample features. Learnable upsamplers aim to make the upsampling process more flexible and effective through parameterized learning. They can adaptively adjust the generated image features to better meet the needs of specific tasks. However, learnable upsamplers are susceptible to checkerboard effects or are not suitable for advanced visual tasks. The dynamic upsampling layer upsamples features by generating content-aware upsampling kernels through dynamic convolution. However, dynamic convolution introduces heavy workload, and the introduction of high-resolution guided features in the upsampling process even increases the amount of computation. Based on this, a point-based dynamic upsampling layer DySample is introduced in the upsampling process, which dynamically samples feature maps using point resampling methods. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide an intelligent animal recognition method for infrared cameras along railways based on deep learning. The recognition method takes YOLOv10 as the benchmark model. In order to better restore the feature map resolution and explore more effective semantic information, a new point sampling-based dynamic upsampling layer DySample is introduced, and an overall Dy-YOLO network model is proposed.

[0005] In order to solve the above technical problems, the present invention is implemented in the following ways:

[0006] An intelligent animal image recognition method for infrared cameras along railways based on deep learning, including a trunk module, a neck module and a detection head module connected in sequence;

[0007] The backbone module includes an input layer Y, a spatial pyramid pooling layer, a position space attention layer, two spatial channel decoupling downsampling layers, three Conv layers and three C2f layers, a C2fCIB layer and an output layer, wherein the output layer is a feature map of five scales P1, P2, P3, P4 and P5, wherein the feature maps P3, P4 and P5 are input to the neck module for feature fusion;

[0008] The neck module includes an input layer, a spatial channel decoupling downsampling layer, a Conv layer, a C2f layer, two dynamic upsampling layers, three C2fCIB layers and an output layer, wherein the input layer is the feature maps P3, P4 and P5 output by the trunk module, and the output layer is the feature maps Y1, Y2 and Y3 input to the detection head module;

[0009] The detection head module includes a one-to-one head and a one-to-many head, each head is divided into a classification branch and a regression branch, wherein each true value in the one-to-one head corresponds to one positive sample, and each true value in the one-to-many head corresponds to multiple positive samples.

[0010] Furthermore, the specific process of the backbone module is as follows:

[0011] Input features of the input layer Two feature maps are output respectively through two two-dimensional convolution Conv layers (conv2d+BN+SiLU) with a step size of 2 P2 outputs a feature map through a C2f layer and a Conv layer P3 outputs feature maps after a C2f layer and SCDown spatial channel decoupling downsampling layer (conv2d+GroupConv) P4 passes through a C2f layer and a SCDown spatial channel decoupling downsampling layer to reduce the feature resolution by half and double the number of channels, then passes through a C2fCIB layer to halve the number of channels, and finally passes through a SPPF spatial pyramid pooling layer and a PSA position spatial attention layer to output the feature map.

[0012]

[0013] The specific process of the neck module is as follows:

[0014] Input layer input After the DySample dynamic upsampling layer Cat in the channel direction, then output through a C2fCIB layer P4' is sampled by DySample and In the channel direction, Cat, and then output through a C2f layer Y1 passes through a Conv layer (conv+BN+SiLU) and Cat in the channel direction, and then output the feature map through a C2fCIB layer After Y2 passes through an SCDdown spatial channel decoupling downsampling layer, the feature map resolution is reduced by half and the number of channels is doubled. Then it is Cat with P5 in the channel direction, and finally a C2fCIB layer is used to halve the number of channels to output the feature map.

[0015] The specific process of the detection head module is as follows:

[0016] In the one-to-one head, the feature map passes through two Conv layers and a two-dimensional convolution to obtain the prediction box parameters of the one-to-one head After four Conv layers and a two-dimensional convolution, we get the one-to-one classification score. The two are spliced ​​together in the channel direction to obtain the final one-to-one prediction result;

[0017] In one-to-many heads, the feature map passes through two Conv layers and a two-dimensional convolution to obtain the prediction box parameters of one-to-many heads. After two Conv layers and a two-dimensional convolution, we get the classification score of one-to-many. The two are spliced ​​together in the channel direction to obtain the final one-to-many prediction result.

[0018] In order to better restore the feature resolution, the present invention adopts the DySample dynamic upsampling layer and uses grouped convolution to calculate the initial offset O' (offset) and dynamic range factor (scope) of the sampling position.

[0019] The DySample dynamic upsampling layer takes the input features Divide into g groups along the channel dimension, and generate the initial offset O'(offset) and dynamic range factor (scope) respectively through two 3×3 group convolution operations. The specific process expression is as follows:

[0020] O'=GroupConv(X)

[0021] scope = conv2d(X)

[0022] in, s represents the upsampling factor (scale);

[0023] By using the sigmoid function and a static factor, the dynamic range factor is controlled within the range of [0, 0.5] to constrain the offset range of the sampling position. The expression is as follows:

[0024] O=0.5·sigmoid(scope)·O'

[0025] Add the offset O to the original sampling grid G, and then use pixel shuffle to get the sampling set Reshape the input feature X and the sample set S respectively and After that, the upsampled feature map is generated through grid sampling. Its expression is as follows:

[0026] X'=GridSample(R(PixelShuffle(O+G)))

[0027] X' is concatenated with the feature map of the same resolution output by the backbone module in the channel direction, and then subsequent feature fusion is performed.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention proposes a new wildlife recognition network. The Dy-YOLO network not only integrates multi-scale semantic information, but also the neck module can fuse coarse-grained and fine-grained semantic information of multiple scales, enhance the details and clarity of the image, help retain key features, improve recognition accuracy, reduce network parameters, and improve computational efficiency; at the same time, it improves the defects of dynamic upsampling. By combining the DySample dynamic upsampling layer with the network model, point resampling is used to restore the resolution of the feature map, thereby improving the representation ability of the network and improving the accuracy of classification and positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of the Dy-YOLO network structure of the present invention.

[0031] Figure 2 It is a schematic diagram of the backbone module structure of the present invention.

[0032] Figure 3 This is a schematic diagram of the neck module structure of the present invention.

[0033] Figure 4 It is a schematic diagram of the structure of the detection head module of the present invention.

[0034] Figure 5 Schematic diagram of the DySample dynamic upsampling layer structure of the present invention.

[0035] Figure 6 This is a schematic diagram of the wild animal recognition results in the LoTE-Animal wild dataset of the present invention.

[0036] Figure 7 It is a schematic diagram of the wild animal recognition results in the wild_animal dataset of the present invention. DETAILED DESCRIPTION

[0037] The specific implementation of the present invention is further described in detail below with reference to the accompanying drawings and specific examples.

[0038] like Figures 1 to 4 As shown, the intelligent animal image recognition method of infrared cameras along the railway based on deep learning includes a trunk module, a neck module and a detection head module connected in sequence;

[0039] The backbone module includes an input layer Y, a spatial pyramid pooling layer, a position space attention layer, two spatial channel decoupling downsampling layers, three Conv layers and three C2f layers, a C2fCIB layer and an output layer, wherein the output layer is a feature map P1, P2, P3, P4 and P5 of five scales, wherein the feature maps P3, P4 and P5 are input to the neck module for feature fusion and used for subsequent detection;

[0040] The neck module includes an input layer, a spatial channel decoupling downsampling layer, a Conv layer, a C2f layer, two dynamic upsampling layers, three C2fCIB layers and an output layer, wherein the input layer is the feature maps P3, P4 and P5 output by the trunk module, and the output layer is the feature maps Y1, Y2 and Y3 input to the detection head module;

[0041] The detection head module includes a one-to-one head and a one-to-many head, each head is divided into a classification branch and a regression branch, wherein each true value in the one-to-one head corresponds to one positive sample, and each true value in the one-to-many head corresponds to multiple positive samples.

[0042] Furthermore, the specific process of the backbone module is as follows:

[0043] Input features of the input layer Two feature maps are output respectively through two two-dimensional convolution Conv layers (conv2d+BN+SiLU) with a step size of 2 P2 outputs a feature map through a C2f layer and a Conv layer P3 outputs feature maps after a C2f layer and SCDown spatial channel decoupling downsampling layer (conv2d+GroupConv) P4 passes through a C2f layer and a SCDown spatial channel decoupling downsampling layer to reduce the feature resolution by half and double the number of channels, then passes through a C2fCIB layer to halve the number of channels, and finally passes through a SPPF spatial pyramid pooling layer and a PSA position spatial attention layer to output the feature map.

[0044]

[0045] The specific process of the neck module is as follows:

[0046] Input layer input After the DySample dynamic upsampling layer Cat in the channel direction, then output through a C2fCIB layer P4' is sampled by DySample and In the channel direction, Cat, and then output through a C2f layer Y1 passes through a Conv layer (conv+BN+SiLU) and Cat in the channel direction, and then output the feature map through a C2fCIB layer After Y2 passes through an SCDdown spatial channel decoupling downsampling layer, the feature map resolution is reduced by half and the number of channels is doubled. Then it is Cat with P5 in the channel direction, and finally a C2fCIB layer is used to halve the number of channels to output the feature map.

[0047] The specific process of the detection head module is as follows:

[0048] Taking Y1 as an example, in the one-to-one head, the feature map Y1 passes through two Conv layers and a two-dimensional convolution to obtain the prediction box parameters of the one-to-one head After four Conv layers and a two-dimensional convolution, we get the one-to-one classification score. The two are spliced ​​together in the channel direction to obtain the final one-to-one prediction result;

[0049] In one-to-many heads, the feature map Y1 passes through two Conv layers and a two-dimensional convolution to obtain the prediction box parameters of one-to-many heads After two Conv layers and a two-dimensional convolution, we get the classification score of one-to-many. The two heads are concatenated in the channel direction to obtain the final one-to-many prediction result. During training, the two heads are optimized together with the model; during inference, only the one-to-one head is used for prediction.

[0050] In order to better restore the feature resolution, the present invention adopts the DySample dynamic upsampling layer and uses grouped convolution to calculate the initial offset O' (offset) and dynamic range factor (scope) of the sampling position.

[0051] like Figure 5 As shown in the figure, each cube represents the corresponding feature tensor. The variable name of the tensor is located inside each cube, and the size of the tensor is indicated outside each cube. Divide into g groups along the channel dimension, and generate the initial offset O'(offset) and dynamic range factor (scope) respectively through two 3×3 group convolution operations. The specific process expression is as follows:

[0052] O'=GroupConv(X)

[0053] scope = conv2d(X)

[0054] in, s represents the upsampling factor (scale);

[0055] By using the sigmoid function and a static factor of 0.5, the dynamic range factor is controlled within the range of [0, 0.5], constraining the offset range of the sampling position. The expression is as follows:

[0056] O=0.5·sigmoid(scope)·O'

[0057] Add the offset O to the original sampling grid G, and then use pixel shuffle to get the sampling set Reshape the input feature X and the sample set S respectively and After that, the upsampled feature map is generated through grid sampling. Its expression is as follows:

[0058] X'=GridSample(R(PixelShuffle(O+G)))

[0059] Among them, the last dimension 2 of S represents the x and y coordinates, X' is spliced ​​with the feature map of the same resolution output by the backbone module in the channel direction, and then the subsequent feature fusion is performed; the gridsample function uses the position in the sampling set S to resample the input feature X into

[0060] Example:

[0061] The present invention adopts the LoTE-Animal wild dataset and the wild_animal dataset, uses the AdamW optimizer, and sets the momentum and weight decay to 0.937 and 5×10 respectively. -4 , the initial learning rate is 1×10 -3 . The LoTE-Animal wild dataset contains data of 11 endangered wild animals in different seasons, weather, day and night, of which the training set contains 15,385 annotated images, the validation set contains 2,200 annotated images, and the test set contains 4,402 annotated images; the wild_animal dataset contains data of 109 wild animals in day and night environments, with 6,148 annotated images in daytime environments and 4,186 annotated images in night environments. The wild_animal dataset is divided into training set, validation set and test set, of which the training set contains 6,079 annotated images, the validation set contains 2,027 annotated images, and the test set contains 2,027 annotated images; all animal data in daytime environments constitute the dataset wild_day, which contains 6,148 annotated images; all animal data in night environments constitute the dataset wild_night, which contains 4,186 annotated images; the models trained on the wild_all dataset are tested on wild_all, wild_day and wild_night respectively. All training and test set images are resized to 640×640 and normalized.

[0062] In order to evaluate the performance of the recognition method of the present invention, this application uses the average precision (mAP) and mAP 50 Two indicators are used, and the calculation formulas of each indicator are as follows:

[0063]

[0064] Among them, TP represents the number of pixels correctly detected as positive samples, FP represents the number of pixels incorrectly detected as positive samples, FN represents the number of pixels incorrectly detected as negative samples, AP represents the average accuracy of a class of samples, and mAP represents the average of all categories of average precision (AP). 50 Represents the mAP value when the IOU threshold is 0.5. mAP and mAP 50 The value range is [0, 1]. The larger the mAP value, the closer the algorithm recognition result is to the labeled result, and the better the recognition effect is. If the precision rate P is lower, the recognition is less accurate. If the recall rate R is lower, more animals are not recognized.

[0065] The present invention conducted quantitative experiments on two datasets, LoTE-Animal wild and wild_animal. In the wild_animal dataset, animal data in diurnal environment, daytime environment and night environment were tested respectively. The quantitative results of the LoTE-Animal wild dataset are shown in Figure 2. Figure 6 As shown in the figure, each row corresponds to an animal image. The first column is the original animal image, the second column is the true value image, and the third to sixth columns are the model recognition results. By identifying the mAP values ​​listed at the bottom of the image, Dy-YOLO's animal detection results in day and night environments are closest to the true values. The quantitative results of the wild_animal dataset are shown in the figure. Figure 7 As shown, each row corresponds to an animal image, the first column is the original animal image, the second column is the true value image, the third to sixth columns are the model recognition results, the first three rows are the animal detection results in the day and night environment, the middle three rows are the animal detection results in the day environment, and the last three rows are the animal detection results in the night environment;

[0066] The detection performance of different network structures on the LoTE-Animal wild dataset is shown in the following table. Dy-YOLO performs better than other network structures in both indicators, and the detection performance is significantly improved. Compared with YOLOv10, Dy-YOLO's mAP and mAP 50 The increases are 1.2% and 0.8% respectively, indicating that the dynamic upsampling layer DySample can improve the performance of YOLOv10.

[0067]

[0068] The comparison of detection performance of different network structures on the wild_animal dataset is shown in the following table. Dy-YOLO performs better than other network structures in both indicators, and the detection performance has been significantly improved. Compared with YOLOv10, Dy-YOLO's mAP and mAP in day and night environments are 50 The mAP and mAP in daylight environment are improved by 0.6% and 0.7% respectively. 50 The improvements are 0.4% and 1% respectively, indicating that the DySample dynamic upsampling layer can improve the performance of Dy-YOLO.

[0069]

[0070] At the same time, in order to prove the effectiveness of the DySample dynamic upsampling layer, the present invention uses YOLOv10 as the benchmark model, replaces the nearest neighbor interpolation method in the two upsampling layers in its neck module with DySample, and conducts an ablation experiment on the wild_animal dataset. The results are shown in the following table. Compared with YOLOv10, the mAP value and mAP value of DySample are 50 The increase is 0.6% and 0.7% respectively. Therefore, the evaluation index of Dy-YOLO is better than that of YOLOv10, indicating that the dynamic upsampling layer DySample can improve the performance of Dy-YOLO.

[0071]

[0072] The above description is only an implementation mode of the present invention. It is stated again that for ordinary technicians in this technical field, several improvements can be made to the present invention without departing from the principle of the present invention. These improvements are also included in the protection scope of the claims of the present invention.

Claims

1. A method for intelligently identifying animal images from infrared cameras along railways based on deep learning, characterized by: It includes a trunk module, a neck module and a detection head module which are connected in sequence; The backbone module includes an input layer Y, a spatial pyramid pooling layer, a position space attention layer, two spatial channel decoupling downsampling layers, three Conv layers and three C2f layers, a C2fCIB layer and an output layer, wherein the output layer is a feature map of five scales P1, P2, P3, P4 and P5, wherein the feature maps P3, P4 and P5 are input to the neck module for feature fusion; The neck module includes an input layer, a spatial channel decoupling downsampling layer, a Conv layer, a C2f layer, two dynamic upsampling layers, three C2fCIB layers and an output layer, wherein the input layer is the feature maps P3, P4 and P5 output by the trunk module, and the output layer is the feature maps Y1, Y2 and Y3 input to the detection head module; The detection head module includes a one-to-one head and a one-to-many head, each head is divided into a classification branch and a regression branch, wherein each true value in the one-to-one head corresponds to one positive sample, and each true value in the one-to-many head corresponds to multiple positive samples.

2. The method for intelligently identifying animal images from infrared cameras along railways based on deep learning according to claim 1 is characterized in that: The specific process of the backbone module is as follows: Input features of the input layer Two feature maps are output respectively through two two-dimensional convolution Conv layers with a step size of 2 P2 outputs feature maps through a C2f layer and a Conv layer P3 outputs feature maps after a C2f layer and SCDown spatial channel decoupling downsampling layer P4 passes through a C2f layer and a SCDown spatial channel decoupling downsampling layer to reduce the feature resolution by half and double the number of channels, then passes through a C2fCIB layer to halve the number of channels, and finally passes through a SPPF spatial pyramid pooling layer and a PSA position spatial attention layer to output the feature map. The specific process of the neck module is as follows: Input layer input After the DySample dynamic upsampling layer Cat in the channel direction, then output through a C2fCIB layer P4' is sampled by DySample and In the channel direction, Cat, and then output through a C2f layer Y1 passes through a Conv layer and Cat in the channel direction, and then output the feature map through a C2fCIB layer After Y2 passes through an SCDdown spatial channel decoupling downsampling layer, the feature map resolution is reduced by half and the number of channels is doubled. Then it is Cat with P5 in the channel direction, and finally a C2fCIB layer is used to halve the number of channels to output the feature map. The specific process of the detection head module is as follows: In the one-to-one head, the feature map passes through two Conv layers and a two-dimensional convolution to obtain the prediction box parameters of the one-to-one head After four Conv layers and a two-dimensional convolution, we get the one-to-one classification score. The two are spliced ​​together in the channel direction to obtain the final one-to-one prediction result; In one-to-many heads, the feature map passes through two Conv layers and a two-dimensional convolution to obtain the prediction box parameters of one-to-many heads. After two Conv layers and a two-dimensional convolution, we get the classification score of one-to-many. The two are spliced ​​together in the channel direction to obtain the final one-to-many prediction result.

3. The method for intelligently identifying animal images from infrared cameras along railways based on deep learning according to claim 2 is characterized in that: The DySample dynamic upsampling layer takes the input features Divide into g groups along the channel dimension, and generate the initial offset O' and dynamic range factor respectively through two 3×3 grouped convolution operations. The specific process expression is as follows: O'=GroupConv(X) scope = conv2d(X) in, s represents the upsampling factor; By using the sigmoid function and a static factor, the dynamic range factor is controlled within the range of [0, 0.5] to constrain the offset range of the sampling position. The expression is as follows: O=0.5·sigmoid(scope)·O' Add the offset O to the original sampling grid G, and then get the sampling set by pixel transformation Reshape the input feature X and the sample set S respectively and After that, the upsampled feature map is generated through grid sampling Its expression is as follows: X'=GridSample(R(PixelShuffle(O+G))).

Images