Marine mammal individual identification method

Abstract

The invention provides a marine mammal individual identification method, which belongs to the field of computer image identification, is based on a convolutional neural network model, combines an attention mechanism, multi-scale feature processing and a depth separable convolution technology, and comprises four steps of data acquisition, data enhancement, model construction and model evaluation. Through the application of the method, the facial key features of the marine mammals can be accurately captured, the parameter quantity and the calculation demand of model operation are reduced, and efficient and accurate identification of the marine mammal individuals under the background of a small data set is realized; and the marine mammal individual population identification challenges of high cost, low efficiency, unfriendliness and the like in the traditional method are solved. Therefore, the marine mammal individual identification method has good marine mammal individual identification capability, and can be widely applied to the fields of marine species investigation monitoring and protection management.

Description

Technical Field

[0001] The present invention relates to the technical field of computer image recognition, and in particular to a method for identifying individual marine mammals. Background Art

[0002] Individual identification and filing of rare and endangered marine mammals is of great significance in understanding their population size, monitoring population dynamics, and implementing conservation management. Generally speaking, when choosing individual identification methods, people need to weigh the cost, timeliness, and accuracy. The main methods for individual identification of marine mammals currently include manual surveys, satellite beacon tracking, and DNA testing.

[0003] The inventors of the present application have found in their research that the survey cost of this method of conducting manual surveys on marine mammal populations with the help of icebreakers and airplanes is high, the survey time and survey route are restricted by geographical environment conditions and prior knowledge, and the accuracy of individual identification is affected by subjective experience. Satellite beacon tracking surveys can obtain the continuous activity status of marine mammal individuals within a certain period of time, but most beacons will fall off from January to March, and the timing of releasing harbor seals in the wild is accidental, which is not practical in actual large-scale monitoring. Although DNA detection based on blood samples and the environment can be used for individual identification of marine mammals, this method has a high individual identification rate, but its cost is high and the encounter rate is low, making it difficult to be widely used in individual identification and monitoring of wild populations. In summary, although many attempts have been made in the past to investigate and monitor marine mammal populations (individuals), these methods have failed to achieve effective identification of individuals.

[0004] In recent years, face recognition based on facial features has brought new ideas to the identification of individual wild animals. In the field of computer image recognition, the progress of deep learning technology has provided strong support for computer image feature learning. Therefore, this patent proposes an efficient, convenient, low-cost and operational method for individual identification of marine mammals based on image data and convolutional neural network models. Summary of the invention

[0005] The embodiment of the present application provides a method for identifying individual marine mammals, which aims to accurately identify individual marine mammals and solve the challenges of high cost, low efficiency and unfriendly problems in traditional methods, thereby providing scientific and technological support for the refined protection and management of marine mammals.

[0006] In order to solve the above technical problems, the technical solution proposed in this application is:

[0007] The present invention provides a method for identifying individual marine mammals, comprising:

[0008] Step 1: Data collection, obtaining image data of marine mammals;

[0009] Step 2: Data enhancement: using data enhancement technology to randomly enhance the original image data;

[0010] Step 3: Model construction, based on convolutional neural networks and facial features, a marine mammal individual recognition model is constructed. The model is based on the Darknet network model structure, integrating attention mechanism and multi-scale feature processing technology to achieve facial recognition of different marine mammal individuals.

[0011] Furthermore, in step 3, a downsampling convolution layer is used to perform feature extraction and dimensionality reduction on the enhanced image data, and the downsampling convolution layer uses a convolution kernel of 3×3 size and step size of 2; in the feature enhancement layer (P-AC), a feature extraction (SlimC2F) module is used to connect multiple convolution layers to extract and fuse the enhanced image data, and at the same time, the attention mechanism is combined to enhance the model's recognition ability for key facial features of marine mammals; the feature information of image data of different scales is integrated through a multi-scale feature fusion layer (SPPF); each channel of the image data is averaged and pooled through a global average pooling layer (GAP) to generate a final feature vector; finally, the feature vector is sent to a fully connected layer (FC) for classification and recognition of individual marine mammals.

[0012] Furthermore, in the feature enhancement layer (P-AC) of step 3, the deep learning-based feature enhancement layer (P-AC) first uses the lightweight improved C2F structure (CSP Bottleneck with 2Convolutions Fast) to extract and fuse the features of the facial images of marine mammals, and then introduces the CBAM module (Convolutional Block Attention Module) to further enhance the recognition of key feature areas of the faces of marine mammals.

[0013] Furthermore, in the feature extraction (SlimC2F) module of step 3, the SlimC2F structure divides the input feature map into two parts based on the number of channels for processing. One part is directly transmitted, and the other part is subjected to residual processing and then merged with the directly transmitted part in the channel dimension. In the residual processing, the residual block adopts 3×3 depthwise separable convolution, and the depthwise separable convolution splits the standard convolution into two steps: depthwise convolution and point-by-point convolution.

[0014] Furthermore, in step 3, the attention mechanism introduces a convolutional block attention module (CBAM), which enhances the response to key channel features and highlights important areas in the input image data by applying an attention mechanism to the input image data in the channel and spatial dimensions respectively.

[0015] Furthermore, in step 3, the multi-scale feature fusion layer SPPF (Spatial Pyramid Pooling-Fast) captures multi-scale features in the image through pooling windows of different sizes, thereby enhancing the robustness of the model.

[0016] Furthermore, the model can more effectively capture local detail features of 1×1 and 3×3 receptive fields, fuse medium-range contextual information of 5×5 receptive fields, and understand the global layout of 7×7 receptive fields in image data.

[0017] Furthermore, it also includes:

[0018] Step 4: Model evaluation: Use accuracy, precision, recall, and F1-score (the harmonic mean of precision and recall) to evaluate the performance of the individual recognition model.

[0019] On the other hand, the present application also provides an electronic device, comprising: a processor; a memory for storing a computer program; wherein the processor is used to execute the computer program stored in the memory to implement the aforementioned method for identifying individual marine mammals.

[0020] On the other hand, the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements the aforementioned method for identifying individual marine mammals.

[0021] Compared with the prior art, the method for identifying individual marine mammals of the present invention has achieved the following beneficial technical effects:

[0022] This method can accurately capture the key facial features of marine mammals, reduce the number of parameters and computing requirements of convolutional model operations, achieve efficient and accurate identification of individual marine mammals in the context of a smaller data set, and solve the challenges of marine mammal individual population identification such as high cost, low efficiency and unfriendlyness in traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A flow chart of a method for identifying individual marine mammals provided in an embodiment of the present invention.

[0025] Figure 2 A schematic diagram of facial images of a method for identifying individual marine mammals provided by an embodiment of the present invention.

[0026] Figure 3 A schematic diagram of image enhancement for a method for identifying individual marine mammals provided in an embodiment of the present invention.

[0027] Figure 4 A DarkNet improved network structure diagram of a method for identifying individual marine mammals provided in an embodiment of the present invention.

[0028] Figure 5 A network structure visualization diagram of a method for identifying individual marine mammals provided in an embodiment of the present invention.

[0029] Figure 6 A depth-separable convolution structure diagram of a method for identifying individual marine mammals provided in an embodiment of the present invention.

[0030] Figure 7 Channel and spatial attention mechanism structure diagram of a method for identifying individual marine mammals provided by an embodiment of the present invention

[0031] Figure 8 CBAM module diagram of a method for identifying individual marine mammals provided by an embodiment of the present invention

[0032] Fig. 9 An improved SPPF structure diagram of a method for identifying individual marine mammals provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] The present invention provides a method for identifying individual marine mammals, comprising:

[0035] Step 1: Data collection, obtain image data of marine mammals; use shooting equipment (such as portable mobile phone terminals) to unify the shooting position (such as aiming at the face), angle, spacing (such as 2 meters away from the individual) and duration (such as shooting for 10 minutes) and other operating standards to shoot videos of marine mammals. Then convert the captured video frame by frame into static images, and select photos that clearly show the complete facial features of marine mammals. During the screening process, exclude images with blurred images, incomplete faces, and poor shooting angles (such as looking down, looking up, or front rotation angles exceeding 45°). The number of valid images of each individual marine mammal should be no less than 100 (including all angles).

[0036] Step 2: Data enhancement: using data enhancement technology, randomly enhance the original image data; using data enhancement technologies such as rotation, translation, mirror inversion, scaling and adding noise, randomly enhance the original image at a ratio of 1:4. These operations not only enrich the diversity of training data, but also simulate various changes that marine mammals may encounter in real environments, such as different lighting conditions, facial postures, etc. Through data enhancement, the model's application ability in real scenes is improved, so that it can maintain high recognition accuracy and stability when facing complex changes in the real environment.

[0037] Step 3: Model construction, based on convolutional neural networks and facial features, a marine mammal individual recognition model is constructed. The model is based on the Darknet network model structure, integrating attention mechanism and multi-scale feature processing technology to achieve facial recognition of different marine mammal individuals.

[0038] Furthermore, in step 3, a downsampling convolution layer is used to perform feature extraction and dimensionality reduction on the enhanced image data, and the downsampling convolution layer uses a convolution kernel of 3×3 size and step size of 2; in the feature enhancement layer (P-AC), a feature extraction (SlimC2F) module is used to connect multiple convolution layers to extract and fuse the enhanced image data, and at the same time, the attention mechanism is combined to enhance the model's recognition ability for key facial features of marine mammals; the feature information of image data of different scales is integrated through a multi-scale feature fusion layer (SPPF); each channel of the image data is averaged and pooled through a global average pooling layer (GAP) to generate a final feature vector; finally, the feature vector is sent to a fully connected layer (FC) for classification and recognition of individual marine mammals.

[0039] In the present application, preferably, in the feature enhancement layer (P-AC) of step 3, the feature enhancement layer (P-AC) based on deep learning first uses a lightweight and improved C2F structure (CSP Bottleneck with 2ConvolutionsFast) to extract and fuse the features of the facial images of marine mammals, and then introduces the CBAM module (ConvolutionalBlock Attention Module) to further enhance the recognition of key feature areas of the faces of marine mammals.

[0040] In the present application, preferably, in the feature extraction (SlimC2F) module of step 3, the SlimC2F structure divides the input feature map into two parts based on the number of channels for processing, one part is directly transmitted, and the other part is subjected to residual processing and then merged with the directly transmitted part in the channel dimension; in the residual processing, the residual block adopts 3×3 depthwise separable convolution, and the depthwise separable convolution splits the standard convolution into two steps: depthwise convolution and pointwise convolution. This processing method accelerates the transmission of information and enhances the feature extraction capability of the model.

[0041] In the present application, preferably, in step 3, the attention mechanism introduces a convolutional block attention module CBAM, and the CBAM module enhances the response to key channel features and highlights important areas in the input image data by applying an attention mechanism to the input image data in the channel and spatial dimensions respectively.

[0042] In the present application, preferably, in step 3, the multi-scale feature fusion layer SPPF (Spatial Pyramid Pooling-Fast) captures multi-scale features in the image through pooling windows of different sizes, thereby enhancing the robustness of the model and improving the model's ability to capture multi-scale features of the image.

[0043] The following combination Figure 1-9 Further introduce the embodiments of the present application:

[0044] Example 1

[0045] A method for identifying an individual harbor seal comprises the following steps:

[0046] Step 1: Data collection, the spotted seal face dataset comes from Qingdao Haichang Polar Ocean World. By using a portable mobile phone terminal, the shooting site, angle, and duration were unified, and multiple spotted seals were filmed at multiple time periods. In order to ensure the quality of the dataset, the captured videos were converted into static images frame by frame, and photos that clearly showed the complete facial features of the spotted seals were selected. During the screening process, images with blurred images, incomplete faces, and poor shooting angles (such as looking down, looking up, or front rotation angles exceeding 45°) were excluded. Finally, a total of 3914 valid image data of 14 spotted seals were obtained, with 100 to 300 facial images of each spotted seal. Figure 2 A partial image of a harbor seal's face is shown.

[0047] Step 2: Data enhancement. In order to improve the generalization and robustness of model recognition, this study uses data enhancement techniques such as rotation, translation, mirror inversion, scaling, and adding noise to perform random enhancement processing on the original image at a ratio of 1:4. These operations not only enrich the diversity of training data, but also simulate the various changes that spotted seals may encounter in real environments, such as different lighting conditions, facial postures, etc. Through data enhancement, the model's ability to be applied in real-world scenarios is improved, so that it can maintain high recognition accuracy and stability when facing complex changes in the real environment. Some data enhancement effects are as follows: Figure 3 shown.

[0048] Step 3: Model construction, based on the Convolutional Neural Network (CNN) spotted seal facial recognition model, based on the Darknet network model structure, integrating the attention mechanism and multi-scale feature processing technology to achieve facial recognition of different spotted seal individuals. The network structure is as follows: Figure 4 As shown. The downsampling convolution layer (Conv3×3_Downsampling) uses a convolution kernel of 3×3 size and stride 2 for feature extraction and dimensionality reduction. The feature enhancement layer (P-AC) connects multiple convolution layers through the SlimC2F module to extract and fuse features, and combines the attention mechanism to enhance the model's ability to recognize key facial features. The multi-scale feature fusion layer (SPPF) enhances the model's adaptability to scale changes by integrating feature information of different scales. The global average pooling layer (GAP) performs average pooling on each channel of the feature map to generate the final feature vector. Finally, the feature vector is sent to the fully connected layer (FC) for inference and outputs the corresponding category. The network structure is visualized as shown below. Figure 5As shown. The P-AC (Post-Attention CSP) feature enhancement layer based on deep learning first uses the lightweight improved C2F structure (CSP Bottleneck with 2Convolutions Fast) to extract and fuse the features of the spotted seal facial image, and then introduces the CBAM module (Convolutional Block Attention Module) to further enhance the recognition of the key feature areas of the spotted seal face. The SlimC2F structure divides the input feature map into two parts based on the number of channels for processing. One part is directly transmitted, and the other part is residually learned and then merged with the directly transmitted part in the channel dimension. This processing method accelerates the transmission of information and enhances the feature extraction ability of the model. In addition, a 3×3 depthwise separable convolution is used in the residual block. The depthwise separable convolution splits the standard convolution into two steps: depthwise convolution (convolution for each input channel) and pointwise convolution (1×1 convolution, used to combine features from depthwise convolution), as shown in Figure 6 As shown in Figure 1, after the input feature map of size W×H×3 undergoes depthwise separable convolution, the output feature map of size W×H×4 is obtained. Figure 4 For example, the SlimC2F structure in the example above shows that the number of parameters for the standard convolution from the 384 channels of the input feature map to the 192 channels of the output feature map is (3×3×384)×192=663552. The number of parameters required for the depthwise separable convolution is (3×3×1)×384+(1×1×128)×192=28032, which greatly reduces the number of parameters and computational requirements of the model. In the case of spotted seal facial recognition, accurately locating local information with recognizable features is crucial to improving model performance. For this purpose, the convolutional block attention module CBAM is introduced, such as Figure 7 As shown in Figure 2, the CBAM module not only enhances the response to key channel features but also further highlights the important areas in the input feature map by applying attention mechanisms to the input feature map in the channel and spatial dimensions respectively. Figure 8 As shown, given the input feature map F∈R W×H×C , will be sequentially combined with the channel attention weight Mc∈R generated by the CBAM module 1×1×C , spatial attention weight Mc∈R W×H×1 Perform element-by-element multiplication. Introduce the multi-scale feature fusion layer SPPF (Spatial Pyramid Pooling-Fast), which captures multi-scale features in the image through pooling windows of different sizes, thereby enhancing the robustness of the model. Fig. 9As shown in the figure, in order to better match the multi-scale information of the feature map in the spotted seal face recognition task, for the 7×7 size feature map output by the final P-AC module, this study adjusts the pooling kernel size in the traditional SPPF structure from 5×5 to 3×3, and connects three pooling layers with a step size of 1 in series to achieve a step-by-step growth receptive field of 1×1, 3×3, 5×5, and 7×7. Through this method, the model can not only capture local detail features more effectively (1×1, 3×3 receptive fields), but also integrate medium-range contextual information (5×5 receptive field) and understand the global layout of the feature map (7×7 receptive field), which improves the model's ability to capture multi-scale features of the image.

[0049] Step 4: Model evaluation. In order to evaluate the effectiveness and performance of the constructed spotted seal individual recognition model, this embodiment designs and conducts multiple groups of experiments. The preliminary experiment adopts the ablation research method to evaluate the contribution of each component to the overall performance of the model by removing or replacing the key components in the model, so as to determine the optimal configuration of the model. Subsequently, the optimal configuration model is compared with the mainstream model in the current field to explore its performance characteristics and effectiveness. Next, the experiment will focus on the impact of background information on the recognition effect and explore the performance changes of the model under different background conditions. Finally, by systematically reducing the size of the training data set, the impact of the amount of data on the performance of the model is explored to verify the applicability of the model in data-limited scenarios. After a series of experimental verifications, the recognition accuracy of the spotted seal face recognition model reached 98.35%, and its precision, recall and F1 scores reached 98.30%, 98.31%, and 98.31%, showing the practical application potential of the model in related fields.

[0050] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying individual marine mammals, characterized in that: include: Step 1: Data collection, obtaining image data of marine mammals; Step 2: Data enhancement: using data enhancement technology to randomly enhance the original image data; Step 3: Model construction, based on convolutional neural networks and facial features, a marine mammal individual recognition model is constructed. The model is based on the Darknet network model structure, integrating attention mechanism and multi-scale feature processing technology to achieve facial recognition of different marine mammal individuals.

2. A method for identifying individual marine mammals according to claim 1, characterized in that: In step 3, a downsampling convolution layer is used to perform feature extraction and dimension reduction on the enhanced image data, and the downsampling convolution layer uses a convolution kernel of 3×3 size and a step size of 2; In the feature enhancement layer (P-AC), the feature extraction (SlimC2F) module is used to connect multiple convolutional layers to extract and fuse the enhanced image data. At the same time, the attention mechanism is combined to enhance the model's ability to recognize key facial features of marine mammals. The feature information of image data at different scales is integrated through the multi-scale feature fusion layer (SPPF). The global average pooling layer (GAP) is used to perform average pooling on each channel of the image data to generate the final feature vector; finally, the feature vector is sent to the fully connected layer (FC) for classification and identification of individual marine mammals.

3. A method for identifying individual marine mammals according to claim 2, characterized in that: In the feature enhancement layer (P-AC) of step 3, the deep learning-based feature enhancement layer (P-AC) first uses the lightweight improved C2F structure (CSP Bottleneck with 2Convolutions Fast) to extract and fuse the features of marine mammal facial images, and then introduces the CBAM module (Convolutional Block Attention Module) to further enhance the recognition of key feature areas of marine mammal faces.

4. A method for identifying individual marine mammals according to claim 2, characterized in that: In the feature extraction (SlimC2F) module of step 3, the SlimC2F structure divides the input feature map into two parts based on the number of channels for processing. One part is directly transmitted, and the other part is subjected to residual processing and then merged with the directly transmitted part in the channel dimension. In the residual processing, the residual block adopts 3×3 depthwise separable convolution (Depthwise SeparableConvolution), and the depthwise separable convolution splits the standard convolution into two steps: depthwise convolution and point-by-point convolution.

5. A method for identifying individual marine mammals according to claim 2, characterized in that: In step 3, the attention mechanism introduces a convolutional block attention module (CBAM). The CBAM module enhances the response to key channel features and highlights important areas in the input image data by applying an attention mechanism to the input image data in the channel and spatial dimensions respectively.

6. A method for identifying individual marine mammals according to claim 2, characterized in that: In step 3, the multi-scale feature fusion layer SPPF (Spatial Pyramid Pooling-Fast) captures multi-scale features in the image through pooling windows of different sizes, thereby enhancing the robustness of the model.

7. A method for identifying individual marine mammals according to claim 6, characterized in that: The model can more effectively capture local detail features of 1×1 and 3×3 receptive fields, fuse medium-range contextual information of 5×5 receptive fields, and understand the global layout of 7×7 receptive fields in image data.

8. A method for identifying individual marine mammals according to claim 1, characterized in that: Also includes: Step 4: Model evaluation: Use accuracy, precision, recall, and F1-score (the harmonic mean of precision and recall) to evaluate the performance of the individual recognition model.

9. An electronic device, characterized in that: include: processor; A memory for storing computer programs; wherein the processor is used to execute the computer programs stored in the memory to implement the method for identifying individual marine mammals as described in claim 1.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method for identifying individual marine mammals according to claim 1 is implemented.

Images