A method for identifying individual marine mammals

Abstract

The present invention provides a method for identifying individual marine mammals, which belongs to the field of computer image recognition. The method is based on a convolutional neural network model, combined with an attention mechanism, multi-scale feature processing, and depth-separable convolution technology, and includes four steps: data acquisition, data enhancement, model construction, and model evaluation. Through the application of this method, it is possible to accurately capture the key facial features of marine mammals, reduce the number of parameters and computing requirements of model operations, and achieve efficient and accurate identification of individual marine mammals in the context of a smaller data set, thereby solving the challenges of high cost, low efficiency, and unfriendly marine mammal individual population identification in traditional methods. Therefore, the marine mammal individual identification method of the present application has good marine mammal individual identification capabilities and can be widely used in the fields of marine species survey, monitoring, and protection management.

Description

Technical Field

[0001] The present invention relates to the technical field of computer image recognition, and in particular to a method for identifying individual marine mammals. Background Art

[0002] Identifying and registering rare and endangered marine mammals is crucial for understanding their populations, monitoring population dynamics, and implementing conservation management. Generally, choosing an identification method requires a balance between cost, timeliness, and accuracy. Currently, the main methods for identifying marine mammals include manual surveys, satellite beacon tracking, and DNA testing.

[0003] The inventors of the present application have discovered in their research that the cost of conducting manual surveys of marine mammal populations using icebreakers and airplanes is high, the survey time and route are restricted by geographical conditions and prior knowledge, and the accuracy of individual identification is affected by subjective experience. Satellite beacon tracking surveys can obtain the continuous activity status of marine mammal individuals within a certain period of time, but most beacons fall off from January to March, and the timing of releasing harbor seals in the wild is accidental, making them less practical in actual large-scale monitoring. Although DNA detection based on blood samples and the environment can be used for individual identification of marine mammals, this method has a high individual identification rate, but it is expensive and has a low encounter rate, making it difficult to be widely used in individual identification and monitoring of wild populations. In summary, although many attempts have been made in the past to investigate and monitor marine mammal populations (individuals), these methods have failed to achieve effective identification of individuals.

[0004] In recent years, facial recognition based on facial features has provided new insights into wildlife individual identification. In the field of computer image recognition, advances in deep learning technology have provided strong support for computer image feature learning. Therefore, this patent proposes an efficient, convenient, low-cost, and operational method for individual marine mammal identification based on image data and a convolutional neural network model. Summary of the Invention

[0005] The embodiment of the present application provides a method for identifying individual marine mammals, which aims to accurately identify individual marine mammals and solve the challenges of high cost, low efficiency and unfriendly problems in traditional methods, thereby providing scientific and technological support for the refined protection and management of marine mammals.

[0006] In order to solve the above technical problems, the technical solution proposed in this application is:

[0007] The present invention provides a method for identifying individual marine mammals, comprising:

[0008] Step 1: Data collection, obtaining image data of marine mammals;

[0009] Step 2: Data enhancement, using data enhancement technology to perform random enhancement processing on the original image data;

[0010] Step 3: Model construction: Based on convolutional neural networks and facial features, a marine mammal individual recognition model is constructed. The model is based on the Darknet network model structure, integrates the attention mechanism and multi-scale feature processing technology, and realizes facial recognition of different marine mammal individuals.

[0011] Furthermore, in step 3, a downsampling convolution layer is used to perform feature extraction and dimensionality reduction on the enhanced image data. The downsampling convolution layer adopts a convolution kernel of 3×3 size and stride 2; in the feature enhancement layer (P-AC), a feature extraction (SlimC2F) module is used to connect multiple convolution layers to extract and fuse the enhanced image data, and at the same time, the attention mechanism is combined to enhance the model's recognition ability for key facial features of marine mammals; the feature information of image data of different scales is integrated through the multi-scale feature fusion layer (SPPF); each channel of the image data is average pooled through the global average pooling layer (GAP) to generate the final feature vector; finally, the feature vector is sent to the fully connected layer (FC) for classification and identification of individual marine mammals.

[0012] Furthermore, in the feature enhancement layer (P-AC) of step 3, the deep learning-based feature enhancement layer (P-AC) first uses the lightweight improved C2F structure (CSP Bottleneck with 2Convolutions Fast) to extract and fuse the features of marine mammal facial images, and then introduces the CBAM module (Convolutional Block Attention Module) to further enhance the recognition of key feature areas of marine mammal faces.

[0013] Furthermore, in the feature extraction (SlimC2F) module of step 3, the SlimC2F structure divides the input feature map into two parts based on the number of channels for processing. One part is directly passed, and the other part is subjected to residual learning processing and then merged with the directly passed part in the channel dimension. In the residual learning processing, the residual block adopts 3×3 depthwise separable convolution. The depthwise separable convolution splits the standard convolution into two steps: depthwise convolution and point-by-point convolution.

[0014] Furthermore, in step 3, the attention mechanism introduces a convolutional block attention module (CBAM). The CBAM module enhances the response to key channel features and highlights important areas in the input image data by applying an attention mechanism to the input image data in the channel and spatial dimensions respectively.

[0015] Furthermore, in step 3, the multi-scale feature fusion layer SPPF (Spatial Pyramid Pooling-Fast) captures multi-scale features in the image through pooling windows of different sizes, thereby enhancing the robustness of the model.

[0016] Furthermore, the model can more effectively capture local detail features of 1×1 and 3×3 receptive fields, fuse medium-range contextual information of 5×5 receptive fields, and understand the global layout of 7×7 receptive fields in image data.

[0017] Furthermore, it also includes:

[0018] Step 4: Model evaluation: Use accuracy, precision, recall, and F1 score (the harmonic mean of precision and recall) to evaluate the performance of the individual recognition model.

[0019] On the other hand, the present application also provides an electronic device, comprising: a processor; a memory for storing a computer program; wherein the processor is used to execute the computer program stored in the memory to implement the aforementioned method for identifying individual marine mammals.

[0020] On the other hand, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the aforementioned method for identifying individual marine mammals.

[0021] Compared with the existing technology, the method for identifying individual marine mammals of the present invention has achieved the following beneficial technical effects:

[0022] This method can accurately capture the key facial features of marine mammals, reduce the number of parameters and computing requirements of convolutional model operations, achieve efficient and accurate identification of marine mammal individuals in the context of smaller data sets, and solve the challenges of marine mammal individual population identification such as high cost, low efficiency and unfriendlyness in traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0024] Figure 1 This is a flow chart of a method for identifying individual marine mammals provided by an embodiment of the present invention.

[0025] Figure 2 A schematic diagram of facial images of a method for identifying individual marine mammals provided by an embodiment of the present invention.

[0026] Figure 3 A schematic diagram of image enhancement for a method for identifying individual marine mammals provided by an embodiment of the present invention.

[0027] Figure 4 A diagram of the improved DarkNet network structure for a method for identifying individual marine mammals provided in an embodiment of the present invention.

[0028] Figure 5 A network structure visualization diagram of a method for identifying individual marine mammals provided by an embodiment of the present invention.

[0029] Figure 6 A diagram of the depthwise separable convolutional structure of a method for identifying individual marine mammals provided by an embodiment of the present invention.

[0030] Figure 7 A channel and spatial attention mechanism structure diagram of a marine mammal individual recognition method provided by an embodiment of the present invention

[0031] Figure 8 CBAM module diagram of a method for identifying individual marine mammals provided by an embodiment of the present invention

[0032] Figure 9 This is a diagram of the improved SPPF structure of a method for identifying individual marine mammals provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0034] The present invention provides a method for identifying individual marine mammals, comprising:

[0035] Step 1: Data collection, obtain image data of marine mammals; use shooting equipment (such as portable mobile phone terminals) to unify the shooting position (such as aiming at the face), angle, spacing (such as 2 meters away from the individual) and duration (such as shooting for 10 minutes) and other operating standards to shoot videos of marine mammals. The shot video is then converted into a static image frame by frame, and photos that clearly show the complete facial features of marine mammals are selected. During the screening process, images with blurred images, incomplete faces, and poor shooting angles (such as looking down, looking up, or front rotation angles exceeding 45°) are excluded. The number of valid images of each individual marine mammal should be no less than 100 (including all angles).

[0036] Step 2: Data augmentation. Data augmentation techniques are used to randomly enhance the original image data. These techniques, including rotation, translation, mirror inversion, scaling, and noise addition, are used to randomly enhance the original images at a ratio of 1:4. These operations not only enrich the diversity of the training data but also simulate the various variations that marine mammals may encounter in real-world environments, such as varying lighting conditions and facial poses. This data augmentation improves the model's ability to be applied in real-world scenarios, enabling it to maintain high recognition accuracy and stability despite complex environmental variations.

[0037] Step 3: Model construction: Based on convolutional neural networks and facial features, a marine mammal individual recognition model is constructed. The model is based on the Darknet network model structure, integrates the attention mechanism and multi-scale feature processing technology, and realizes facial recognition of different marine mammal individuals.

[0038] Furthermore, in step 3, a downsampling convolution layer is used to perform feature extraction and dimensionality reduction on the enhanced image data. The downsampling convolution layer adopts a convolution kernel of 3×3 size and stride 2; in the feature enhancement layer (P-AC), a feature extraction (SlimC2F) module is used to connect multiple convolution layers to extract and fuse the enhanced image data, and at the same time, the attention mechanism is combined to enhance the model's recognition ability for key facial features of marine mammals; the feature information of image data of different scales is integrated through the multi-scale feature fusion layer (SPPF); each channel of the image data is average pooled through the global average pooling layer (GAP) to generate the final feature vector; finally, the feature vector is sent to the fully connected layer (FC) for classification and identification of individual marine mammals.

[0039] In the present application, preferably, in the feature enhancement layer (P-AC) of step 3, the feature enhancement layer (P-AC) based on deep learning first uses the lightweight improved C2F structure (CSP Bottleneck with 2ConvolutionsFast) to extract and fuse the features of the facial images of marine mammals, and then introduces the CBAM module (ConvolutionalBlock Attention Module) to further enhance the recognition of key feature areas of the faces of marine mammals.

[0040] In the present application, preferably, in the feature extraction (SlimC2F) module of step 3, the SlimC2F structure divides the input feature map into two parts based on the number of channels for processing, one part is directly transmitted, and the other part is subjected to residual learning processing and then merged with the directly transmitted part in the channel dimension; in the residual learning processing, the residual block adopts 3×3 depthwise separable convolution (Depthwise Separable Convolution), and the depthwise separable convolution splits the standard convolution into two steps: depthwise convolution and point-by-point convolution. This processing method accelerates the transmission of information and enhances the feature extraction capability of the model.

[0041] In the present application, preferably, in step 3, the attention mechanism introduces a convolutional block attention module CBAM, which enhances the response to key channel features and highlights important areas in the input image data by applying an attention mechanism to the input image data in the channel and spatial dimensions respectively.

[0042] In the present application, preferably, in step 3, the multi-scale feature fusion layer SPPF (Spatial Pyramid Pooling-Fast) captures multi-scale features in the image through pooling windows of different sizes, thereby enhancing the robustness of the model and improving the model's ability to capture multi-scale features of the image.

[0043] The following combination Figure 1-9 Further introduce the embodiments of this application:

[0044] Example 1

[0045] A method for identifying an individual harbor seal comprises the following steps:

[0046] Step 1: Data collection. The spotted seal facial dataset comes from Qingdao Haichang Polar Ocean World. Using a portable mobile phone terminal, the shooting location, angle, and duration were standardized, and videos of multiple spotted seals were taken at multiple time periods. To ensure the quality of the dataset, the captured videos were converted frame by frame into static images, and photos that clearly showed the complete facial features of the spotted seals were selected. During the screening process, images with blurred images, incomplete faces, and poor shooting angles (such as looking down, looking up, or frontal rotation angles exceeding 45°) were excluded. Ultimately, a total of 3,914 valid image data of 14 spotted seals were obtained, with 100 to 300 facial images of each spotted seal. Figure 2 A partial image of a harbor seal's face is shown.

[0047] Step 2: Data enhancement. In order to improve the generalization and robustness of model recognition, this study uses data enhancement techniques such as rotation, translation, mirror inversion, scaling, and adding noise to perform random enhancement processing on the original image at a ratio of 1:4. These operations not only enrich the diversity of training data, but also simulate various changes that spotted seals may encounter in real environments, such as different lighting conditions, facial postures, etc. Through data enhancement, the model's ability to be applied in real-world scenarios is improved, enabling it to maintain high recognition accuracy and stability when facing complex changes in the real environment. Some data enhancement effects are as follows: Figure 3 shown.

[0048] Step 3: Model construction, based on the Convolutional Neural Network (CNN) spotted seal face recognition model, based on the Darknet network model structure, integrating the attention mechanism and multi-scale feature processing technology to achieve facial recognition of different spotted seal individuals. The network structure is as follows: Figure 4 As shown. The downsampling convolution layer (Conv3×3_Downsampling) uses a convolution kernel of size 3×3 and a stride of 2 for feature extraction and dimensionality reduction. The feature enhancement layer (P-AC) connects multiple convolution layers through the SlimC2F module to extract and fuse features, and at the same time combines the attention mechanism to enhance the model's ability to recognize key facial features. The multi-scale feature fusion layer (SPPF) enhances the model's adaptability to scale changes by integrating feature information of different scales. The global average pooling layer (GAP) performs average pooling on each channel of the feature map to generate the final feature vector. Finally, the feature vector is sent to the fully connected layer (FC) for inference and outputs the corresponding category. The network structure is visualized as shown below. Figure 5As shown. The P-AC (Post-Attention CSP) feature enhancement layer based on deep learning first uses the lightweight improved C2F structure (CSP Bottleneck with 2Convolutions Fast) to extract and fuse the features of the spotted seal facial image, and then introduces the CBAM module (Convolutional Block Attention Module) to further enhance the recognition of the key feature areas of the spotted seal face. The SlimC2F structure divides the input feature map into two parts based on the number of channels for processing. One part is directly transmitted, and the other part is merged with the directly transmitted part in the channel dimension after residual learning. This processing method accelerates the transmission of information and enhances the feature extraction ability of the model. In addition, a 3×3 depthwise separable convolution is used in the residual block. Depthwise separable convolution splits the standard convolution into two steps: depthwise convolution (convolution for each input channel) and point-by-point convolution (1×1 convolution, used to combine features from depthwise convolution), as shown. Figure 6 As shown in , the input feature map of size W×H×3 is subjected to depthwise separable convolution, and the output feature map of size W×H×4 is obtained. Figure 4 Taking the SlimC2F structure in

[15] as an example, the number of parameters required for standard convolution from the 384 channels of the input feature map to the 192 channels of the output feature map is (3×3×384)×192=663552. The number of parameters required for depthwise separable convolution is (3×3×1)×384+(1×1×128)×192=28032, which greatly reduces the number of parameters and computational requirements of the model. In the case of spotted seal facial recognition, accurately locating local information with recognizable features is crucial to improving model performance. For this purpose, the convolutional block attention module (CBAM) is introduced, such as Figure 7 As shown in , the CBAM module not only enhances the response to key channel features but also further highlights the important areas in the input feature map by applying attention mechanisms to the input feature map in the channel and spatial dimensions respectively. Figure 8 As shown, given the input feature map F∈R W×H×C , will be sequentially combined with the channel attention weight Mc∈R generated by the CBAM module 1×1×C , spatial attention weight Mc∈R W×H×1 Perform element-by-element multiplication. Introduce the multi-scale feature fusion layer SPPF (Spatial Pyramid Pooling-Fast) to capture multi-scale features in the image through pooling windows of different sizes, thereby enhancing the robustness of the model. Figure 9As shown in the figure, to better match the multi-scale information of the feature map in the spotted seal facial recognition task, this study adjusts the pooling kernel size in the traditional SPPF structure from 5×5 to 3×3 for the 7×7 feature map output by the final P-AC module, and connects three pooling layers with a stride of 1 to achieve a step-by-step increase in receptive fields of 1×1, 3×3, 5×5, and 7×7. This method not only allows the model to more effectively capture local detail features (1×1 and 3×3 receptive fields), but also integrates medium-range contextual information (5×5 receptive field) and understands the global layout of the feature map (7×7 receptive field), improving the model's ability to capture multi-scale image features.

[0049] Step 4: Model Evaluation. To evaluate the effectiveness and performance of the constructed spotted seal individual recognition model, this example designed and conducted multiple experiments. Initial experiments employed an ablation study method, removing or replacing key components within the model to assess their contribution to the overall model performance and determine the optimal model configuration. Subsequently, the optimal configuration model was compared with leading models in the field to explore its performance characteristics and effectiveness. Next, the experiment focused on the impact of background information on recognition results, exploring how the model's performance changes under different background conditions. Finally, by systematically reducing the size of the training dataset, the impact of data volume on model performance was explored to verify the model's applicability in data-limited scenarios. After a series of experimental verifications, the spotted seal facial recognition model achieved a recognition accuracy of 98.35%, with precision, recall, and F1 scores reaching 98.30%, 98.31%, and 98.31%, respectively, demonstrating the model's potential for practical application in related fields.

[0050] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for identifying individual marine mammals, characterized in that: include: Step 1: Data collection, obtaining image data of marine mammals; Step 2: Data enhancement, using data enhancement technology to perform random enhancement processing on the original image data; Step 3: Model construction: Based on convolutional neural networks and facial features, a marine mammal individual recognition model is constructed. The model is based on the Darknet network model structure, integrates the attention mechanism and multi-scale feature processing technology, and realizes facial recognition of different marine mammal individuals; In step 3, a downsampling convolution layer is used to perform feature extraction and dimensionality reduction on the enhanced image data. The downsampling convolution layer uses a convolution kernel of 3×3 size and a stride of 2; In the feature enhancement layer P-AC, the feature extraction module SlimC2F is used to connect multiple convolutional layers to extract and fuse the enhanced image data. At the same time, the attention mechanism is combined to enhance the model's ability to recognize key facial features of marine mammals. The multi-scale feature fusion layer SPPF integrates feature information of image data at different scales. The global average pooling layer (GAP) performs average pooling on each channel of the image data to generate the final feature vector. Finally, the feature vector is sent to the fully connected layer (FC) for classification and identification of individual marine mammals. In the feature extraction module SlimC2F of step 3, the SlimC2F structure divides the input feature map into two parts based on the number of channels for processing. One part is directly passed, and the other part is subjected to residual learning processing and then merged with the directly passed part in the channel dimension. In the residual learning processing, the residual block uses a 3×3 depth-wise separable convolution, and the depth-wise separable convolution splits the standard convolution into two steps: depth-wise convolution and point-by-point convolution.

2. A method for identifying individual marine mammals according to claim 1, characterized in that: In step 3, the feature enhancement layer P-AC based on deep learning first uses the lightweight improved C2F structure to extract and fuse the features of marine mammal facial images, and then introduces the CBAM module to further enhance the recognition of key feature areas of marine mammal faces.

3. A method for identifying individual marine mammals according to claim 1, characterized in that: In step 3, the attention mechanism introduces a convolutional block attention module (CBAM). The CBAM module enhances the response to key channel features and highlights important areas in the input image data by applying an attention mechanism to the input image data in the channel and spatial dimensions respectively.

4. A method for identifying individual marine mammals according to claim 1, characterized in that: In step 3, the multi-scale feature fusion layer SPPF captures multi-scale features in the image through pooling windows of different sizes, thereby enhancing the robustness of the model.

5. A method for identifying individual marine mammals according to claim 4, characterized in that: The model can more effectively capture local detail features of 1×1 and 3×3 receptive fields, fuse medium-range contextual information of 5×5 receptive fields, and understand the global layout of 7×7 receptive fields in image data.

6. A method for identifying individual marine mammals according to claim 1, characterized in that: Also includes: Step 4: Model evaluation, use accuracy, precision, recall and F1 score to evaluate the performance of the individual recognition model.

7. An electronic device, characterized in that: include: processor; A memory for storing a computer program; wherein the processor is used to execute the computer program stored in the memory to implement the method for identifying individual marine mammals according to claim 1.

8. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by the processor, the method for identifying individual marine mammals according to claim 1 is implemented.

Images