A marine mammal call classification method based on IVGG-ASNet

Abstract

The present invention belongs to the technical field of marine mammal call recognition and classification, and specifically relates to a marine mammal call classification method based on IVGG-ASNet. The method converts the acquired marine mammal call data into Mel spectrograms as feature representations to form a data set; introduces enhanced convolution blocks to effectively capture the time and frequency characteristics of marine mammal call data by using multi-scale convolution kernels; introduces strip mixing convolution blocks in stages 4 and 5 of each convolution layer, respectively, to effectively enhance the model's perception of long-distance dependencies and characteristics of different frequency bands; designs and adds an automatic search module to enable the model to automatically learn and combine the optimal feature structures during training. Experimental results show that the method provided by the present invention effectively improves the accuracy of marine mammal call classification.

Description

Technical Field

[0001] The present invention belongs to the technical field of marine mammal call recognition and classification, and particularly relates to a marine mammal call classification method based on IVGG-ASNet. Background Art

[0002] In recent years, marine mammal call recognition and classification technology has played a significant role in protecting marine biodiversity. Marine mammal calls contain a wealth of information about species, individuals, and behavior. Identifying and classifying them helps assess population size, distribution range, migration routes, and reproductive behavior, providing a scientific basis for developing effective conservation strategies. Marine mammals are sensitive to environmental changes, and changes in their call characteristics can reflect environmental pressures such as marine noise pollution and climate change, providing a new means for marine environmental monitoring. However, traditional marine mammal call recognition and classification methods rely primarily on expert manual identification and feature engineering-based machine learning methods, which have the following drawbacks: manual identification is time-consuming and labor-intensive in the face of massive amounts of acoustic data; traditional machine learning methods require manual feature design, and the feature extraction process is cumbersome and relies on expert experience, making it difficult to capture subtle differences in calls. Therefore, traditional classification methods struggle to effectively identify and distinguish the calls of different marine mammals. Summary of the Invention

[0003] To address the above technical issues, the present invention provides a marine mammal call classification method based on IVGG-ASNet. This method designs and integrates an automatic search module to improve classification accuracy. The enhanced convolution block enhances the network's modeling capabilities for time and frequency domain features through parallel convolution paths. The strip-mixed convolution block strengthens the ability to capture multi-band, long-range dependent speech features. The design and integration of the automatic search module enables the network structure to have dynamic adjustment and adaptive combination capabilities, automatically selecting the optimal convolution structure combination scheme based on different input data. This method can significantly improve the accuracy of marine mammal call classification and promote the conservation of marine biodiversity.

[0004] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions:

[0005] A marine mammal call classification method based on IVGG-ASNet, the method comprising the following steps:

[0006] S1: Convert the acquired marine mammal call data into Mel spectrograms as feature representation to form a dataset;

[0007] S2: Using the dataset to construct and train an IVGG-ASNet classification model, the IVGG-ASNet classification model includes an enhanced convolution block, a stripe mixing convolution block, and an automatic search module;

[0008] S3: Use the trained IVGG-ASNet classification model to classify the calls of marine mammals to be classified.

[0009] Furthermore, in step S1, marine mammal call data is obtained from the Watkins marine mammal sound database, and the obtained marine mammal call data is preprocessed by clipping, denoising, and resampling, and the preprocessed marine mammal call data is converted into a Mel spectrogram as a feature representation.

[0010] Furthermore, in step S2, the enhanced convolution block uses a multi-scale convolution kernel to capture the temporal and frequency characteristics of the marine mammal call data.

[0011] Furthermore, the enhanced convolution block consists of three parallel convolutions, namely a 1×3 horizontal kernel, a 3×1 vertical kernel, and a 3×3 standard convolution kernel, and the outputs of the enhanced convolution block are summed to enrich the feature space;

[0012] Assume that the input feature map of the enhanced convolution block is X, with a size of H×W×D, where H is the height, W is the width, and D is the number of channels;

[0013] The expressions of the convolution operation and output fusion of the enhanced convolution block are as follows:

[0014] F 1×3 = Conv 1×3 (X) ;

[0015] F 3×1 = Conv 3×1 (X);

[0016] F 3×3 = Conv 3×3 (X) ;

[0017] F ECB = F 1×3 + F 3×1 + F 3×3 ;

[0018] Among them, F 1×3 It represents the feature map obtained by convolution operation on the input feature map X using 1×3 convolution kernel, F 3×1 It represents the feature map obtained by convolution operation on the input feature map X using 3×1 convolution kernel, F 3×3 It represents the feature map obtained by convolution operation on the input feature map X using 3×3 convolution kernel, F ECB is the final output feature map of the enhanced convolution block.

[0019] Furthermore, in step S2, the strip mixed convolution block includes a strip convolution block and a pyramid convolution block; the strip mixed convolution block fuses the output features of the strip convolution block and the pyramid convolution block, and outputs a final feature map.

[0020] Furthermore, the strip hybrid convolution block fuses the output features of the strip convolution block and the pyramid convolution block to output a final feature map, specifically including:

[0021] (1) Design of strip convolution block: The strip convolution block consists of horizontal and vertical convolution kernels;

[0022] The horizontal convolution path uses a convolution kernel size of 1×n to perform a convolution operation on the input feature map along the time axis. The output feature map of the horizontal convolution path is recorded as:

[0023] F h = Conv 1×n (X1) ;

[0024] Among them, X1 is the input feature map, the size is H×W×C, H is the height, W is the width, and C is the number of channels of the input feature map; F h is the horizontal feature map output by the horizontal convolution path, with a size of H×(W-n+1)×C′; C′ is the number of channels of the output feature map;

[0025] The vertical convolution path uses a convolution kernel size of n×1 to perform a convolution operation on the input feature map along the frequency axis. The output feature map of the vertical convolution path is recorded as:

[0026] F v = Conv 1×n (X1) ;

[0027] Among them, F v is the vertical feature map output by the vertical convolution path, with a size of (H-n+1)×W×C′;

[0028] (2) Design of pyramid convolution block: The pyramid convolution block adopts convolution kernels of three sizes: small scale, medium scale and large scale;

[0029] Among them, the convolution kernel size of the small-scale convolution path is 1×1×1; the convolution kernel size of the medium-scale convolution path is 3×3×3; the convolution kernel size of the large-scale convolution path is 5×5×5;

[0030] The three parallel paths of small-scale convolution path, medium-scale convolution path and large-scale convolution path are used to perform convolution operation on the input feature map X1 respectively. The expression is:

[0031] F p1=Conv1×1(X1);

[0032] F p3 =Conv3×3(X1);

[0033] F p5 =Conv5×5(X1);

[0034] Among them, F p1 、F p3 、F p5 The output feature maps are respectively for the small-scale convolution path, the medium-scale convolution path, and the large-scale convolution path;

[0035] (3) Strip hybrid convolution block output fusion: The output features of the strip convolution block and the pyramid convolution block after parallel processing are fused:

[0036] The horizontal feature map F output by the strip convolution block h With the vertical feature map F v Unify the feature dimensions to obtain a uniform-sized strip convolution block output feature map F s :

[0037] F s = Align(F h ,F v ) ;

[0038] The output feature map F of the convolution path of each scale of the pyramid convolution block p1 、F p3 、F p5 Align and add the fusion to get a unified pyramid convolution block output feature map F p :

[0039] F p = Align(F p1 , F p3 , F p5 ) = F p1 + F p3 + F p5 ;

[0040] The strip mixed convolution block concatenates the two fused path feature maps, and then performs channel compression and feature integration through a 1×1 convolution kernel to obtain the final feature map F SMCB :

[0041] F SMCB = Conv 1×1 (Concat(F s ,F p )) .

[0042] Furthermore, in step S2, the workflow of the automatic search module includes:

[0043] (1) Weight fusion and normalization: W0 is the preset weight vector, W1 is the learnable weight vector, and W2 is the normalized weight vector; W0 is set based on prior knowledge and is used to guide the initial learning of the model; W1 is a parameter that needs to be learned during the model training process and is used to measure the importance of each module;

[0044] (2) Introducing threshold , compare W2 with the preset threshold T to generate a binary mask. If W2 is greater than the threshold, the corresponding position in the mask is true, indicating that the corresponding module will be selected; if W2 is less than the threshold, the corresponding position in the mask is false, indicating that the corresponding module will be excluded, which can be expressed as:

[0045] ;

[0046] in, m i is the mask value;

[0047] Determine the activated modules based on the mask:

[0048] ;

[0049] Among them, B={b1,b2,…,bn} is the candidate module set, bi represents the i-th module; is the set of activated modules, that is, only the modules with mask value 1 are selected to participate in subsequent calculations;

[0050] (3) The outputs of the activated modules are weighted and fused to obtain the final output result Y;

[0051] ;

[0052] in, m i is the mask value. If it is 1, it means that the corresponding module is activated and participates in weighting; wi is the weight of the corresponding module; bi is the output of the i-th activation module, and N is the total number of activated modules.

[0053] The beneficial effects of the present invention are:

[0054] This proposed method for marine mammal call classification based on IVGG-ASNet incorporates three key innovations and improvements: first, it introduces an enhanced convolution block, which enhances the network's modeling capabilities for time-domain and frequency-domain features through parallel convolution paths; second, it introduces a striped hybrid convolution block in the high-order convolution stage to enhance its ability to capture multi-band and long-range dependent speech features; and third, it designs and integrates an automatic search module, enabling the network structure to dynamically adjust and adaptively combine, automatically selecting the optimal convolutional structure combination based on varying input data. During training, a ten-fold cross-validation method is employed to ensure the stability of the results and the generalization of the model. Experimental results demonstrate that IVGG-ASNet outperforms mainstream classification models such as VGG16, ResNetSE, Res2Net, and CNN14 in key metrics such as precision, recall, and F1 score. This proposed method significantly advances the research and development of biomimetic covert underwater acoustic communication technology and makes a significant contribution to protecting marine animal biodiversity and building an environmentally friendly maritime nation. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is the overall flow chart of the IVGG-ASNet model according to an embodiment of the present invention;

[0056] Figure 2 This is a diagram of the enhanced convolution block structure of an embodiment of the present invention;

[0057] Figure 3 This is a structural diagram of a stripe hybrid convolution block according to an embodiment of the present invention;

[0058] Figure 4 This is a technical flow chart of the automatic search module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the technical means, creative features and effects achieved by the present invention easy to understand, the technical solutions in the embodiments of the present invention will be further clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0060] In recent years, marine mammal call classification technology has received widespread attention. However, due to the rapid development of ocean acoustics, the amount of acoustic data obtained from the ocean has exceeded the limit of manual processing. Therefore, designing an efficient and accurate marine mammal call classification algorithm has become a research hotspot. This example proposes a marine mammal call classification method based on IVGG-ASNet, which includes the following steps:

[0061] S1: Convert the acquired marine mammal call data into Mel spectrograms as feature representation to form a dataset;

[0062] S2: Using the dataset to construct and train an IVGG-ASNet classification model, the IVGG-ASNet classification model includes an enhanced convolution block, a stripe mixing convolution block, and an automatic search module;

[0063] The IVGG-ASNet classification model uses the VGG16 network as the backbone network, introduces the enhanced convolution block into the VGG network to replace the conventional convolution block in each convolution layer; adds one stripe hybrid convolution block to the fourth and fifth convolution stages of the VGG network respectively; and adds an automatic search module to the IVGG-ASNet classification model to enable the model to dynamically adjust the network structure according to data distribution and task requirements;

[0064] S3: Use the trained IVGG-ASNet classification model to classify the calls of marine mammals to be classified.

[0065] In step S1 of this embodiment, marine mammal call data is obtained from the Watkins marine mammal sound database, and the obtained marine mammal call data is preprocessed by clipping, denoising, and resampling. The preprocessed marine mammal call data is converted into a Mel spectrogram as a feature representation.

[0066] Step S1 specifically includes: downloading marine mammal call data and pre-processing it. The data set used in the present invention comes from the Watkins Marine Mammal Sound Database, which provides a variety of marine mammal call clips recorded in real marine environments.

[0067] The present invention selects the calls of four marine mammals that are widely distributed in my country's waters and have relatively large data volumes for download. After operations such as cropping, denoising, and resampling, 4190 call data samples with a duration of 1 second are finally obtained and manually labeled. The labeled content is the species to which each 1-second call sample belongs; there is a separate annotation file, which contains the path of each audio file and its corresponding label; the file contains a column of audio file paths and another column of corresponding species labels.

[0068] In the data preprocessing stage, the present invention performs feature analysis on randomly selected call samples. The features of different call samples are similar in the time domain, but significantly different in the frequency domain. Therefore, the feature representation selected by the present invention mainly comes from the frequency domain. Among them, the Mel scale can perform frequency conversion according to the auditory perception of the human ear and retain the important characteristics of the signal. In the speech recognition task, its accuracy and computational efficiency are better than other representations. Therefore, the present invention chooses to use the Mel spectrum as the feature representation. The Mel frequency formula is:

[0069] (1);

[0070] Wherein, f is the frequency expressed in Hertz, M is the frequency expressed in Mel, and the present invention sets the number of Mel scales to 128.

[0071] In step S2 of this embodiment, an enhanced convolution block is introduced into the VGG network to replace the conventional convolution block in each convolution layer; the enhanced convolution block uses a multi-scale convolution kernel to capture the temporal and frequency characteristics of the marine mammal call data.

[0072] Figure 1 The figure is an overall flow chart of the IVGG-ASNet classification model. As shown in the figure, the IVGG-ASNet model uses the VGG16 network as the backbone network, and the VGG network is the VGG16 network. In the IVGG-ASNet classification model, the three fully connected layers and the last pooling layer of the VGG16 network are removed, and the conventional convolution blocks of the VGG16 network in each convolution layer are replaced with enhanced convolution blocks.

[0073] The enhanced convolution block structure is shown in the figure Figure 2 As shown in Figure 2, the core design concept of the enhanced convolution block is to effectively capture the temporal and frequency characteristics of marine mammal call data by using multi-scale convolution kernels. Marine mammal calls contain rich biological information, which is often manifested in the different frequencies and time sequences of the sound. By combining convolution kernels of different sizes, the enhanced convolution block can capture data features at different scales, thereby achieving a more comprehensive understanding of the data content.

[0074] The enhanced convolution block consists of three parallel convolutions, namely a 1×3 horizontal kernel, a 3×1 vertical kernel, and a 3×3 standard convolution kernel. The outputs of the enhanced convolution block are summed to enrich the feature space;

[0075] Assume that the input feature map of the enhanced convolution block is X, with a size of H×W×D, where H is the height, W is the width, and D is the number of channels;

[0076] The expressions of the convolution operation and output fusion of the enhanced convolution block are as follows:

[0077] F1×3 = Conv 1×3 (X) ;

[0078] F 3×1 = Conv 3×1 (X);

[0079] F 3×3 = Conv 3×3 (X) ;

[0080] F ECB = F 1×3 + F 3×1 + F 3×3 ;

[0081] Among them, F 1×3 It represents the feature map obtained by convolution operation on the input feature map X using 1×3 convolution kernel, F 3×1 It represents the feature map obtained by convolution operation on the input feature map X using 3×1 convolution kernel, F 3×3 It represents the feature map obtained by convolution operation on the input feature map X using 3×3 convolution kernel, F ECB is the final output feature map of the enhanced convolution block.

[0082] Among them, the 1×3 convolution (horizontal convolution kernel) can capture changes in the audio signal along the time axis and is suitable for capturing long-term audio features. The 3×1 convolution (vertical convolution kernel) is used to identify changes in frequency, for example, different frequency bands respond to different sound signals. The 3×3 standard convolution kernel helps to extract local time-frequency information, enabling the network to effectively process short-term high-frequency changes. The output feature maps of each convolution path are added and fused, improving the model's feature expression capabilities while controlling the number of parameters and avoiding the introduction of redundant information.

[0083] In step S2 of this embodiment, the strip mixed convolution block added to the VGG network includes a strip convolution block and a pyramid convolution block; the strip mixed convolution block fuses the output features of the strip convolution block and the pyramid convolution block to output a final feature map.

[0084] In the IVGG-ASNet model built based on the VGG16 network, the present invention adds a strip mixed convolution block (SMCB) to the fourth convolution stage (Conv4_x) and the fifth convolution stage (Conv5_x) of the network backbone structure, respectively. Figure 3As shown in the figure, the module consists of two parts: Strip Convolution Block (SCB) and Pyramid Convolution Block (PCB). By integrating the SMCB module in these two high-order convolution stages, the present invention effectively enhances the network's modeling ability for long-distance dependencies and multi-band complex features, thereby making the model more adaptable and robust when processing data with high variability and nonlinear characteristics such as marine mammal calls.

[0085] The strip hybrid convolution block fuses the output features of the strip convolution block and the pyramid convolution block to output a final feature map, specifically including:

[0086] (1) Design of strip convolution blocks: The strip convolution blocks are composed of horizontal and vertical convolution kernels, which help to establish long-distance dependencies;

[0087] The horizontal convolution path uses a convolution kernel size of 1×n to perform a convolution operation on the input feature map along the time axis. The output feature map of the horizontal convolution path is recorded as:

[0088] F h = Conv 1×n (X) ;

[0089] Among them, X1 is the input feature map, the size is H×W×C, H is the height, W is the width, and C is the number of channels of the input feature map; F h It is the horizontal feature map output by the horizontal convolution path, with a size of H×(W-n+1)×C′; C′ is the number of channels of the output feature.

[0090] The vertical convolution path uses a convolution kernel size of n×1 to perform a convolution operation on the input feature map along the frequency axis. The output feature map of the vertical convolution path is recorded as:

[0091] F v = Conv n×1 (X) ;

[0092] Among them, F v is the vertical feature map output by the vertical convolution path, with a size of (H-n+1)×W×C′;

[0093] The functions of the horizontal convolution path and the vertical convolution path are to capture the long-distance dependencies of the input data in the time dimension and frequency dimension respectively, so as to significantly expand the receptive field of the network and enhance the model's ability to capture the features of global patterns. This structure is particularly suitable for capturing global features such as long-term trends or frequency band changes in the calls of marine mammals.

[0094] (2) Design of pyramid convolution block: The pyramid convolution block adopts convolution kernels of three sizes: small scale, medium scale and large scale;

[0095] Among them, the convolution kernel size of the small-scale convolution path is 1×1×1; the convolution kernel size of the medium-scale convolution path is 3×3×3; the convolution kernel size of the large-scale convolution path is 5×5×5;

[0096] The three parallel paths of small-scale convolution path, medium-scale convolution path and large-scale convolution path are used to perform convolution operation on the input feature map X1 respectively. The expression is:

[0097] F p1 =Conv1×1(X1);

[0098] F p3 =Conv3×3(X1);

[0099] F p5 =Conv5×5(X1);

[0100] Among them, F p1 、F p3 、F p5 The output feature maps are respectively for the small-scale convolution path, the medium-scale convolution path, and the large-scale convolution path;

[0101] The output feature map sizes of the three convolution paths vary due to the different convolution kernel sizes. Therefore, before fusion, appropriate spatial dimension cropping or zero padding is required to align the dimensions and unify the dimensions after adjustment.

[0102] The output feature maps of the small-scale convolution path, the medium-scale convolution path, and the large-scale convolution path are cropped or zero-filled in the spatial dimension to align the dimensions. After adjustment, the dimensions are unified to H′×W′×C′′, where H′ represents the height of the output feature map after processing by the pyramid convolution block, W′ represents the width of the output feature map after processing by the pyramid convolution block, and C′′ represents the number of channels of the output feature map after processing by the pyramid convolution block.

[0103] Through the above-mentioned multi-scale convolution kernel structure, the pyramid convolution block can effectively capture the fine feature information of the input features at different scales, and is particularly suitable for capturing the detailed features and local nonlinear changes in different frequency bands in the call signals of marine mammals.

[0104] (3) Strip hybrid convolution block output fusion: The output features of the strip convolution block and the pyramid convolution block after parallel processing are fused:

[0105] The horizontal feature map F output by the strip convolution block h With the vertical feature map F vUnify the feature dimensions to obtain a uniform-sized strip convolution block output feature map F s :

[0106] F s = Align(F h ,F v ) ;

[0107] The output feature map F of the convolution path of each scale of the pyramid convolution block p1 、F p3 、F p5 Align and add the fusion to get a unified pyramid convolution block output feature map F p :

[0108] F p = Align(F p1 , F p3 , F p5 ) = F p1 + F p3 + F p5 ;

[0109] The strip mixed convolution block concatenates the two fused path feature maps, and then performs channel compression and feature integration through a 1×1 convolution kernel to obtain the final feature map F SMCB :

[0110] F SMCB = Conv 1×1 (Concat(F s ,F p )) .

[0111] Through the above fusion method, the stripe hybrid convolution block can simultaneously extract the long-distance dependencies and local multi-scale information of the input data, achieving more efficient and comprehensive feature representation.

[0112] In step S2 of this embodiment, an automatic search module is designed and added to enable the model to automatically learn and combine the optimal feature structure during training, dynamically adjust the network structure according to data distribution and task requirements, and optimize classification performance; the automatic search module is a module based on learnable weights, and the technical flow chart of the automatic search module is as follows: Figure 4As shown in the figure, the core innovation of this module lies in its implementation of a plug-and-play modular mechanism, which greatly improves the flexibility of the network structure and the versatility of the model. At the same time, this module enables the model to automatically learn and dynamically select an optimal module combination from multiple candidate modules during training, and finally achieve the fusion of module outputs through an adaptive weight mechanism. The introduction of this module effectively gives the model the ability to dynamically adjust its internal structure, allowing it to self-optimize based on the characteristics of specific data and task requirements. This automated structural adjustment reduces reliance on manual design, breaks through the limitations of traditional neural networks in module selection, enhances the adaptability and robustness of the model, and provides strong support for coping with complex and changing data environments and task requirements.

[0113] In step S2, the workflow of the automatic search module includes:

[0114] (1) Weight fusion and normalization: W0 is the preset weight vector, W1 is the learnable weight vector, and W2 is the normalized weight vector; W0 is set based on prior knowledge and is used to guide the initial learning of the model; W1 is a parameter that needs to be learned during the model training process and is used to measure the importance of each module;

[0115] W1 and W0 are fused through element-by-element multiplication. The significance of fusion lies in combining the model's learning ability with human experience, fully leveraging existing knowledge and experience to accelerate the model's learning process and improve model performance. The fused weight vector is normalized using the Softmax function to generate the normalized weight vector W2. The Softmax function converts the weight vector into a probability distribution, ensuring that the sum of all weights is 1.

[0116] (2) Introducing threshold , compare W2 with the preset threshold T to generate a binary mask. If W2 is greater than the threshold, the corresponding position in the mask is true, indicating that the corresponding module will be selected; if W2 is less than the threshold, the corresponding position in the mask is false, indicating that the corresponding module will be excluded, which can be expressed as:

[0117] ;

[0118] in, m i is the mask value; the threshold T controls the sparsity of module selection. By adjusting the threshold, we can control the sparsity of module selection, thereby achieving a trade-off between model performance and efficiency. The significance of introducing the threshold is to control the number of module activations, that is, only selecting a few important modules rather than participating in the calculation. This can reduce the computational complexity of the model, improve the efficiency and generalization ability of the model, and prevent overfitting.

[0119] Determine the activated (selected) module based on the mask:

[0120] ;

[0121] Among them, B={b1,b2,…,bN} is the set of candidate modules, bi represents the i-th module; is the set of activated modules, that is, only modules with a mask value of 1 are selected to participate in subsequent calculations.

[0122] Based on the generated mask, the corresponding block is selected from the candidate block set b1, b2, …, bN. Only blocks whose corresponding positions in the mask are true are selected. The selected blocks are then applied to the input data. Dynamic block selection is the core of this mechanism. It enables the model to automatically learn the network structure, allowing it to automatically select the most appropriate block for processing based on the characteristics of the input data and its own learning process. This makes the model more adaptable and robust, and better able to cope with different tasks and data distributions.

[0123] (3) The outputs of the activated modules are weighted and fused to obtain the final output result Y:

[0124] .

[0125] in, m i It is the mask value. If it is 1, it means that the module is activated and participates in weighting. w i is the weight of the module; b i It is i The output of the activation module, N is the total number of activated modules.

[0126] Weighted fusion preserves the strengths of each block, improving the expressiveness and discriminability of features. Ultimately, the combined features are fed into a classifier to output a prediction. By learning appropriate weights, the model automatically adjusts the contributions of different modules to achieve optimal performance.

[0127] The method of this embodiment also includes: designing baseline comparison experiments and ablation experiments to evaluate the effectiveness of the IVGG-ASNet classification model. The performance of the model is evaluated using three evaluation indicators: precision, recall, and F1 score:

[0128] ;

[0129] ;

[0130] ;

[0131] Where Precision is the ratio of correctly predicted positive samples to all samples predicted as positive by the model, True Positives is the correctly predicted positive sample, False Positives is the incorrectly predicted positive sample, so True Positives plus False Positives is all samples predicted as positive by the model;

[0132] Recall is the ratio of correctly predicted positive samples to all samples correctly predicted as positive by the model, where True Positives are correctly predicted as positive samples, False Negatives are incorrectly predicted as negative samples, and True Positives plus False Negatives are all samples correctly predicted as positive by the model.

[0133] The F1 score is the harmonic mean of precision and recall. Precision focuses on evaluating the accuracy of the prediction results, while Recall focuses on measuring the coverage of the prediction results. The F1 score can comprehensively consider the precision and recall of the model to more comprehensively evaluate the model performance.

[0134] This paper selected four currently mainstream and high-performance audio classification models for baseline comparison experiments to evaluate the effectiveness of the IVGG-ASNet model proposed in this paper in the task of marine mammal call classification. The selected baseline models are VGG16, Res2Net, ResNetSE, and CNN14. Model training adopts a unified parameter configuration, Adam is selected as the optimizer, the initial learning rate is set to 0.0001, the number of training rounds is 200, and the batch size is 32. The dataset is divided into 10-fold cross-validation, that is, all samples are randomly divided into 10 non-overlapping subsets, each subset accounts for approximately 10% of the total dataset. 9 of the subsets are used as training sets each time, and the remaining subset is used as a validation set to ensure the stability and generalization ability of the model.

[0135] To explore the contribution of each core module in the IVGG-ASNet model to overall performance, we designed an ablation experiment. We removed each of the model's three key submodules—the enhanced convolution block, the striped hybrid convolution block, and the automatic search module—and constructed corresponding variant models. We then conducted a comparative analysis using the same dataset, training parameters, and evaluation methods. By comparing the precision, recall, and F1 score of each model variant, we assessed the actual impact of each module on classification performance.

[0136] The present invention proposes a marine mammal call classification method based on IVGG-ASNet. The method designs a classification model based on an improved VGG16 network and an automatic search block. The improved VGG16 network uses an enhanced convolution block to replace its conventional convolution block in each convolution layer, and adds a strip mixing convolution block in the last two stages of each convolution layer to improve the model's feature extraction capabilities at different scales. At the same time, the present invention designs an automatic search block to give the model dynamic structure combination and adaptive learning capabilities. Ultimately, the model is able to autonomously learn, select and combine the optimal convolution structure during the training process. The marine mammal call classification method based on IVGG-ASNet proposed in the present invention can effectively overcome the shortcomings of existing methods, improve recognition and classification efficiency and accuracy, and provide strong technical support for marine mammal ecological monitoring, marine environmental monitoring, and bioacoustic research.

[0137] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for a person skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to replace some of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions claimed to be protected by the present invention.

Claims

1. A marine mammal call classification method based on IVGG-ASNet, characterized in that: The method comprises the following steps: S1: Convert the acquired marine mammal call data into Mel spectrograms as feature representation to form a dataset; S2: Using the dataset to construct and train an IVGG-ASNet classification model, the IVGG-ASNet classification model includes an enhanced convolution block, a stripe mixing convolution block, and an automatic search module; S3: Use the trained IVGG-ASNet classification model to classify the calls of marine mammals to be classified; In step S2, the workflow of the automatic search module includes: (1) Weight fusion and normalization: W0 is the preset weight vector, W1 is the learnable weight vector, and W2 is the normalized weight vector; W0 is set according to prior knowledge and is used to guide the initial learning of the model; W1 is a parameter that needs to be learned during the model training process and is used to measure the importance of each module; W1 and W0 are fused by element-by-element multiplication, and the fused weight vector is normalized by the Softmax function to generate the normalized weight vector W2; (2) Introducing threshold , compare W2 with the preset threshold T to generate a binary mask. If W2 is greater than the threshold, the corresponding position in the mask is true, indicating that the corresponding module will be selected; if W2 is less than the threshold, the corresponding position in the mask is false, indicating that the corresponding module will be excluded, which can be expressed as: ； in, m i is the mask value; Determine the activated modules based on the mask: ； Where B={b1,b2,…,b n } is a set of candidate modules, b i represents the i-th module; is the set of activated modules, that is, only the modules with mask value 1 are selected to participate in subsequent calculations; (3) The outputs of the activated modules are weighted and fused to obtain the final output result Y; ； in, m i Is the mask value. If it is 1, it means that the corresponding module is activated and participates in weighting; w i is the weight of the corresponding module; b i is the output of the i-th activation module, and N is the total number of activated modules.

2. A marine mammal call classification method based on IVGG-ASNet according to claim 1, characterized in that: In step S1, marine mammal call data is obtained from the Watkins marine mammal sound database, and the obtained marine mammal call data is preprocessed by clipping, denoising, and resampling, and the preprocessed marine mammal call data is converted into a Mel spectrogram as a feature representation.

3. The marine mammal call classification method based on IVGG-ASNet according to claim 1, characterized in that: In step S2, the enhanced convolution block uses multi-scale convolution kernels to capture the temporal and frequency characteristics of marine mammal call data.

4. A marine mammal call classification method based on IVGG-ASNet according to claim 3, characterized in that: The enhanced convolution block consists of three parallel convolutions, namely a 1×3 horizontal kernel, a 3×1 vertical kernel, and a 3×3 standard convolution kernel. The outputs of the enhanced convolution block are summed to enrich the feature space; Assume that the input feature map of the enhanced convolution block is X, with a size of H×W×D, where H is the height, W is the width, and D is the number of channels; The expressions of the convolution operation and output fusion of the enhanced convolution block are as follows: F 1×3 = Conv 1×3 (X) ; F 3×1 = Conv 3×1 (X); F 3×3 = Conv 3×3 (X) ; F ECB = F 1×3 + F 3×1 + F 3×3 ; Among them, F 1×3 It represents the feature map obtained by convolution operation on the input feature map X using 1×3 convolution kernel, F 3×1 It represents the feature map obtained by convolution operation on the input feature map X using 3×1 convolution kernel, F 3×3 It represents the feature map obtained by convolution operation on the input feature map X using 3×3 convolution kernel, F ECB is the final output feature map of the enhanced convolution block.

5. The marine mammal call classification method based on IVGG-ASNet according to claim 1, characterized in that: In step S2, the strip mixed convolution block includes a strip convolution block and a pyramid convolution block; the strip mixed convolution block fuses the output features of the strip convolution block and the pyramid convolution block, and outputs a final feature map.

6. The marine mammal call classification method based on IVGG-ASNet according to claim 5, characterized in that: The strip hybrid convolution block fuses the output features of the strip convolution block and the pyramid convolution block to output a final feature map, specifically including: (1) Design of strip convolution block: The strip convolution block consists of horizontal and vertical convolution kernels; The horizontal convolution path uses a convolution kernel size of 1×n to perform a convolution operation on the input feature map along the time axis. The output feature map of the horizontal convolution path is recorded as: F h = Conv 1×n (X1) ; Among them, X1 is the input feature map, the size is H×W×C, H is the height, W is the width, and C is the number of channels of the input feature map; F h is the horizontal feature map output by the horizontal convolution path, with a size of H×(W-n+1)×C′; C′ is the number of channels of the output feature map; The vertical convolution path uses a convolution kernel size of n×1 to perform a convolution operation on the input feature map along the frequency axis. The output feature map of the vertical convolution path is recorded as: F v = Conv 1×n (X1) ; Among them, F v is the vertical feature map output by the vertical convolution path, with a size of (H-n+1)×W×C′; (2) Design of pyramid convolution block: The pyramid convolution block adopts convolution kernels of three sizes: small scale, medium scale and large scale; Among them, the convolution kernel size of the small-scale convolution path is 1×1×1; the convolution kernel size of the medium-scale convolution path is 3×3×3; the convolution kernel size of the large-scale convolution path is 5×5×5; The three parallel paths of small-scale convolution path, medium-scale convolution path and large-scale convolution path are used to perform convolution operation on the input feature map X1 respectively. The expression is: F p1 =Conv1×1(X1); F p3 =Conv3×3(X1); F p5 =Conv5×5(X1); Among them, F p1 、F p3 、F p5 The output feature maps are respectively for the small-scale convolution path, the medium-scale convolution path, and the large-scale convolution path; (3) Strip hybrid convolution block output fusion: The output features of the strip convolution block and the pyramid convolution block after parallel processing are fused: The horizontal feature map F output by the strip convolution block h With the vertical feature map F v Unify the feature dimensions to obtain a uniform-sized strip convolution block output feature map F s : F s = Align(F h ,F v ) ; The output feature map F of the convolution path of each scale of the pyramid convolution block p1 、F p3 、F p5 Align and add the fusion to get a unified pyramid convolution block output feature map F p : F p = Align(F p1 ,F p3 ,F p5 ) = F p1 + F p3 + F p5 ; The strip mixed convolution block concatenates the two fused path feature maps, and then performs channel compression and feature integration through a 1×1 convolution kernel to obtain the final feature map F SMCB : F SMCB = Conv 1×1 (Concat(F s ,F p )) 。

Images