A deep learning-based OBS whale whistle signal detection and recognition method

Abstract

The application discloses a kind of OBS whale whistle signal detection identification method based on deep learning, belong to underwater signal identification classification technical field;Method includes: the original OBS record data is preprocessed;Periodic signal detection is carried out using STA / LTA-CV detection algorithm, the algorithm is combined with long short time average ratio and coefficient of variation constraint, effectively suppress non-stationary noise interference;A double-branch deep learning model is constructed, waveform branch uses CLSTM network to extract time domain feature, and time-frequency diagram branch uses ResNet50-CBAM network to extract time-frequency feature;Cross-modal feature fusion is realized by transformer encoder, and dynamic weight module is introduced to adaptively modal weighting;Finally, the whale signal class and confidence are output through classification head network;The method has the advantages of high detection efficiency, high recognition accuracy, and provides an effective technical means for marine bioacoustics research.

Description

Technical Field

[0001] This invention belongs to the field of underwater signal recognition and classification technology, and in particular relates to a deep learning-based OBS whale whistle signal detection and recognition method. Background Technology

[0002] Underwater signal detection and identification methods based on Ocean Bottom Seismometers (OBS) can generally be divided into two independent stages: signal detection and signal identification. Firstly, in the signal detection stage, existing methods mainly include energy detection, time-domain and time-frequency domain methods, statistical detection methods, matched filtering and correlation detection, and machine learning methods, depending on the processing object and framework. In summary, existing detection methods share the following common limitations: First, the detection accuracy of statistical detection, matched filtering, and machine learning methods usually relies on prior information; template construction and dataset refinement require a certain understanding of the time-frequency domain attributes of the target signal. Second, time-frequency domain methods, represented by wavelet transform, have complex parameter tuning and high computational costs. Third, existing methods generally have weak adaptability to underwater non-stationary noise and insufficient directionality in detecting weakly periodic signals, such as whale whistles, resulting in complex initial screening results and a heavy burden of subsequent manual verification.

[0003] In the signal recognition stage, existing methods mainly include: manual inspection, template matching, and machine learning methods. Existing recognition methods have two common limitations: First, the strong dependence on prior information restricts the applicability of the methods to new or unknown signal categories; second, when the sound source environment of the received signal is complex, the signal-to-noise ratio is low, or the manual feature extraction scheme is not compatible with the model structure, the classification accuracy will decrease significantly.

[0004] Therefore, in order to solve and improve the above-mentioned problems of existing cetacean signal detection and recognition methods, there is an urgent need for an underwater bioacoustic signal detection and recognition method with strong generalization ability and robustness that can effectively adapt to the application requirements of real underwater complex noise environment. Summary of the Invention

[0005] The purpose of this invention is to provide a deep learning-based method for detecting and recognizing OBS whale whistle signals, addressing the shortcomings of existing technologies such as strong reliance on prior information, poor adaptability to non-stationary noise, low sensitivity in detecting periodic signals, insufficient accuracy in complex acoustic environments, and heavy manual review burden. This invention combines the STA / LTA-CV detection algorithm with a CLSTM-ResNet dual-branch deep learning model. In the detection stage, it efficiently filters suspected periodic signals with extremely low computational cost. In the recognition stage, it achieves high-accuracy signal classification through multi-feature fusion and deep learning multi-module modeling.

[0006] To achieve the above objectives, this invention provides a deep learning-based method for detecting and recognizing OBS whale whistle signals, comprising the following steps: S1. Preprocess the raw seismic records, including removing instrument response, detrending, filtering, and converting file formats; S2. Initial screening of periodic signals based on STA / LTA-CV; S3. Organize and pair the suspected target signal segments obtained in S2 according to the dual-channel input format of the original time-series waveform data and its corresponding time-frequency image to construct a batch inference dataset. S4. Input the batch inference dataset obtained in S3 into the pre-trained CLSTM-ResNet dual-branch deep learning model with fused attention mechanism to perform batch automatic recognition and classification on each signal segment. S5. Based on the classification head network, the classification labels and corresponding confidence scores output by the model are summarized and statistically analyzed to generate complete signal recognition and classification results.

[0007] Preferably, the specific content of S2 is as follows: S21. Set the parameter values ​​to be used in the initial signal screening stage; S22. Calculate the short-time average energy within the sliding window. With long-term average energy The ratio of the characteristic function is obtained. The expression is as follows: ; ; ; in, For short time window length, For the length of the long window, The average energy over a short time window. The average energy over a long window. For the instantaneous energy of the signal, The ratio of characteristic functions; S23, when A short-term event is determined to be triggered when the value is greater than the trigger threshold and the minimum interval condition is met. The event ends when the value is below the release threshold. S24. Taking the first short-term event detected by STA / LTA as the starting point, and continuously detecting m short-term events within a fixed window length, it is determined that the sample fragment contains a periodic signal caused by biological activity. The CV value is calculated as follows: ; in, This represents the number of events detected under the corresponding window length. Represents standard deviation, Represents the mean. The time interval between adjacent short-term events. This represents the average value of the intervals within the current window. S25. Calculate the standard deviation and mean of the event intervals of the continuous short-term events detected under this window length to obtain the CV value; if the CV value of the segment is lower than the preset threshold, the sample segment has a high degree of determinism in the time dimension, rather than random pulse interference caused by occasional seabed activities such as turbulence or biological collisions, and it is extracted as a sample to be classified and identified; when the interval between two adjacent short-term events exceeds the preset maximum period, it is determined that the current whale vocalization event has ended, and the detection is restarted in the next window.

[0008] Preferably, S4 specifically includes: S41. Construct a dual-branch deep learning model, including a waveform branch and a time-frequency graph branch; S42. Perform cross-modal fusion of waveform features and time-frequency plot features using a Transformer encoder; S43. Introduce a dynamic weighting module to adaptively modally weight the fused features and generate the final fused feature vector. Preferably, the specific content of S41 is as follows: The waveform branch uses a long short-term memory network CLSTM with embedded one-dimensional convolutional modules to capture long-term temporal dependencies, while extracting local features of the waveform. The time-frequency graph branch uses a deep residual network ResNet50 with fused convolutional block attention module CBAM. It utilizes the residual learning framework to alleviate the gradient vanishing problem in deep network training and adaptively focuses on the voiceprint region in the time-frequency graph by weighting the channel attention submodule and spatial attention submodule in CBAM in sequence. Preferably, the specific details of adaptive focusing on the acoustic pattern region in the time-frequency diagram in S412 through sequential weighting of the channel attention submodule and the spatial attention submodule in CBAM are as follows: S411. Perform global average pooling and global max pooling on the input feature map F in the spatial dimensions [H, W] respectively, and aggregate the spatial background information to obtain two one-dimensional channel descriptors. and The dimensions are all [B, C, 1, 1], and the expression is as follows: ; ; Will and The inputs are fed into a shared multilayer perceptron (MLP) for nonlinear feature mapping. The MLP contains a hidden layer with a channel reduction rate of r. The mapping outputs are summed element-wise and then passed through a sigmoid activation function to generate channel attention weight vectors. Its dimensions are [B, C, 1, 1], and its expression is as follows: ; The forward computation formula for MLP is as follows: ; In the formula, The dimension is [C / r, C]. The dimension is [C, C / r]; Channel attention weights Element-wise multiplication with the original input feature map F yields the channel reconstructed feature map. Its dimensions are [B, C, H, W], and its expression is as follows: ; S412. In terms of channel dimension, channel reconstruction features Average pooling and max pooling are performed separately to extract strong features and weak background of the spatial region, resulting in two two-dimensional spatial descriptor maps. and Their dimensions are all [B, 1, H, W], and their expressions are as follows: ; ; Will and The features are concatenated along the channel dimension to obtain joint features of dimension [B, 2, H, W]. A receptive field convolutional layer is used to compress the number of channels to 1, followed by a sigmoid activation function to generate a spatial attention weight map. Its dimensions are [B, 1, H, W], and its expression is as follows: ; Spatial attention weights With channel reconstruction features Perform element-wise multiplication to output the final reconstructed feature map. Its dimensions are [B, C, H, W], and its expression is as follows: .

[0009] Preferably, the specific content of S42 is as follows: set up and These are the input waveform features and image features, respectively. Given the self-attention weight matrix, the fusion expression is as follows: ; Right now: ; The fusion module then outputs the following expression: ; in, , , Train the parameter matrix for the Transformer. Generate a query vector based on the dimensions of the projected vector. key vector AND value vector ; After self-attention interaction, the output features complete cross-modal interaction, generating enhanced features containing global information. To integrate new waveform features from image modal information, To create new image features by fusing waveform modal information; Introducing a dynamic weight module to enhance the output of the Transformer and Perform adaptive modal fusion; The output modal features are then subjected to linear projection mapping to obtain scalar importance scores, as shown in the following expression: ; in, For the learned linear projection matrix, For the corresponding modal index, For bias terms; go through Function normalization yields the dynamic fusion weights, expressed as follows: ; Dynamic fusion weights must meet the following constraints. .

[0010] Preferably, the specific content of S43 is as follows: The model adaptively allocates the contribution ratio of different modalities based on the characteristics of the input samples, and forms a fused feature vector by weighted summation, as shown in the following expression: ; Preferably, in S5, during the model inference phase, the system first processes the unnormalized log probability output by the model. Logits conduct Softmax Normalization is performed to calculate the predicted probability distribution of a sample belonging to each category, as shown in the following expression: ; The maximum probability is the prediction confidence of the sample, and the corresponding category is the prediction label.

[0011] Preferably, the dataset contains 7504 samples of blue whales, fin whales, mixed fin whales, and whales without noise. The ratio of training set, validation set, and test set is 7:1.5:1.5. During the training phase, data augmentation strategies such as waveform random noise, time shifting, masking, image color jittering, and random erasure are adopted.

[0012] Preferably, the system based on the above recognition method includes: a preprocessing module, a STA / LTA-CV periodic detection module, a dual-channel data construction module, a ConvLSTM-ResNet dual-branch deep learning model module, a feature fusion and dynamic weight module, and a classification output module.

[0013] Therefore, the beneficial effects of the OBS whale whistle signal detection and recognition method based on deep learning described above are as follows: (1) The STA / LTA-CV detection algorithm has low computational complexity and can process a large amount of continuous OBS data in a very short time. The detection speed is improved compared with the traditional time-frequency domain method, providing technical support for large-scale data processing. (2) By introducing the coefficient of variation constraint, this method makes full use of the periodicity of whale whistle signals and can still maintain high detection sensitivity in complex noise environments; (3) The CLSTM-ResNet dual-branch model achieves a high recognition accuracy on the test set by fully utilizing the complementary information of time-domain waveforms and time-frequency images through multimodal feature fusion. (4) The CBAM attention mechanism and dynamic weight module enable the model to adaptively focus on key feature regions, maintain stable recognition performance under different signal-to-noise ratios and different acoustic environments, and significantly improve generalization ability.

[0014] (5) The STA / LTA-CV detection algorithm has a low dependence on signal templates. The CLSTM-ResNet model automatically extracts features through end-to-end learning, reducing the reliance on manual feature engineering and expert experience. (6) The entire detection and identification process is fully automated, requiring no manual intervention, which greatly improves data processing efficiency and reduces labor costs.

[0015] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0016] Figure 1This is a flowchart of a deep learning-based OBS method for detecting and recognizing whale whistling signals according to the present invention. Figure 2 Here is a diagram showing the specific architecture of the classification model CLSTM-ResNet; Figure 3 The following are sample distribution diagrams, where (a) is a distribution diagram of the number of samples in each category, and (b) is a diagram of the sample set partitioning. Figure 4 The waveform mode weights on the test set corresponding to the optimal results of CLSTM-ResNet Nuclear density distribution map; Figure 5 This is an example diagram for application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages disclosed in the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention and are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0018] It should be noted that the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as a process, method, system, product, or server that includes a series of steps or units, not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or device.

[0019] The following is combined Figures 1-5 The embodiments of the present invention will be described in detail below.

[0020] Example 1 like Figure 1 As shown, this embodiment provides a deep learning-based OBS whale whistle signal detection and recognition method, including the following steps: Step S1: Data preprocessing; First, the raw OBS record data is preprocessed, specifically including: Instrument response removal: Based on the transfer function of the OBS instrument, the influence of the instrument response on the signal is removed to restore the true seafloor seismic signal; Detrending: A linear detrending method is used to eliminate DC components and long-term trend changes in the signal; Band-pass filtering: Design a 4th-order Butterworth band-pass filter with a passband frequency range of 5 - 60 Hz, covering the main call frequency bands of baleen whales, and effectively removing low-frequency ground motion noise and high-frequency interference; Format conversion: Convert the original data format to the standard CSV format for subsequent processing.

[0021] The sampling rates of the preprocessed signals are mainly 50 Hz and 125 Hz. Each data sample has a duration of approximately 2600 seconds and contains background noise and various whale signals.

[0022] Step S2: STA / LTA-CV detection algorithm; This algorithm combines the traditional short-term and long-term average ratio method with coefficient of variation constraint, and the specific implementation is as follows: 1. Parameter optimization settings; After traversal experiments with grid search, the optimal parameter settings determined are shown in Table 1: Table 1 Parameter settings of STA / LTA-CV detection algorithm

[0023] 2. Calculation of STA / LTA feature function; First, calculate the short-time window average energy and the long-time window average energy : ; ; where, is the instantaneous energy of the signal, is the short-time window length, is the long-time window length.

[0024] Then calculate the feature function ratio : ; 3. Trigger detection logic; When >on_thr and meets the min_gap constraint, trigger event detection; when <off_thr, end event detection. At the same time, it is required that the duration of a single trigger is not less than min_dur.

[0025] 4. Detection of periodic signal sequences (CV screening); The algorithm uses a sliding window. Each time, m = 3 consecutive events are extracted as the basic event group. First, perform a hard interval constraint determination: It is required that the peak time intervals between adjacent events must all be between 10 s and 60 s; After applying the constraints, immediately calculate the coefficient of variation for that set of intervals: ; like CV If the value is ≤0.4, it is established as a periodic seed sequence.

[0026] 5. Dynamic expansion mechanism; After establishing the periodic seed sequence, dynamic expansion begins: If the interval between the next event and the end of the current sequence is ≤20s and the relative deviation of this interval from the average interval of the current sequence is ≤0.3 (i.e., the tolerance is within 30%), then they are merged. Simultaneously update the average interval of the sequence and continue to expand until the pattern is broken or the time is interrupted; The window will then jump and slide to the end of the processed sequence in order to find the next independent sequence.

[0027] 6. Sequence fusion and labeling; For each detected periodic sequence, a time window is constructed from 10 seconds before the first peak to 10 seconds after the last peak. If the interval between two windows is ≤20 seconds, a union fusion is performed to select and mark the subsequences, and the independent information of each subsequence is fully preserved.

[0028] Step S3: Sample construction; The suspected target signal segments obtained from S2 are organized and paired according to the dual-channel input format of the original time-series waveform data and its corresponding time-frequency image to construct a batch inference dataset. Step S4: Construction of a dual-branch deep learning model; like Figure 2 As shown: 1. The waveform branch (CLSTM) uses a Long Short-Term Memory (LSTM) network with embedded one-dimensional convolutional modules to capture long-range temporal dependencies and extract local features of the waveform. Specific parameters are shown in Table 2. Table 2 CLSTM Waveform Branch Parameters

[0029] 2. The time-frequency graph branch (ResNet50-CBAM) adopts a deep residual network ResNet50 that integrates the convolutional block attention module (CBAM). It utilizes the residual learning framework to alleviate the gradient vanishing problem during deep network training and adaptively focuses on the speaker region in the time-frequency graph through sequential weighting of the channel attention submodule and spatial attention submodule in CBAM. The spectrogram generation parameters are shown in Table 3. Table 3 Parameters for generating spectrum diagrams

[0030] The ResNet50-CBAM branch parameters are shown in Table 4: Table 4 ResNet50-CBAM branch parameters

[0031] 3. CBAM Attention Module; It includes a channel attention submodule and a spatial attention submodule.

[0032] Channel attention mechanism: Global average pooling and global maximum pooling are performed on the input feature map F in the spatial dimensions [H, W] to aggregate spatial background information and obtain two one-dimensional channel descriptors. and The dimensions are all [B, C, 1, 1], and the expression is as follows: ; ; Will and The inputs are fed into a shared multilayer perceptron (MLP) for non-linear feature mapping. The MLP contains a hidden layer with a channel reduction rate of r (default r=16). The mapped outputs are element-wise summed and then passed through a sigmoid activation function to generate channel attention weight vectors. Its dimensions are [B, C, 1, 1], and its expression is as follows: ; The forward computation formula for MLP is as follows: ; In the formula, The dimension is [C / r, C]. The dimension is [C, C / r]; Channel attention weights Element-wise multiplication (Broadcasting mechanism) is performed with the original input feature map F to obtain the channel reconstruction feature map. Its dimensions are [B, C, H, W], and its expression is as follows: ; Spatial attention mechanism: On the channel dimension (dim=1), the channel reconstruction features are... Average pooling and max pooling are performed separately to extract strong features and weak background of the spatial region, resulting in two two-dimensional spatial descriptor maps. and Their dimensions are all [B, 1, H, W], and their expressions are as follows: ; ; Will and Concatenate the channels to obtain joint features of dimension [B, 2, H, W]; compress the number of channels to 1 through a receptive field convolutional layer, and then generate a spatial attention weight map through a sigmoid activation function. Its dimensions are [B, 1, H, W], and its expression is as follows: ; Spatial attention weights With channel reconstruction features Perform element-wise multiplication to output the final reconstructed feature map. Its dimensions are [B, C, H, W], and its expression is as follows: .

[0033] Step S5: Transformer cross-modal fusion; The Transformer encoder parameters are shown in Table 5: Table 5 Transformer encoder parameters

[0034] set up and These are the input waveform features and image features, respectively. Given the self-attention weight matrix, the fusion expression is as follows: ; Right now: ; The fusion module then outputs the following expression: ; in, , , Train the parameter matrix for the Transformer. Generate a query vector based on the dimensions of the projected vector. key vector AND value vector ; After self-attention interaction, the output features complete cross-modal interaction, generating enhanced features containing global information. To integrate new waveform features from image modal information, To create new image features by fusing waveform modal information; Introducing a dynamic weight module to enhance the output of the Transformer and Perform adaptive modal fusion; The output modal features are then subjected to linear projection mapping to obtain scalar importance scores, as shown in the following expression: ; in, For the learned linear projection matrix, For the corresponding modal index, For bias terms; go through Function normalization yields the dynamic fusion weights, expressed as follows: ; Dynamic fusion weights must meet the following constraints. .

[0035] Step S6: Dynamic weight module; The model adaptively allocates the contribution ratio of different modalities based on the characteristics of the input samples, and forms a fused feature vector by weighted summation, as shown in the following expression: ; Step S7: Classification Head Network; The network structure parameters of the classification head are shown in Table 6: Table 6 Classification Head Network Structure Parameters

[0036] In S5, during the model inference phase, the system first processes the unnormalized log probability output by the model. Logits conduct Softmax Normalization is performed to calculate the predicted probability distribution of a sample belonging to each category, as shown in the following expression: ; The maximum probability is the prediction confidence of the sample, and the corresponding category is the prediction label.

[0037] Example 2 Dataset construction and training; 1. Dataset construction; This study collected four types of signal samples based on measured data from the 7D satellite network: blue whale, fin whale, mixed fin whale signals, and background noise without whales. The dataset contains 7504 samples, and the distribution of samples in each category is as follows: Figure 3 (a) and Figure 3 As shown in (b) of the diagram.

[0038] Each sample consists of an original waveform with 30,000 sampling points (padding with zeros if the length is insufficient) and its corresponding time-frequency plot (PNG format). The ratio of the training set, validation set, and test set is set to 7:1.5:1.5, and a fixed random seed (seed=42) is used for partitioning.

[0039] 2. Data augmentation strategies; To improve the model's generalization ability, this study employs data augmentation strategies on the training set, which are executed in real-time during the data loading phase: Waveform mode: Add random Gaussian noise (N(0, 0.01)), random time shift (maximum shift amount is ±10% of sequence length), and random mask (mask length is 5%~10% of sequence length). Image modalities: color jitter (brightness, contrast, and saturation set to 0.2), random erasure (trigger probability set to 50%, erasure area 2%~10%).

[0040] 3. Training configuration; The training configuration is shown in Table 7: Table 7 Training Configuration

[0041] Example 3 Model performance evaluation; 1. Evaluation indicators; The evaluation metrics for the model are accuracy, precision (P), recall (R), and F1 score (F1-Score) of the recognition results on the test set.

[0042] 2. Compare the experimental results; To verify the respective contributions of CLSTM and ResNet backbone networks, this invention designed multiple sets of control experiments, and the results are shown in Table 8: Table 8 Comparison of evaluation metrics for the models on the test set

[0043] Ablation experiments show that CLSTM-ResNet achieves accuracy, precision, recall and F1 score of 91.22%, 91.37%, 91.32% and 91.29% respectively, demonstrating excellent noise robustness and classification stability, and verifying the effectiveness of the dual-modal fusion architecture.

[0044] 3. Modal weighting analysis; Figure 4 This demonstrates the waveform mode weights on the test set corresponding to the optimal results of CLSTM-ResNet. The kernel density distribution shows that the model can adaptively adjust the contribution ratio of different modes according to the characteristics of the input samples.

[0045] Practical application examples The results of processing OBS data from station J43AJ from November 2011 to February 2012 indicate that the vocalization activity of blue whales in the station's area mainly occurs between 00:00 and 06:00. The relevant statistical conclusions need to be further verified and expanded with a larger sample size.

[0046] Figure 5 A practical application example is shown, demonstrating that the method of the present invention can effectively detect and identify whale whistle signals in real OBS recordings.

[0047] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A deep learning-based OBS method for detecting and recognizing whale whistle signals, characterized in that, Includes the following steps: S1. Preprocess the raw seismic records, including removing instrument response, detrending, filtering, and converting file formats; S2. Initial screening of periodic signals based on STA / LTA-CV; S3. Organize and pair the suspected target signal segments obtained in S2 according to the dual-channel input format of the original time-series waveform data and its corresponding time-frequency image to construct a batch inference dataset. S4. Input the batch inference dataset obtained in S3 into the pre-trained CLSTM-ResNet dual-branch deep learning model with fused attention mechanism to perform batch automatic recognition and classification on each signal segment. S5. Based on the classification head network, the classification labels and corresponding confidence scores output by the model are summarized and statistically analyzed to generate complete signal recognition and classification results.

2. The OBS whale whistle signal detection and recognition method based on deep learning according to claim 1, characterized in that, The specific details of S2 are as follows: S21. Set the parameter values ​​to be used in the initial signal screening stage; S22. Calculate the short-time average energy within the sliding window. With long-term average energy The ratio of the characteristic function is obtained. The expression is as follows: ； ； ； in, For short time window length, For the length of the long window, The average energy over a short time window. The average energy over a long window. For the instantaneous energy of the signal, The ratio of characteristic functions; S23, when A short-term event is determined to be triggered when the value is greater than the trigger threshold and the minimum interval condition is met. The event ends when the value is below the release threshold. S24. Taking the first short-term event detected by STA / LTA as the starting point, and continuously detecting m short-term events within a fixed window length, it is determined that the sample fragment contains a periodic signal caused by biological activity. Calculate... CV The value, expressed as follows: ； in, This represents the number of events detected under the corresponding window length. Represents standard deviation, Represents the mean. The time interval between adjacent short-term events. This represents the average value of the intervals within the current window. S25. Calculate the standard deviation and mean of the event intervals of the continuous short-term events detected under this window length to obtain the CV value; if the CV value of the segment is lower than the preset threshold, the sample segment has a high degree of determinism in the time dimension, rather than random pulse interference caused by turbulence, biological collisions, or occasional activities on the seabed, and it is extracted as a sample to be classified and identified; when the interval between two adjacent short-term events exceeds the preset maximum period, it is determined that the current whale vocalization event has ended, and the detection is restarted in the next window.

3. The OBS whale whistle signal detection and recognition method based on deep learning according to claim 2, characterized in that, S4 specifically includes: S41. Construct a dual-branch deep learning model, including a waveform branch and a time-frequency graph branch; S42. Perform cross-modal fusion of waveform features and time-frequency plot features using a Transformer encoder; S43. Introduce a dynamic weighting module to adaptively modally weight the fused features and generate the final fused feature vector.

4. The OBS whale whistle signal detection and recognition method based on deep learning according to claim 3, characterized in that, The specific content of S41 is as follows: The waveform branch uses a long short-term memory network CLSTM with embedded one-dimensional convolutional modules to capture long-term temporal dependencies, while extracting local features of the waveform. The time-frequency graph branch uses a deep residual network ResNet50 with fused convolutional block attention module CBAM. It utilizes the residual learning framework to alleviate the gradient vanishing problem in deep network training and adaptively focuses on the voiceprint region in the time-frequency graph by sequentially weighting the channel attention submodule and spatial attention submodule in CBAM.

5. The OBS whale whistle signal detection and recognition method based on deep learning according to claim 4, characterized in that, The specific details of adaptive focusing on the acoustic pattern region in the time-frequency diagram in S412 through sequential weighting of the channel attention submodule and spatial attention submodule in CBAM are as follows: S411. Perform global average pooling and global max pooling on the input feature map F in the spatial dimensions [H, W] respectively, and aggregate the spatial background information to obtain two one-dimensional channel descriptors. and The dimensions are all [B, C, 1, 1], and the expression is as follows: ； ； Will and The inputs are fed into a shared multilayer perceptron (MLP) for nonlinear feature mapping. The MLP contains a hidden layer with a channel reduction rate of r. The mapping outputs are then summed element-wise and passed through a sigmoid activation function to generate channel attention weight vectors. Its dimensions are [B, C, 1, 1], and its expression is as follows: ； The forward computation formula for MLP is as follows: ； In the formula, The dimension is [C / r, C]. The dimension is [C, C / r]; Channel attention weights Element-wise multiplication with the original input feature map F yields the channel reconstruction feature map. Its dimensions are [B, C, H, W], and its expression is as follows: ； S412. In terms of channel dimension, channel reconstruction features Average pooling and max pooling are performed separately to extract strong features and weak background of the spatial region, resulting in two two-dimensional spatial descriptor maps. and Their dimensions are all [B, 1, H, W], and their expressions are as follows: ； ； Will and The features are concatenated along the channel dimension to obtain joint features of dimension [B, 2, H, W]. A receptive field convolutional layer is used to compress the number of channels to 1, followed by a sigmoid activation function to generate a spatial attention weight map. Its dimensions are [B, 1, H, W], and its expression is as follows: ； Spatial attention weights With channel reconstruction features Perform element-wise multiplication to output the final reconstructed feature map. Its dimensions are [B, C, H, W], and its expression is as follows: 。 6. The OBS whale whistle signal detection and recognition method based on deep learning according to claim 3, characterized in that, The specific details of S42 are as follows: set up and These are the input waveform features and image features, respectively. Given the self-attention weight matrix, the fusion expression is as follows: ； Right now: ; The fusion module then outputs the following expression: ； in, , , Train the parameter matrix for the Transformer. Generate a query vector based on the dimensions of the projected vector. key vector AND value vector ; After self-attention interaction, the output features complete cross-modal interaction, generating enhanced features containing global information. To integrate new waveform features from image modal information, To create new image features by fusing waveform modal information; Introducing a dynamic weight module to enhance the output of the Transformer and Perform adaptive modal fusion; The output modal features are then subjected to linear projection mapping to obtain scalar importance scores, as shown in the following expression: ； in, For the learned linear projection matrix, For the corresponding modal index, For bias terms; go through Function normalization yields the dynamic fusion weights, expressed as follows: ； Dynamic fusion weights must meet the following constraints. .

7. The OBS whale whistle signal detection and recognition method based on deep learning according to claim 3, characterized in that, The specific content of S43 is as follows: The model adaptively allocates the contribution ratio of different modalities based on the characteristics of the input samples, and forms a fused feature vector by weighted summation, as shown in the following expression: 。 8. The OBS whale whistle signal detection and recognition method based on deep learning according to claim 7, characterized in that, In S5, during the model inference phase, the system first processes the unnormalized log probability output by the model. Logits conduct Softmax Normalization is performed to calculate the predicted probability distribution of a sample belonging to each category, as shown in the following expression: ； The maximum probability is the prediction confidence of the sample, and the corresponding category is the prediction label.

9. The OBS whale whistle signal detection and recognition method based on deep learning according to claim 8, characterized in that, The dataset contains 7504 samples from blue whales, fin whales, mixed fin whales, and whales without noise. The ratio of training set, validation set, and test set is 7:1.5:1.

5. During the training phase, data augmentation strategies such as waveform random noise, time shifting, masking, image color jittering, and random erasure are adopted.

10. The OBS whale whistle signal detection and recognition method based on deep learning according to claim 9, characterized in that, The system based on the above recognition method includes: a preprocessing module, a STA / LTA-CV periodic detection module, a dual-channel data construction module, a CLSTM-ResNet dual-branch deep learning model module, a feature fusion and dynamic weight module, and a classification output module.

Images