A spatial, temporal and frequency-shifted method for detecting overlapping sound events of marine cetaceans
By using the spatial time-frequency shift network module and the asymmetric residual convolution module in the detection of overlapping sound events in marine cetaceans, combined with the bidirectional LSTM model, the problem of low sound event detection accuracy in complex acoustic environments is solved, and a higher Detection accuracy and robustness.
Patent Information
- Application Number
- CN202411026344.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-07-30
AI Technical Summary
The prior art is difficult to accurately distinguish and classify multiple synchronous sound sources in complex acoustic environments, resulting in low detection accuracy of overlapping sound events and limited model performance due to sparse labeling data.
The network module adopts spatial time-frequency shift, and redistributes the time-frequency distribution through discrete short-time Fourier transform and logarithmic Mel filter group, combines the asymmetric residual convolution module and the bidirectional LSTM model to perform time domain and frequency domain. Feature extraction and enhance the deep feature map through spatial time-frequency shift to reduce overlapping feature interference from different targets.
It significantly improves the accuracy and robustness of overlapping sound event detection, and can more accurately separate and identify different sound events in complex acoustic environments, improving detection performance.
Smart Images

Figure CN118553250B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of ocean acoustics, and in particular relates to a method for detecting overlapping sound events of ocean cetaceans with spatial time-frequency shift. Background Art
[0002] Overlapping sound event detection (SED) is of great significance in improving the accuracy and robustness of sound event detection in complex environments, and is widely used in smart monitoring, smart home, voice assistants, and medical health monitoring. Especially in cetacean ecological protection, scientific research, and environmental monitoring, SED technology monitors cetacean population dynamics, reveals their behavior patterns, assesses the health of the marine environment, and quantifies the impact of noise pollution on cetaceans by accurately detecting and analyzing overlapping underwater sound events. This not only helps to protect endangered species and formulate effective protection measures, but also promotes the development of acoustic ecology and deep learning technology. However, due to high background noise and complex and changeable acoustic environments. It is difficult to distinguish and accurately classify multiple synchronous sound sources, and it is difficult to achieve effective overlapping sound event detection.
[0003] In order to achieve overlapping sound event detection, many deep learning algorithms have been proposed. For example, CRNN is a powerful model that combines convolutional layers for feature extraction and recursive layers for temporal modeling. These networks handle overlapping sound events by capturing spatial and temporal features. In recent years, some studies have proposed inserting holes between convolution kernels to expand the receptive field of the convolution kernel, that is, using the dilated convolution method. This method allows the model to capture the features of the input data in a larger range, so that it can better understand and model long-term dependencies in time series. In addition, there are studies that use multi-scale convolution kernels to capture the fine-grained and coarse-grained features of sound events, and combine ensemble learning methods to effectively reduce frame-level prediction errors. However, these methods focus more on the overall time-frequency features of the audio, and do not perform deep feature extraction on the time domain and frequency domain signals of the features respectively. Therefore, the features output by the convolution module are not sufficient, which affects the detection performance of the model.
[0004] In addition, a major challenge of the overlapping sound event detection (SED) task is the scarcity of available annotated datasets. Due to the complexity and diversity of overlapping sound events in practical applications, it is costly and time-consuming to obtain high-quality, large-scale annotated datasets. In order to overcome the problem of data scarcity, data augmentation technology has become an important means to improve model performance. Summary of the invention
[0005] To solve the above-mentioned problems, the present invention provides a method for detecting overlapping sound events of marine cetaceans with spatial time-frequency shift. The method combines a network module with spatial time-frequency shift, enhances deep time-frequency features, and captures more contextual information under time and frequency shift. In the scenario where multiple target underwater sound sources overlap, the method of the present invention can reduce the mutual interference of overlapping features of different targets and effectively improve the detection accuracy of overlapping sound events.
[0006] The present application aims to address the shortcomings of overlapping sound event detection technology and proposes a spatial time-frequency shifted method for detecting overlapping sound events of marine cetaceans, which is used to detect cetacean audio signals with overlapping sound events and can effectively improve the detection accuracy of overlapping sound events in scenarios with overlapping multi-target underwater sound sources.
[0007] To achieve the above objectives, the present application provides the following solution: a method for detecting overlapping sound events of marine cetaceans with spatial time-frequency shift. It includes:
[0008] After obtaining and preprocessing the underwater acoustic signal, the time-frequency distribution of the preprocessed underwater acoustic signal is obtained by discrete short-time Fourier transform, and the time-frequency distribution is redistributed in combination with the logarithmic Mel filter bank;
[0009] Construct an asymmetric residual convolution module, input the redistributed time-frequency feature map, and extract the deep time-frequency feature map;
[0010] A network module for spatial time-frequency shift is proposed to perform time-domain spatial shift and frequency-domain spatial shift expansion and enhancement on the deep time-frequency feature map, and then fuse the enhanced deep feature map.
[0011] The enhanced deep feature map is input into the bidirectional LSTM model, and the fully connected layer and sigmoid layer are used to obtain the final output deep feature map.
[0012] Based on the final output deep feature map, the overlapping sound event category of each frame is predicted, and the score, F1-score and error rate of the detection results are estimated.
[0013] Preferably, the process of obtaining the time-frequency distribution of the pre-processed underwater acoustic signal by discrete short-time Fourier transform and reallocating the time-frequency distribution by combining the logarithmic Mel filter bank includes:
[0014] Audio signals for cetaceans Perform discrete short-time Fourier transform (STFT) to obtain the whale audio signal The time-frequency distribution of :
[0015]
[0016] in is the window function, It's a signal In time and frequency The result of the discrete short-time Fourier transform is is the length of the window function.
[0017] After obtaining the time-frequency distribution, the time-frequency distribution is redistributed through the logarithmic Mel filter bank. The Mel frequency scale approximates the human ear's perception of sound and is defined as:
[0018]
[0019] The filter bank is designed on the Mel frequency scale. The response function of each filter is triangular in shape with a center frequency of , the output of the filter, the log Mel energy The Mel filter bank is defined as: , Represents the index of the filter:
[0020]
[0021] Logarithmic Mel energy of the Mel filter output :
[0022]
[0023] in, represents the index of the timeframe, represents the index of the Mel filter bank, is the index of the discrete short-time Fourier transform;
[0024] Based on the reallocated time-frequency feature map Get the input feature vector , where 1 is the number of channels, is the number of time frames, is the number of Mel filter banks.
[0025] Preferably, the asymmetric residual convolution module comprises:
[0026] Input layer, used to receive input signals;
[0027] The first convolutional layer contains a The convolution kernel and a The convolution kernels are connected to the input layer respectively;
[0028] A first cascade layer, cascading the outputs of the two convolution branches of the first convolution layer;
[0029] The second convolutional layer contains a The convolution kernel is connected to the output of the first cascade layer;
[0030] A batch normalization layer, used for performing batch normalization processing on the output of the second convolutional layer;
[0031] The third convolutional layer contains a The convolution kernel is connected to the input layer;
[0032] A second cascade layer, cascading the output of the batch normalization layer and the output of the third convolutional layer;
[0033] The output layer is connected to the second cascade layer and outputs the processed signal.
[0034] The process of inputting the reallocated time-frequency feature map into the asymmetric residual convolution module of the sound event detection model for feature extraction to obtain the deep time-frequency feature map includes:
[0035] The time-frequency feature map to be redistributed The asymmetric residual convolution module of the input sound event detection model, the first convolution layer of the asymmetric residual convolution module Convolution kernel and The output of the convolution kernel is cascaded at the first cascade layer and then input to the second convolution layer;
[0036] The second cascade layer cascades the output of the batch normalization layer with the output of the third convolutional layer to obtain a final output signal; based on the final output signal, the deep time-frequency feature map is obtained. ,in is the number of output channels of the third convolutional layer, is the number of time frames, is the number of Mel filter banks.
[0037] Preferably, the network module of the space-time-frequency shift includes:
[0038] Given an input deep time-frequency feature map ,in is the number of channels, is the number of time frames, is the number of Mel filter banks. Along the channel, the input feature map is split into an even number of channels, where is the number of channel divisions, we get Group feature map, then the number of channels contained in each partition for:
[0039]
[0040] The feature maps after each group division are:
[0041]
[0042] For each set of feature maps, perform spatial left shift and right shift in the time domain and frequency domain respectively. Concatenate the shifted feature maps of different branches to obtain the global feature map:
[0043]
[0044] Preferably, the network module that proposes spatial time-frequency shift performs time-domain spatial shift and frequency-domain spatial shift expansion and enhancement on the deep time-frequency feature map, and the process of fusing the enhanced deep feature map includes:
[0045] Channel expansion convolution layer, receiving deep time-frequency feature map And perform channel expansion to obtain the extended feature vector ;
[0046] The feature vector is divided into three-channel feature vectors, and the extended feature vector Divide into eigenvectors on the channels , and ;in, , and ;
[0047] The time shift module, the feature vector Perform spatial shift on the time scale and generate a spatial shift vector ;
[0048] The frequency shift module is used to shift the feature vector Perform spatial shift on the frequency scale to generate a spatial shift vector ;
[0049] The feature vector No spatial shift is performed, directly mapped to spatial vector ;
[0050] The global average pooling layer is used to transform the spatial displacement vector , space vector , and the spatial displacement vector Pooling is performed to obtain the global information vectors of the three branches respectively. , and ;
[0051] The concatenation layer is used to concatenate the global information vector , and Along the three-branch channel, vector splicing is performed;
[0052] Linear layer, which performs linear transformation on the concatenated vectors to generate weight vectors ;
[0053] The softmax layer processes the weights after linear transformation to obtain the weight vector optimized by the softmax layer;
[0054] Split weight layer, split the optimized weight vector into metric space shift vector The weight vector of the three branches of importance , and ;
[0055] The output layer converts the weight vector and the spatial displacement vector Multiply them together to get the deep feature map enhanced by the spatial shift module :
[0056]
[0057] Preferably, the overlapping sound event detection process comprises:
[0058] The deep feature map enhanced by the spatial shift module , input the bidirectional LSTM network to obtain the feature vector of the time series ;
[0059] The feature vector of the time series Input the fully connected layer and the sigmoid layer to obtain the sound event detection results of each frame of the underwater audio signal ;
[0060] Will With the label data in the dataset By performing frame-level comparison, we can obtain the TP, TN, FP, and FN indicators of the model. Based on these indicators, we can calculate evaluation indicators such as accuracy, F1-score, and error rate.
[0061] Compared with the prior art, the present invention has the following advantages and technical effects:
[0062] (1) This application uses an asymmetric residual convolution module for feature extraction. Through parallel convolution layers with different convolution kernel sizes, time domain and frequency domain features are extracted separately and then merged. This method combines the time domain and frequency domain features of the signal, extracts more robust features related to sound events, and significantly improves detection accuracy and noise resistance.
[0063] (2) The spatial shift module of the present application increases the diversity of data and the richness of features by splitting the input features into multiple sub-channel groups along the channel dimension and performing left and right shift operations on these sub-channel groups in the time domain and frequency domain. This module effectively enhances the model's ability to detect overlapping sound events and improves the robustness and detection accuracy in complex acoustic environments. Through this spatial shift operation, the model can more accurately separate and identify different sound events, thereby improving overall performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0064] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0065] Figure 1 A schematic diagram of a method flow of an embodiment of the present application;
[0066] Figure 2 The logarithmic Mel energy map of the overlapping sound event signal of cetaceans according to the embodiment of the present application;
[0067] Figure 3 This is a structural diagram of a whale overlapping sound event detection model according to an embodiment of the present application;
[0068] Figure 4 This is a structural diagram of a residual convolution mixing module according to an embodiment of the present application;
[0069] Figure 5 This is a structural diagram of a spatial shift module according to an embodiment of the present application;
[0070] Figure 6 A schematic diagram of a spatial shift operation according to an embodiment of the present application;
[0071] Figure 7 This is a diagram of the detection results of overlapping cetacean sound events according to an embodiment of the present application. DETAILED DESCRIPTION
[0072] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0073] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0074] like Figure 1 As shown, this embodiment provides a method for detecting overlapping sound events of marine cetaceans with spatial time-frequency shift, comprising:
[0075] S1. After obtaining and preprocessing the underwater acoustic signal, the time-frequency distribution of the preprocessed underwater acoustic signal is obtained by discrete short-time Fourier transform, and the time-frequency distribution is reallocated in combination with the logarithmic Mel filter bank; the time-frequency characteristic diagram of the reallocated whale signal in this embodiment is as follows Figure 2 shown.
[0076] In this embodiment, step S1 includes the following process:
[0077] S1 first detects cetacean audio signals Perform discrete short-time Fourier transform (STFT) to obtain the whale audio signal The time-frequency distribution of :
[0078]
[0079] in is the window function, It's a signal In time and frequency The result of the discrete short-time Fourier transform is is the length of the window function.
[0080] After obtaining the time-frequency distribution, the time-frequency distribution is redistributed through the logarithmic Mel filter bank. The Mel frequency scale approximates the human ear's perception of sound and is defined as:
[0081]
[0082] The filter bank is designed on the Mel frequency scale. The response function of each filter is triangular in shape with a center frequency of , the output of the filter, the log Mel energy The Mel filter bank is defined as: , Represents the index of the filter:
[0083]
[0084] Logarithmic Mel energy of the Mel filter output :
[0085]
[0086] in, represents the index of the timeframe, represents the index of the Mel filter bank, is the index of the discrete short-time Fourier transform
[0087] Based on the reallocated time-frequency feature map Get the input feature vector , where 1 is the number of channels, is the number of time frames, is the number of Mel filter banks.
[0088] S2. Input the redistributed time-frequency feature map of the cetacean signal into the asymmetric residual convolution module of the sound event detection model for feature extraction to obtain a deep time-frequency feature map;
[0089] In this embodiment, if Figure 4 As shown, the asymmetric residual convolution module includes:
[0090] Input layer, used to receive input signals;
[0091] The first convolutional layer contains a The convolution kernel and a The convolution kernels are connected to the input layer respectively;
[0092] A first cascade layer, cascading the outputs of the two convolution branches of the first convolution layer;
[0093] The second convolutional layer contains a The convolution kernel is connected to the output of the first cascade layer;
[0094] A batch normalization layer, used for performing batch normalization processing on the output of the second convolutional layer;
[0095] The third convolutional layer contains a The convolution kernel is connected to the input layer;
[0096] A second cascade layer, cascading the output of the batch normalization layer and the output of the third convolutional layer;
[0097] The output layer is connected to the second cascade layer and outputs the processed signal.
[0098] Furthermore, the reallocated time-frequency feature map is input into the asymmetric residual convolution module of the sound event detection model for feature extraction, and the process of obtaining the deep time-frequency feature map includes:
[0099] The redistributed time-frequency feature map in S1 The asymmetric residual convolution module of the input sound event detection model, the first convolution layer of the asymmetric residual convolution module Convolution kernel and The output of the convolution kernel is cascaded at the first cascade layer and then input to the second convolution layer;
[0100] The second cascade layer cascades the output of the batch normalization layer with the output of the third convolutional layer to obtain a final output signal; based on the final output signal, the deep time-frequency feature map is obtained. ,in is the number of output channels of the third convolutional layer, is the number of time frames, is the number of Mel filter banks.
[0101] S3. The deep time-frequency feature map is respectively extended and enhanced by time-domain spatial shift and frequency-domain spatial shift, and the enhanced deep feature map is obtained by fusion.
[0102] In this embodiment, if Figure 6 As shown, the network module of space-time-frequency shift includes:
[0103] Given an input deep time-frequency feature map ,in is the number of channels, is the number of time frames, is the number of Mel filter banks. Along the channel, the input feature map is split into an even number of channels, where is the number of channel divisions, we get Group feature map, then the number of channels contained in each partition for:
[0104]
[0105] The feature maps after each group division are:
[0106]
[0107] For each set of feature maps, spatial left shift and right shift are performed in the time domain and frequency domain respectively:
[0108] Taking the time domain shift as an example, Group channels to shift left:
[0109]
[0110] For Shift the group right:
[0111]
[0112] in .
[0113] The shifted feature maps of different branches are concatenated to obtain the global feature map:
[0114]
[0115] Further optimization scheme, in this embodiment, If it is 4, the number of channels in each group is for:
[0116]
[0117] The input features are divided into Groups, respectively :
[0118]
[0119] The feature maps after each two groups of division are shifted left and right respectively. Taking the spatial shift in the time domain as an example, , To shift left:
[0120]
[0121]
[0122] right , To shift right:
[0123]
[0124]
[0125] The shifted feature maps of different branches are concatenated to obtain the global feature map:
[0126]
[0127] Next, perform the same spatial shift operation on the frequency scale, and finally concatenate it along the channel into a spatial shift vector .
[0128] In this embodiment, if Figure 5 As shown in the figure, the network module of spatial time-frequency shift performs time-domain spatial shift and frequency-domain spatial shift expansion and enhancement on the deep time-frequency feature map, and fuses the enhanced deep feature map:
[0129] Channel expansion convolution layer, receiving deep time-frequency feature map And perform channel expansion to obtain the extended feature vector ;
[0130] The feature vector is divided into three-channel feature vectors, and the extended feature vector Divide into eigenvectors on the channels , and ;in, , and ;
[0131] The time shift module, the feature vector Perform spatial shift on the time scale and generate a spatial shift vector ;
[0132] The frequency shift module is used to shift the feature vector Perform spatial shift on the frequency scale to generate a spatial shift vector ;
[0133] The feature vector No spatial shift is performed, directly mapped to spatial vector ;
[0134] The global average pooling layer is used to transform the spatial displacement vector , space vector , and the spatial displacement vector Pooling is performed to obtain the global information vectors of the three branches respectively. , and ;
[0135] The concatenation layer is used to concatenate the global information vector , and Along the three-branch channel, vector splicing is performed;
[0136] Linear layer, which performs linear transformation on the concatenated vectors to generate weight vectors ;
[0137] The softmax layer processes the weights after linear transformation to obtain the weight vector optimized by the softmax layer;
[0138] Split weight layer, split the optimized weight vector into metric space shift vector The weight vector of the three branches of importance , and ;
[0139] The output layer converts the weight vector and the spatial displacement vector Multiply them together to get the deep feature map enhanced by the spatial shift module :
[0140]
[0141] S4. The overlapping sound event detection process includes:
[0142] The deep feature map enhanced by the spatial shift module , input the bidirectional LSTM network to obtain the feature vector of the time series ;
[0143] The feature vector of the time series Input the fully connected layer and the sigmoid layer to obtain the sound event detection results of each frame of the underwater audio signal ;
[0144] Will With the label data in the dataset By comparing the frames, we can get the TP, TN, FP, and FN indicators of the model. Based on these indicators, we can calculate the evaluation indicators such as accuracy, F1-score, and error rate. After testing, the model of this application can achieve an F1-score of 0.932 and an error rate of 0.09. The PSDS evaluation score is 0.903.
[0145] At this point, overlapping acoustic event signals of North Atlantic right whales Through Figure 3 The sound event detection network shown in the figure obtains the results about the probability of different categories of sound events in each frame signal. ,like Figure 7 As shown in the figure, the horizontal axis represents the time duration of the audio signal, and the vertical axis marks the possible sound event categories. The results of model recognition are represented by red bars, while the start and end of the actual sound events in the audio are marked by blue bars.
[0146] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A spatial, temporal and frequency-shifted method for detecting overlapping sound events of marine cetaceans. It is characterized in that include: After obtaining and preprocessing the underwater acoustic signal, the time-frequency distribution of the preprocessed underwater acoustic signal is obtained by discrete short-time Fourier transform, and the time-frequency distribution is redistributed in combination with the logarithmic Mel filter bank; Construct an asymmetric residual convolution module, input the redistributed time-frequency feature map, and extract the deep time-frequency feature map; A network module for spatial time-frequency shift is proposed to perform time-domain spatial shift and frequency-domain spatial shift expansion and enhancement on the deep time-frequency feature map, and then fuse the enhanced deep feature map. The enhanced deep feature map is input into the bidirectional LSTM model, and the fully connected layer and sigmoid layer are used to obtain the final output deep feature map; Based on the final output deep feature map, predict the overlapping sound event category of each frame and estimate the score, F1-score and error rate of the detection results; The network module that proposes spatial time-frequency shift performs time-domain spatial shift and frequency-domain spatial shift expansion and enhancement on the deep time-frequency feature map, and the process of fusing the enhanced deep feature map includes: Channel expansion convolution layer, receiving deep time-frequency feature map And perform channel expansion to obtain the extended feature vector ; The feature vector is divided into three-channel feature vectors, and the extended feature vector Divide into eigenvectors on the channels , and ;in, , and ; The time shift module, the feature vector Perform spatial shift on the time scale and generate a spatial shift vector ; The frequency shift module is used to shift the feature vector Perform spatial shift on the frequency scale to generate a spatial shift vector ; The feature vector No spatial shift is performed, directly mapped to spatial vector ; The global average pooling layer is used to transform the spatial displacement vector , space vector , and the spatial displacement vector Pooling is performed to obtain the global information vectors of the three branches respectively. , and ; The concatenation layer is used to concatenate the global information vector , and Along the three-branch channel, vector splicing is performed; Linear layer, which performs linear transformation on the concatenated vectors to generate weight vectors ; The softmax layer processes the weights after linear transformation to obtain the weight vector optimized by the softmax layer; Split weight layer, split the optimized weight vector into metric space shift vector The weight vector of the three branches of importance , and ; The output layer converts the weight vector and the spatial displacement vector Multiply them together to get the deep feature map enhanced by the spatial shift module : ; The network module of the space-time-frequency shift includes: Given an input deep time-frequency feature map ,in is the number of channels, is the number of time frames, is the number of Mel filter groups; along the channel, the input feature map is divided into an even number of channels, where is the number of channel divisions, we get Group feature map, then the number of channels contained in each partition for: The feature maps after each group division are: For each set of feature maps, spatial left shift and right shift are performed in the time domain and frequency domain respectively, and the shifted feature maps of different branches are concatenated to obtain the global feature map: 。 2. The method for detecting overlapping sound events of marine cetaceans using spatial time-frequency shift according to claim 1, It is characterized in that The time-frequency distribution of the pre-processed underwater acoustic signal is obtained by discrete short-time Fourier transform, and the process of reallocating the time-frequency distribution in combination with the logarithmic Mel filter bank includes: Audio signals for cetaceans Perform discrete short-time Fourier transform to obtain the whale audio signal The time-frequency distribution of : in is the window function, It's a signal In time and frequency The result of the discrete short-time Fourier transform is is the length of the window function; After obtaining the time-frequency distribution, the time-frequency distribution is redistributed through the logarithmic Mel filter bank. The Mel frequency scale approximates the human ear's perception of sound and is defined as: The filter bank is designed on the Mel frequency scale. The response function of each filter is triangular in shape with a center frequency of , the output of the filter, the log Mel energy The Mel filter bank is defined as: , Represents the index of the filter: Logarithmic Mel energy of the Mel filter output : in, represents the index of the timeframe, represents the index of the Mel filter bank, is the index of the discrete short-time Fourier transform; Based on the reallocated time-frequency feature map Get the input feature vector , where 1 is the number of channels, is the number of time frames, is the number of Mel filter banks.
3. The method for detecting overlapping sound events of marine cetaceans using spatial time-frequency shift according to claim 1, It is characterized in that The process of constructing an asymmetric residual convolution module, inputting the reallocated time-frequency feature map, and extracting the deep time-frequency feature map includes: The structure of the constructed asymmetric residual convolution module is as follows: Input layer, used to receive input signals; The first convolutional layer contains a The convolution kernel and a The convolution kernels are connected to the input layer respectively; A first cascade layer, cascading the outputs of the two convolution branches of the first convolution layer; The second convolutional layer contains a The convolution kernel is connected to the output of the first cascade layer; A batch normalization layer, used for performing batch normalization processing on the output of the second convolutional layer; The third convolutional layer contains a The convolution kernel is connected to the input layer; A second cascade layer, cascading the output of the batch normalization layer and the output of the third convolutional layer; an output layer, connected to the second cascade layer, outputting the processed signal; The process of inputting the reallocated time-frequency feature map into the asymmetric residual convolution module of the sound event detection model for feature extraction to obtain the deep time-frequency feature map includes: The time-frequency feature map to be redistributed The asymmetric residual convolution module of the input sound event detection model, the first convolution layer of the asymmetric residual convolution module Convolution kernel and The output of the convolution kernel is cascaded at the first cascade layer and then input to the second convolution layer; The second cascade layer cascades the output of the batch normalization layer with the output of the third convolutional layer to obtain a final output signal; based on the final output signal, the deep time-frequency feature map is obtained. ,in is the number of output channels of the third convolutional layer, is the number of time frames, is the number of Mel filter banks.
4. The method for detecting overlapping sound events of marine cetaceans using spatial time-frequency shift according to claim 1, It is characterized in that The overlapping sound event detection process includes: The deep feature map enhanced by the spatial shift module , input the bidirectional LSTM network to obtain the feature vector of the time series ; The feature vector of the time series Input the fully connected layer and the sigmoid layer to obtain the sound event detection results of each frame of the underwater audio signal ; Will With the label data in the dataset By performing frame-level comparison, we can obtain the TP, TN, FP, and FN indicators of the model, based on which we can calculate the accuracy, F1-score, and error rate evaluation indicators.
