A method and system for cetacean overlapping sound event detection based on adaptive multi-scale synthetic attention

Abstract

The application discloses a kind of based on adaptive multi-scale synthetic attention whale overlapping sound event detection method and system, belong to ocean engineering and marine signal technical field, method includes: collection historical whale sound signal, pre-process and ACT transform historical whale sound signal to obtain transformed data set;Cross-shift feature extraction network is constructed to time-frequency perception, and cross-shift feature extraction network is trained using transformed data to time-frequency perception, to obtain adaptive multi-scale synthetic attention network;Adaptive window size prediction network is constructed, and according to the feature map output by adaptive multi-scale synthetic attention network, the window parameters required for constructing multi-scale attention fusion network are predicted;Multi-scale attention fusion network is constructed based on window parameter and is iterated, to obtain whale overlapping sound event detection model;Real-time whale sound signal is obtained, and real-time whale sound signal is detected using detection model, to obtain detection result.

Description

Technical Field

[0001] This invention belongs to the fields of marine engineering and marine signal technology, specifically relating to a method and system for detecting overlapping sound events in cetaceans based on adaptive multi-scale synthetic attention. Background Technology

[0002] Overlapping sound event detection is crucial for intelligent marine ecological monitoring, enabling autonomous identification and precise localization of sound sources from marine organisms such as whales and dolphins. However, the marine environment is characterized by complex sound source types, with different organisms often vocalizing simultaneously, accompanied by strong background noise. This results in significant time-frequency overlap, non-stationarity, and energy differences in the received signals. Furthermore, marine biological sounds exhibit species dependence, with subtle frequency variations often corresponding to different semantic or behavioral states. Convolution, while translation-invariant, struggles to capture minute frequency differences. Additionally, the durations of different events vary significantly, making it difficult for fixed-scale contexts to adapt to diverse temporal characteristics.

[0003] To enhance the frequency structure perception capability of models, researchers have proposed methods such as frequency dynamic convolution, frequency pyramid modules, and two-dimensional separable convolution to overcome the translation invariance assumption and improve detection performance to some extent. However, existing methods still have many limitations. First, they are insufficient in modeling the semantic differences implied by minute frequency changes, making it difficult to accurately distinguish subtle behavioral states of organisms. Second, uniform-scale context modeling is difficult to adapt to sound events of varying durations, affecting the complete representation of events of different lengths. In addition, there is a lack of explicit modeling mechanisms for overlapping relationships between multiple events, especially at low signal-to-noise ratios, resulting in insufficient detection robustness. Therefore, there is an urgent need to construct a collaborative detection framework that integrates frequency and behavior perception, multi-scale context understanding, and structural interpretability to more effectively improve detection accuracy and generalization ability in complex marine environments. Summary of the Invention

[0004] This invention aims to address the shortcomings of existing technologies and provides the following solutions:

[0005] A method for detecting overlapping sound events in cetaceans based on adaptive multi-scale synthetic attention includes the following steps:

[0006] Historical whale sound signals were collected, preprocessed, and transformed using ACT to obtain the transformed dataset.

[0007] A time-frequency aware cross-offset feature extraction network is constructed, and the transformed data is used to train the time-frequency aware cross-offset feature extraction network to obtain an adaptive multi-scale synthetic attention network.

[0008] Construct an adaptive window size prediction network, and predict the window parameters required to construct the multi-scale attention fusion network based on the feature map output by the adaptive multi-scale synthetic attention network.

[0009] The multi-scale attention fusion network is constructed based on the window parameters and iterated to obtain a cetacean overlapping sound event detection model.

[0010] Real-time whale sound signals are acquired, and the whale overlapping sound event detection model is used to detect the real-time whale sound signals to obtain detection results.

[0011] Preferably, the method for obtaining the transformed dataset includes:

[0012] Historical whale sound signals are collected, and the historical whale sound signals are processed by non-overlapping equal-length frame cropping to obtain the cropped signal;

[0013] The foreground signal and background sound in the cropped signal are synthesized using a soundscape synthesizer library to obtain a synthesized signal. The ACT features of the synthesized signal are then extracted and normalized to obtain the transformed dataset.

[0014] ,

[0015] Where, x N These represent different ACT features.

[0016] Preferred methods for obtaining adaptive multi-scale synthetic attention networks include:

[0017] A time-frequency aware cross-offset feature extraction network is constructed, which includes: a first TFER convolutional unit, a channel expansion unit, a frequency domain dynamic convolutional unit, a time domain dynamic convolutional unit, a first cross-shift unit, a second cross-shift unit, a first weight fusion unit, and a second TFER convolutional unit.

[0018] The transformed data is divided into a training set, a validation set, and a test set according to a preset ratio;

[0019] The time-frequency aware cross-offset feature extraction network is trained using the training set, and the trained network is validated using the validation set to obtain an adaptive multi-scale synthetic attention network.

[0020] Preferably, the method for obtaining the window parameters includes:

[0021] Construct an adaptive window size prediction network, and convolve the feature map output by the adaptive multi-scale synthetic attention network with the Sobel operator to obtain the gradient map vector:

[0022] ,

[0023] in, This represents the gradient map obtained by convolving the Nth feature map with the Sobel operator.

[0024] Squaring the gradient map vector yields the edge intensity map vector:

[0025] ,

[0026] Among them, e N This represents the edge intensity map obtained after squaring the Nth gradient map vector.

[0027] Based on the edge intensity map vector, construct the global edge intensity vector, the maximum response intensity vector, and the salient edge density vector:

[0028] ,

[0029] ,

[0030] ,

[0031] ,

[0032] ,

[0033] ,

[0034] Where Ω represents the global edge intensity vector, Ψ represents the maximum response intensity vector, and Φ represents the salient edge density vector. Indicates the intensity of the Nth global edge. This represents the Nth maximum response intensity. Let n represent the density of the Nth salient edge. f n represents the frequency dimension of the edge intensity map. t Let i represent the i-th frequency dimension of the edge intensity map, j represent the j-th time dimension of the edge intensity map, I represent the threshold function, and L represent a pixel value of the edge intensity map.

[0035] Based on the global edge strength vector, the maximum response strength vector, and the salient edge density vector, construct the large window complexity vector and the small window complexity vector:

[0036] ,

[0037] ,

[0038] in, Represents the large window complexity vector. Represents the small window complexity vector. This represents the complexity parameter of the Nth large window. This represents the complexity parameter of the Nth small window;

[0039] Based on the large window complexity vector and the small window complexity vector, construct the large window vector and the small window vector, and obtain the window parameters:

[0040] ,

[0041] ,

[0042] in, Represents a large window vector. Represents the small window vector. This represents the parameters of the Nth large window. This represents the parameters of the Nth small window.

[0043] Preferably, the method for obtaining the cetacean overlapping sound event detection model includes:

[0044] The multi-scale attention fusion network is constructed based on the window parameters. The multi-scale attention fusion network includes: small-scale input unit, large-scale input unit, small-scale attention fusion unit, large-scale attention fusion unit, second weight fusion unit, GRU unit, linear layer and loss function.

[0045] The multi-scale attention fusion network is iterated, and the iterated model is tested using the test set. If the preset target is met, the whale overlapping sound event detection model is obtained.

[0046] The present invention also provides a cetacean overlapping sound event detection system based on adaptive multi-scale synthetic attention. The detection system applies the detection method described above and includes: a historical data acquisition module, a first model construction module, a window parameter acquisition module, a second model construction module, and a detection module.

[0047] The historical data acquisition module is used to acquire historical whale sound signals, preprocess the historical whale sound signals and perform ACT transformation to obtain the transformed dataset;

[0048] The first model building module is used to build a time-frequency sensing cross-offset feature extraction network, and to train the time-frequency sensing cross-offset feature extraction network using the transformed data to obtain an adaptive multi-scale synthetic attention network.

[0049] The window parameter acquisition module is used to construct an adaptive window size prediction network and predict the window parameters required to construct a multi-scale attention fusion network based on the feature map output by the adaptive multi-scale synthetic attention network.

[0050] The second model building module constructs the multi-scale attention fusion network based on the window parameters and iterates it to obtain a cetacean overlapping sound event detection model;

[0051] The detection module is used to acquire real-time whale sound signals and use the whale overlapping sound event detection model to detect the real-time whale sound signals to obtain detection results.

[0052] Preferably, the workflow of the historical data acquisition module includes:

[0053] Historical whale sound signals are collected, and the historical whale sound signals are processed by non-overlapping equal-length frame cropping to obtain the cropped signal;

[0054] The foreground signal and background sound in the cropped signal are synthesized using a soundscape synthesizer library to obtain a synthesized signal. The ACT features of the synthesized signal are then extracted and normalized to obtain the transformed dataset.

[0055] ,

[0056] Where, x N These represent different ACT features.

[0057] Preferably, the workflow of the first model building module includes:

[0058] A time-frequency aware cross-offset feature extraction network is constructed, which includes: a first TFER convolutional unit, a channel expansion unit, a frequency domain dynamic convolutional unit, a time domain dynamic convolutional unit, a first cross-shift unit, a second cross-shift unit, a first weight fusion unit, and a second TFER convolutional unit.

[0059] The transformed data is divided into a training set, a validation set, and a test set according to a preset ratio;

[0060] The time-frequency aware cross-offset feature extraction network is trained using the training set, and the trained network is validated using the validation set to obtain an adaptive multi-scale synthetic attention network.

[0061] Preferably, the workflow of the window parameter acquisition module includes:

[0062] Construct an adaptive window size prediction network, and convolve the feature map output by the adaptive multi-scale synthetic attention network with the Sobel operator to obtain the gradient map vector:

[0063] ,

[0064] in, This represents the gradient map obtained by convolving the Nth feature map with the Sobel operator.

[0065] Squaring the gradient map vector yields the edge intensity map vector:

[0066] ,

[0067] Among them, e N This represents the edge intensity map obtained after squaring the Nth gradient map vector.

[0068] Based on the edge intensity map vector, construct the global edge intensity vector, the maximum response intensity vector, and the salient edge density vector:

[0069] ,

[0070] ,

[0071] ,

[0072] ,

[0073] ,

[0074] ,

[0075] Where Ω represents the global edge intensity vector, Ψ represents the maximum response intensity vector, and Φ represents the salient edge density vector. Indicates the intensity of the Nth global edge. This represents the Nth maximum response intensity. Let n represent the density of the Nth salient edge. f n represents the frequency dimension of the edge intensity map. t Let i represent the i-th frequency dimension of the edge intensity map, j represent the j-th time dimension of the edge intensity map, I represent the threshold function, and L represent a pixel value of the edge intensity map.

[0076] Based on the global edge strength vector, the maximum response strength vector, and the salient edge density vector, construct the large window complexity vector and the small window complexity vector:

[0077] ,

[0078] ,

[0079] in, Represents the large window complexity vector. Represents the small window complexity vector. This represents the complexity parameter of the Nth large window. This represents the complexity parameter of the Nth small window;

[0080] Based on the large window complexity vector and the small window complexity vector, construct the large window vector and the small window vector, and obtain the window parameters:

[0081] ,

[0082] ,

[0083] in, Represents a large window vector. Represents the small window vector. This represents the parameters of the Nth large window. This represents the parameters of the Nth small window.

[0084] Preferably, the workflow of the second model building module includes:

[0085] The multi-scale attention fusion network is constructed based on the window parameters. The multi-scale attention fusion network includes: small-scale input unit, large-scale input unit, small-scale attention fusion unit, large-scale attention fusion unit, second weight fusion unit, GRU unit, linear layer and loss function.

[0086] The multi-scale attention fusion network is iterated, and the iterated model is tested using the test set. If the preset target is met, the whale overlapping sound event detection model is obtained.

[0087] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0088] (1) The time-frequency sensing cross-offset feature extraction network model of the present invention, combined with ACT transformation, obtains a high-resolution time-frequency map. By using the cross-offset mechanism and time-frequency dynamic convolution strategy, it can adaptively capture key feature regions in complex time-frequency structures, enhance the spatial sensitivity and time-frequency coupling expression ability of the feature map, and improve the feature separation ability of weak targets or overlapping signals.

[0089] (2) The adaptive window size prediction network model of the present invention combines the semantic information of deep feature maps to dynamically predict the optimal window size, match the feature receptive field of targets of different scales, avoid information loss or redundancy caused by fixed windows, and improve the focusing ability of multi-scale attention modules and the time-frequency structure adaptability of the network.

[0090] (3) The multi-scale attention fusion module model of the present invention integrates the deep time-frequency feature map extracted by the multi-branch network and combines the local dense synthetic attention mechanism to realize the saliency enhancement of multi-scale features and the reconstruction of complementary information, effectively improving the recognition accuracy of signals with drastic changes in time-frequency distribution and complex cross-components. Attached Figure Description

[0091] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0092] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention;

[0093] Figure 2 This is a schematic diagram of the structure of the TFER convolutional unit according to an embodiment of the present invention;

[0094] Figure 3 This is a schematic diagram of the structure of the time-frequency sensing cross-offset feature extraction network according to an embodiment of the present invention;

[0095] Figure 4 This is a schematic diagram of the structure of a multi-scale attention fusion network according to an embodiment of the present invention;

[0096] Figure 5 This is a schematic diagram of the confusion matrix of the signal classification structure in embodiment 4 of the present invention;

[0097] Figure 6 This is a schematic diagram showing the changes in PSDS scores of the six detection models in this embodiment of the invention under different signal-to-noise ratios. Detailed Implementation

[0098] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0099] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0100] Example 1:

[0101] In this embodiment, as Figure 1 As shown, a method for detecting overlapping sound events in cetaceans based on adaptive multi-scale synthetic attention includes the following steps:

[0102] S1. Collect historical whale sound signals, preprocess and perform ACT transformation on the historical whale sound signals to obtain the transformed dataset.

[0103] In this embodiment, the method for obtaining the transformed dataset includes:

[0104] Historical whale sound signals were collected, and non-overlapping, equal-length frame cropping was performed on different types of historical whale sound signals to remove invalid or low-energy segments, obtaining effective foreground signals of varying lengths, i.e., the cropped signals. A soundscape synthesizer library was used to synthesize the foreground signals and background sounds from the cropped signals to obtain the synthesized signals. The synthesized audio data was generated under four different signal-to-noise ratio (SNR) conditions: Very Low (−10dB to −5dB), Low (−5dB to 0dB), Medium (0dB to 5dB), and High (5dB to 10dB). The ACT features of the synthesized signals were then extracted and normalized to obtain the transformed dataset.

[0105] ,

[0106] Where, x N These represent different ACT features.

[0107] S2. Construct a time-frequency aware cross-offset feature extraction network, and train the time-frequency aware cross-offset feature extraction network using the transformed data to obtain an adaptive multi-scale synthetic attention network.

[0108] Methods for obtaining adaptive multi-scale synthetic attention networks include:

[0109] Construct a time-frequency sensing cross-offset feature extraction network, as follows: Figure 3As shown, it includes: a first TFER convolutional unit, a channel expansion unit, a frequency domain dynamic convolutional unit, a temporal domain dynamic convolutional unit, a first cross-shift unit, a second cross-shift unit, a first weight fusion unit, and a second TFER convolutional unit. The transformed data is divided into a training set, a validation set, and a test set according to a preset ratio. In this embodiment, the preset ratio is 7:1.5:1.5. The time-frequency aware cross-shift feature extraction network is trained using the training set, and the trained network is validated using the validation set to obtain an adaptive multi-scale synthetic attention network. The TFER convolutional unit is as follows... Figure 2 As shown.

[0110] In this embodiment, the workflow of the adaptive multi-scale synthetic attention network includes: extracting deep features of all ACT features X from the training set data through the first TFER convolutional unit, and outputting a deep feature map as follows:

[0111] ,

[0112] Among them, f N This represents the deep features extracted from the transformed dataset during the Nth training iteration; after being expanded by a factor of 3 using the channel expansion unit, the deep feature map F is expanded to FN. c :

[0113] ,

[0114] Here, F1, F2, and F3 represent the feature data after the channel expansion is 3 times; F1 outputs features after passing through a frequency domain dynamic convolution unit and a first cross-shift unit. for:

[0115] ,

[0116] in, This represents the deep features extracted by the Nth feature data F1 after passing through the frequency domain dynamic convolution unit and the first cross-shift unit; F3 outputs the features after passing through the time domain dynamic convolution unit and the second cross-shift unit. for:

[0117] ,

[0118] in, This represents the deep features extracted from the Nth feature data F3 by the temporal dynamic convolution unit and the second cross-shift unit; then the three branches... F2 and After being input into the weight fusion unit, F is obtained. w :

[0119] ,

[0120] in, Represents the Nth feature data F2 and The characteristics of the corresponding elements after fusion; finally, F w The deep features after fusion are extracted by the second TFER convolutional unit to obtain F. o for:

[0121] ,

[0122] in, Represents the Nth feature data F w Deep features extracted by the second TFER convolutional unit.

[0123] S3. Construct an adaptive window size prediction network. Based on the feature map output by the adaptive multi-scale synthetic attention network, predict the window parameters required to construct the multi-scale attention fusion network.

[0124] Methods for obtaining window parameters include: constructing an adaptive window size prediction network, and convolving the feature map output by the adaptive multi-scale synthetic attention network with the Sobel operator to obtain the gradient map vector.

[0125] ,

[0126] in, This represents the gradient map obtained by convolving the Nth feature map with the Sobel operator; squaring the gradient map vector yields the edge intensity map vector.

[0127] ,

[0128] Among them, e N This represents the edge intensity map obtained by squaring the Nth gradient map vector; based on the edge intensity map vector, construct the global edge intensity vector, the maximum response intensity vector, and the salient edge density vector:

[0129] ,

[0130] ,

[0131] ,

[0132] ,

[0133] ,

[0134] ,

[0135] Where Ω represents the global edge intensity vector, Ψ represents the maximum response intensity vector, and Φ represents the salient edge density vector. Indicates the intensity of the Nth global edge. This represents the Nth maximum response intensity. Let n represent the density of the Nth salient edge. f n represents the frequency dimension of the edge intensity map. t Let represent the time dimension of the edge intensity map, ...

[0136] ,

[0137] ,

[0138] in, Represents the large window complexity vector. Represents the small window complexity vector. This represents the complexity parameter of the Nth large window. Let N represent the complexity parameter of the Nth small window; based on the complexity vectors of the large window and small windows, construct the large window vector and small window vector, and obtain the window parameters:

[0139] ,

[0140] ,

[0141] in, Represents a large window vector. Represents the small window vector. This represents the parameters of the Nth large window. This represents the parameters of the Nth small window.

[0142] S4. Construct a multi-scale attention fusion network based on window parameters and iterate to obtain a cetacean overlapping sound event detection model.

[0143] Methods for obtaining cetacean overlapping sound event detection models include: constructing a multi-scale attention fusion network based on window parameters, such as... Figure 4As shown, it includes: small-scale input unit, large-scale input unit, small-scale attention fusion unit, large-scale attention fusion unit, second weight fusion unit, GRU unit, linear layer and loss function; the multi-scale attention fusion network is iterated and the iterated model is tested using a test set. If the preset target is met, the cetacean overlapping sound event detection model is obtained.

[0144] In this embodiment, the workflow of the multi-scale attention fusion network includes: extracting deep features F from the time-frequency aware cross-offset feature extraction network. o The feature vectors of different sizes are obtained after being input into the network through the input unit and passing through the multi-scale attention fusion unit. and for:

[0145] ,

[0146] ,

[0147] in, Represents the Nth deep feature F o The large-scale feature vector is obtained after multi-scale large-scale attention fusion unit. Represents the Nth deep feature F o The small-scale feature vector is obtained after multi-scale small-scale attention fusion units; the large-scale feature vector is then... With small-scale feature vectors The deep feature vector Y, obtained by weight fusion through the weight fusion unit, is:

[0148] ,

[0149] Where α represents The weight, β represents The weights are then fused to obtain the output fused and enhanced deep feature vector Y. This vector is then passed through a GRU unit, a linear layer, and finally a loss function to obtain the detection result.

[0150] S5. Acquire real-time whale sound signals and use a whale overlapping sound event detection model to detect the real-time whale sound signals and obtain the detection results.

[0151] In this embodiment, the detection results cover four signal-to-noise ratio (SNR) levels: very low (-10 dB to -5 dB), low (-5 dB to 0 dB), medium (0 dB to 5 dB), and high (5 dB to 10 dB). The detection results include four typical North Atlantic right whale acoustic signal types: upcall, gunshot, scream, and moancall. Each audio file is accompanied by a precise timestamp label, indicating the start time, end time, and category of each acoustic event.

[0152] like Figure 5 As shown in the figure, the classification confusion matrix of four types of cetacean acoustic events is presented. It can be seen from the figure that all types of events exhibit high classification accuracy. Among them, moancall has the highest recognition accuracy, reaching 93.6%; scream has a recognition rate of 81.8%. Although some events exhibit some confusion, such as a high misclassification rate between upcall and moancall, the model generally performs well in distinguishing between the various categories.

[0153] like Figure 6 As shown, the PSDS scores of the Conformer, CRNN, PANNs, CNN-Transformer, TDNN-LSTM, and the AWMSA model in this embodiment vary under different signal-to-noise ratios. The model leads in all four signal-to-noise ratios.

[0154] Example 2:

[0155] In this embodiment, a cetacean overlapping sound event detection system based on adaptive multi-scale synthetic attention includes: a historical data acquisition module, a first model construction module, a window parameter acquisition module, a second model construction module, and a detection module.

[0156] The historical data acquisition module is used to collect historical whale sound signals, preprocess the historical whale sound signals and perform ACT transformation to obtain the transformed dataset.

[0157] The workflow of the historical data acquisition module includes: acquiring historical whale sound signals; performing non-overlapping, equal-length frame cropping on the historical whale sound signals to obtain the cropped signal; using a soundscape synthesizer library to synthesize the foreground signal and background sound from the cropped signal to obtain the synthesized signal; then extracting the ACT features of the synthesized signal and normalizing the features to obtain the transformed dataset.

[0158] ,

[0159] Where, x NThese represent different ACT features.

[0160] The first model building module is used to construct a time-frequency sensing cross-offset feature extraction network, and to train the time-frequency sensing cross-offset feature extraction network using the transformed data to obtain an adaptive multi-scale synthetic attention network.

[0161] The workflow of the first model construction module includes: constructing a time-frequency aware cross-offset feature extraction network, which consists of a first TFER convolutional unit, a channel expansion unit, a frequency domain dynamic convolutional unit, a time domain dynamic convolutional unit, a first cross-shift unit, a second cross-shift unit, a first weight fusion unit, and a second TFER convolutional unit; dividing the transformed data into a training set, a validation set, and a test set according to a preset ratio; training the time-frequency aware cross-offset feature extraction network using the training set, and validating the trained network using the validation set, to obtain an adaptive multi-scale synthetic attention network.

[0162] The window parameter acquisition module is used to construct an adaptive window size prediction network. Based on the feature map output by the adaptive multi-scale synthetic attention network, it predicts the window parameters required to construct the multi-scale attention fusion network.

[0163] The workflow of the window parameter acquisition module includes: constructing an adaptive window size prediction network; convolving the feature map output by the adaptive multi-scale synthetic attention network with the Sobel operator to obtain the gradient map vector.

[0164] ,

[0165] in, Let represent the gradient map obtained by convolving the i-th feature map with the Sobel operator, where i = 1, 2, ..., N, and N represents a natural number; squaring the gradient map vector yields the edge intensity map vector.

[0166] ,

[0167] Among them, e i Let represent the edge intensity map obtained by squaring the i-th gradient map vector, where i = 1, 2, ..., N, and N represents a natural number. Based on the edge intensity map vectors, construct the global edge intensity vector, the maximum response intensity vector, and the salient edge density vector:

[0168] ,

[0169] ,

[0170] ,

[0171] ,

[0172] ,

[0173] ,

[0174] Where Ω represents the global edge intensity vector, Ψ represents the maximum response intensity vector, and Φ represents the salient edge density vector. Indicates the intensity of the Nth global edge. This represents the Nth maximum response intensity. Let n represent the density of the Nth salient edge. f n represents the frequency dimension of the edge intensity map. t Let i represent the i-th frequency dimension of the edge intensity map, j represent the j-th time dimension of the edge intensity map, I represent the threshold function, and L represent a pixel value of the edge intensity map.

[0175] Based on the global edge strength vector, the maximum response strength vector, and the salient edge density vector, construct large window complexity vectors and small window complexity vectors:

[0176] ,

[0177] ,

[0178] in, Represents the large window complexity vector. Represents the small window complexity vector. This represents the complexity parameter of the Nth large window. This represents the complexity parameter of the Nth small window;

[0179] Based on the large window complexity vector and the small window complexity vector, construct the large window vector and the small window vector, and obtain the window parameters:

[0180] ,

[0181] ,

[0182] in, Represents a large window vector. Represents the small window vector. This represents the parameters of the Nth large window. This represents the parameters of the Nth small window.

[0183] The second model building module constructs a multi-scale attention fusion network based on window parameters and iterates it to obtain a cetacean overlapping sound event detection model.

[0184] The workflow of the second model construction module includes: constructing a multi-scale attention fusion network based on window parameters. The multi-scale attention fusion network consists of small-scale input units, large-scale input units, small-scale attention fusion units, large-scale attention fusion units, a second weight fusion unit, GRU units, linear layers, and a loss function; iterating the multi-scale attention fusion network and testing the iterated model using a test set. If the preset target is met, the cetacean overlapping sound event detection model is obtained.

[0185] The detection module is used to acquire real-time whale sound signals and use a whale overlapping sound event detection model to detect the real-time whale sound signals and obtain the detection results.

[0186] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for detecting overlapping sound events in cetaceans based on adaptive multi-scale synthetic attention, characterized in that, Includes the following steps: Historical whale sound signals were collected, preprocessed, and transformed using ACT to obtain the transformed dataset. A time-frequency sensing cross-offset feature extraction network is constructed. The transformed data is used to train the time-frequency sensing cross-offset feature extraction network to obtain a trained time-frequency sensing cross-offset feature extraction network. The trained time-frequency sensing cross-offset feature extraction network is then used to perform deep feature extraction on the transformed dataset to obtain a deep feature vector. An adaptive window size prediction network is constructed, and the window parameters required to construct a multi-scale synthetic attention fusion network are predicted based on the deep feature vector. The multi-scale synthetic attention fusion network is constructed based on the window parameters and iterated to obtain a cetacean overlapping sound event detection model. Real-time whale sound signals are acquired, and the whale overlapping sound event detection model is used to detect the real-time whale sound signals to obtain detection results.

2. The cetacean overlapping sound event detection method based on adaptive multi-scale synthetic attention according to claim 1, characterized in that, The methods for obtaining the transformed dataset include: Historical whale sound signals are collected, and the historical whale sound signals are processed by non-overlapping equal-length frame cropping to obtain the cropped signal; The foreground signal and background sound in the cropped signal are synthesized using a soundscape synthesizer library to obtain a synthesized signal. The ACT features of the synthesized signal are then extracted and normalized to obtain the transformed dataset. ， in, x N These represent different ACT features.

3. The cetacean overlapping sound event detection method based on adaptive multi-scale synthetic attention according to claim 1, characterized in that, Methods for obtaining a time-frequency sensed cross-offset feature extraction network after training include: A time-frequency aware cross-offset feature extraction network is constructed, which includes: a first TFER convolutional unit, a channel expansion unit, a frequency domain dynamic convolutional unit, a time domain dynamic convolutional unit, a first cross-shift unit, a second cross-shift unit, a first weight fusion unit, and a second TFER convolutional unit. The transformed data is divided into a training set, a validation set, and a test set according to a preset ratio; The time-frequency sensing cross-offset feature extraction network is trained using the training set and validated using the validation set to obtain the trained time-frequency sensing cross-offset feature extraction network.

4. The cetacean overlapping sound event detection method based on adaptive multi-scale synthetic attention according to claim 3, characterized in that, The methods for obtaining the window parameters include: Construct an adaptive window size prediction network, and convolve it with the Sobel operator based on the deep feature vector to obtain the gradient map vector: ， in, Indicates the first N The gradient map obtained by convolving each feature map with the Sobel operator; Squaring the gradient map vector yields the edge intensity map vector: ， in, e N Indicates the first N The edge intensity map is obtained by squaring the gradient map vectors. Based on the edge intensity map vector, construct the global edge intensity vector, the maximum response intensity vector, and the salient edge density vector: ， ， ， ， ， ， in, Ω Represents the global edge intensity vector. Ψ Represents the maximum response intensity vector. Φ Represents the significant edge density vector. Indicates the first N Global edge strength, Indicates the first N The maximum response intensity, Indicates the first N A significant edge density, n f This represents the frequency dimension of the edge intensity map. n t This represents the time dimension of the edge intensity map. i The first edge intensity map i One frequency dimension, j The first edge intensity map j One time dimension, I Represents the threshold function. L Represents a pixel value in the edge intensity map; Based on the global edge strength vector, the maximum response strength vector, and the salient edge density vector, construct the large window complexity vector and the small window complexity vector: ， ， in, Represents the large window complexity vector. This represents the small window complexity vector. This represents the complexity parameter of the Nth large window. This represents the complexity parameter of the Nth small window; Based on the large window complexity vector and the small window complexity vector, construct the large window vector and the small window vector, and obtain the window parameters: ， ， in, Represents a large window vector. Represents the small window vector. This represents the parameter of the Nth large window. Indicates the first N Small window parameters.

5. The cetacean overlapping sound event detection method based on adaptive multi-scale synthetic attention according to claim 4, characterized in that, The method for obtaining the whale overlapping sound event detection model includes: The multi-scale synthetic attention fusion network is constructed based on the window parameters. The multi-scale synthetic attention fusion network includes: a small-scale input unit, a large-scale input unit, a small-scale synthetic attention fusion unit, a large-scale synthetic attention fusion unit, a second weight fusion unit, a GRU unit, a linear layer, and a loss function. The multi-scale synthetic attention fusion network is iterated, and the iterated model is tested using the test set. If the preset target is met, the cetacean overlapping sound event detection model is obtained.

6. A cetacean overlapping sound event detection system based on adaptive multi-scale synthetic attention, wherein the detection system applies the detection method according to any one of claims 1-5, characterized in that, include: The system includes a historical data acquisition module, a first model construction module, a window parameter acquisition module, a second model construction module, and a detection module. The historical data acquisition module is used to acquire historical whale sound signals, preprocess the historical whale sound signals and perform ACT transformation to obtain the transformed dataset; The first model building module is used to build a time-frequency sensing cross-offset feature extraction network, train the time-frequency sensing cross-offset feature extraction network using the transformed data to obtain a trained time-frequency sensing cross-offset feature extraction network, and use the trained time-frequency sensing cross-offset feature extraction network to perform deep feature extraction on the transformed dataset to obtain a deep feature vector. The window parameter acquisition module is used to construct an adaptive window size prediction network and predict the window parameters required to construct a multi-scale synthetic attention fusion network based on the deep feature vector. The second model building module constructs the multi-scale synthetic attention fusion network based on the window parameters and iterates it to obtain a cetacean overlapping sound event detection model; The detection module is used to acquire real-time whale sound signals and use the whale overlapping sound event detection model to detect the real-time whale sound signals to obtain detection results.

7. The cetacean overlapping sound event detection system based on adaptive multi-scale synthetic attention according to claim 6, characterized in that, The workflow of the historical data acquisition module includes: Historical whale sound signals are collected, and the historical whale sound signals are processed by non-overlapping equal-length frame cropping to obtain the cropped signal; The foreground signal and background sound in the cropped signal are synthesized using a soundscape synthesizer library to obtain a synthesized signal. The ACT features of the synthesized signal are then extracted and normalized to obtain the transformed dataset. ， in, x N These represent different ACT features.

8. The cetacean overlapping sound event detection system based on adaptive multi-scale synthetic attention according to claim 6, characterized in that, The workflow of the first model building module includes: A time-frequency aware cross-offset feature extraction network is constructed, which includes: a first TFER convolutional unit, a channel expansion unit, a frequency domain dynamic convolutional unit, a time domain dynamic convolutional unit, a first cross-shift unit, a second cross-shift unit, a first weight fusion unit, and a second TFER convolutional unit. The transformed data is divided into a training set, a validation set, and a test set according to a preset ratio; The time-frequency sensing cross-offset feature extraction network is trained using the training set and validated using the validation set to obtain the trained time-frequency sensing cross-offset feature extraction network.

9. The cetacean overlapping sound event detection system based on adaptive multi-scale synthetic attention according to claim 8, characterized in that, The workflow of the window parameter acquisition module includes: Construct an adaptive window size prediction network, and convolve it with the Sobel operator based on the deep feature vector to obtain the gradient map vector: ， in, Indicates the first N The gradient map obtained by convolving each feature map with the Sobel operator; Squaring the gradient map vector yields the edge intensity map vector: ， in, e N Indicates the first N The edge intensity map is obtained by squaring the gradient map vectors. Based on the edge intensity map vector, construct the global edge intensity vector, the maximum response intensity vector, and the salient edge density vector: ， ， ， ， ， ， in, Ω Represents the global edge intensity vector. Ψ Represents the maximum response intensity vector. Φ Represents the significant edge density vector. Indicates the first N Global edge strength, Indicates the first N The maximum response intensity, Indicates the first N A significant edge density, n f This represents the frequency dimension of the edge intensity map. n t This represents the time dimension of the edge intensity map. i The first edge intensity map i One frequency dimension, j The first edge intensity map j One time dimension, I Represents the threshold function. L Represents a pixel value in the edge intensity map; Based on the global edge strength vector, the maximum response strength vector, and the salient edge density vector, construct the large window complexity vector and the small window complexity vector: ， ， in, Represents the large window complexity vector. This represents the small window complexity vector. This represents the complexity parameter of the Nth large window. This represents the complexity parameter of the Nth small window; Based on the large window complexity vector and the small window complexity vector, construct the large window vector and the small window vector, and obtain the window parameters: ， ， in, Represents a large window vector. Represents the small window vector. This represents the parameter of the Nth large window. Indicates the first N Small window parameters.

10. The cetacean overlapping sound event detection system based on adaptive multi-scale synthetic attention according to claim 9, characterized in that, The workflow of the second model building module includes: The multi-scale synthetic attention fusion network is constructed based on the window parameters. The multi-scale synthetic attention fusion network includes: a small-scale input unit, a large-scale input unit, a small-scale synthetic attention fusion unit, a large-scale synthetic attention fusion unit, a second weight fusion unit, a GRU unit, a linear layer, and a loss function. The multi-scale synthetic attention fusion network is iterated, and the iterated model is tested using the test set. If the preset target is met, the cetacean overlapping sound event detection model is obtained.

Images