Physical prior guided and multi-resolution underwater acoustic event detection method
By constructing a multi-resolution detection network and embedding a frequency-domain physical prior mask matrix, the problems of multi-class identification and noise misjudgment in underwater acoustic event identification were solved, achieving high-precision detection of earthquakes, whale calls, and air gun pulses. In particular, it improved the identification accuracy of whale call acoustic events and reduced the false alarm rate.
Patent Information
- Application Number
- CN202611090864.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-22
- Publication Date
- 2026-11-06
AI Technical Summary
Existing underwater acoustic event identification methods lack multi-class identification capabilities, cannot simultaneously monitor multiple targets in complex marine acoustic environments, and a single detection network leads to the loss of features of minute acoustic events and misjudgment of noise.
A multi-resolution detection network is constructed, which is combined with a frequency domain physical prior mask matrix. A time-frequency map is generated by STFT and embedded into the detection network to guide multi-scale feature extraction and frequency domain filtering correction, thereby realizing the recognition of earthquakes, whale calls and air gun pulses.
It achieves high-precision recognition of multiple types of acoustic events, improves the recognition accuracy of whale call acoustic events, reduces the probability of false alarms caused by low-frequency noise, and has multi-scale detection capabilities.
Smart Images
Figure CN123053173A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of underwater acoustic signal processing and artificial intelligence technology, specifically to a physical prior guidance and multi-resolution underwater acoustic event detection method. Background Technology
[0002] The real ocean soundscape is highly complex and variable, interwoven with diverse and heterogeneous sound field signals, including natural geological activities (such as submarine earthquakes), marine life vocalizations (such as cetaceans), and human engineering operations (such as air gun exploration). Accurately separating and identifying these typical acoustic events amidst strong background noise and multi-source time-frequency overlap is not only a core challenge in analyzing the characteristics of complex underwater sound fields, but also has significant scientific and applied value for dynamic monitoring of the marine environment, ecological protection, and early warning of geological disasters.
[0003] Currently, deep learning-based target detection technology is widely used in time-frequency map recognition. The core idea is to generate a two-dimensional time-frequency map from underwater acoustic signals collected by hydrophones using a short-time Fourier transform, and then use a convolutional neural network to locate and classify acoustic events in the time-frequency map. However, when faced with the complex and ever-changing real-world ocean soundscape, existing methods still have the following significant limitations:
[0004] Most existing underwater acoustic event identification methods are designed for single categories, such as detecting whale calls or seismic signals alone, lacking the ability to simultaneously identify multiple types of acoustic events. In real-world marine monitoring scenarios, multiple types of acoustic events often occur simultaneously or alternately. For example, seismic activity may be accompanied by airgun exploration operations, and whale migration may overlap with human engineering activities in the time and frequency domains. Single-category identification methods cannot meet the needs of simultaneous multi-target monitoring in complex marine acoustic environments.
[0005] The scale differences of different underwater acoustic events on time-frequency maps are extremely large. For example, whale calls are usually weak targets lasting only about 2 seconds with a very narrow frequency band, accounting for a very small percentage of pixels in the time-frequency map, while earthquakes can last from tens to hundreds of seconds, showing a huge difference in time-frequency scale. Most existing detection networks use standard downsampling levels designed for natural images. Excessive downsampling ratios directly cause the high-frequency spatial features and details of tiny acoustic events to be erased deep in the network, resulting in serious missed detections.
[0006] Furthermore, existing purely data-driven detection networks rely solely on the apparent image texture of time-frequency maps for discrimination. This leads to visually similar sounds being easily misclassified into the same category. For example, when processing high-noise audio, some broadband background noise exhibits sudden high-energy characteristics in the time-frequency map, with image textures highly similar to natural earthquake signals, leading to frequent misclassification as earthquakes by the network. However, in reality, the frequency range of these misclassifying noises is often outside the physical frequency band of the actual earthquake signal. The root cause of this misclassification lies in the fact that existing purely visual detection paradigms deviate from the physical nature of acoustic signals, failing to incorporate the inherent frequency domain distribution patterns of different acoustic events as prior constraints into the decision-making process, thus generating a large number of false alarms in complex contexts. Summary of the Invention
[0007] To address the technical problems existing in the prior art, this invention provides a physical prior guidance and multi-resolution underwater acoustic event detection method, belonging to the field of intelligent underwater acoustic signal processing technology. The method converts underwater acoustic audio data into a time-frequency map using STFT; constructs a multi-resolution detection network to better match the time-frequency characteristics of the underwater acoustic data; constructs a frequency-domain physical prior mask matrix, which can dynamically filter and correct the original classification prediction probability in the frequency domain, and embeds it into the detection head to obtain a physical prior guidance multi-resolution detection network; and evaluates the recognition performance of the trained physical prior guidance multi-resolution detection network. This invention achieves the recognition of three types of underwater acoustic events: earthquakes, whale calls, and airgun pulses. The multi-resolution detection network improves the recognition accuracy of whale calls, and the frequency-domain physical prior mask matrix reduces the false alarm probability caused by low-frequency noise.
[0008] Physically guided and multi-resolution underwater acoustic event detection methods, such as Figure 1 As shown, it includes the following steps:
[0009] Step S1, underwater acoustic data preprocessing:
[0010] Raw acoustic audio data from continuous underwater observations were collected, and the raw acoustic audio data were preprocessed to obtain a time-frequency diagram;
[0011] Step S2, construct a multi-resolution detection network:
[0012] The multi-resolution detection network consists of a backbone network, a neck network, and three decoupled detection heads;
[0013] Step S3, construct the frequency domain physical prior mask matrix:
[0014] For the time-frequency map generated in step S1, a multi-scale frequency domain physical prior mask matrix is constructed according to the frequency constraint rules of different acoustic events.
[0015] Step S4: Embed the frequency domain physical prior mask matrix constructed in step 3 into the multi-resolution detection network constructed in step 2 to obtain the physical prior guided multi-resolution detection network.
[0016] Step S5: Train the physics-prior-guided multi-resolution detection network to obtain the trained physics-prior-guided multi-resolution detection network.
[0017] Step S6: Evaluate the trained physical prior-guided multi-resolution detection network.
[0018] Furthermore, in step S1, the underwater acoustic data preprocessing process is as follows:
[0019] Step S1.1, Acquisition of raw underwater acoustic signals and parameter configuration:
[0020] Continuous acoustic signal acquisition was conducted at underwater observation stations using hydrophones to obtain raw acoustic audio data;
[0021] During acquisition, the sampling rate was set to 24 kHz to ensure complete capture of acoustic events within the target frequency band of 0-1 kHz; the raw acoustic audio data acquired each time were in discrete time series format. Stored in the form of;
[0022] Step S1.2: Perform a Short-Time Fourier Transform (STFT) on the original acoustic audio data to generate a time-frequency representation;
[0023] The mathematical definition of STFT is: for discrete time series The STFT value at time frame index m and frequency point index k for:
[0024] (1);
[0025] in, It is a sliding window. For the intra-frame index position, , For frame length, For sampling point index; For time frame indexing, The imaginary unit, For the first The frame shift size is , For frame shift, For frequency point index, It is a constant. Pi;
[0026] The frequency range of the time-frequency representation is 0-12kHz;
[0027] Step S1.3: Perform frequency segment rendering on the time-frequency representation to amplify low-frequency details and obtain a time-frequency diagram;
[0028] First, based on the spectral distribution characteristics of three types of acoustic events—earthquakes, whale calls, and air gun pulses—0-1000Hz was selected as the effective analysis bandwidth.
[0029] Then, a frequency-band differentiated rendering strategy is used to render the time-frequency representation, generating a time-frequency map with an aspect ratio of 1:3:
[0030] The frequency axis is divided into two intervals: a low-frequency band and a high-frequency band. The low-frequency band is 0-150Hz, and the high-frequency band is 150-1000Hz. The low-frequency band occupies the lower 1 / 3 of the image height, and the high-frequency band occupies the upper 2 / 3 of the image height. This effectively amplifies the subtle time-frequency texture features of earthquakes and whale calls in the low-frequency region, ultimately generating a time-frequency map with an aspect ratio of 1:3. The generated time-frequency map and the features of the three types of acoustic events are shown in Figure 2.
[0031] Step S1.4: Annotate the time-frequency graph:
[0032] On the time-frequency plot image, rectangular bounding boxes are used to label the three types of acoustic events. The bounding boxes enclose the time-frequency energy profile of the acoustic events, containing the complete start and end times of the events on the time axis and covering the effective frequency band of the events on the frequency axis.
[0033] The rectangular bounding box annotation adopts a common annotation format for object detection, and each bounding box records five-tuple information. ;in, 0, 1, and 2 represent earthquakes, whale calls, and air gun pulse acoustic events, respectively; and These are the normalized coordinates of the bounding box center point in the time-frequency plot; and The normalized width and height of the bounding box;
[0034] Step S1.5, Dataset Partitioning and Data Augmentation:
[0035] The labeled time-frequency graph data were randomly divided into training set, validation set and test set in a ratio of 7:2:1.
[0036] Data augmentation is performed on the training set samples to improve generalization ability; the validation and test sets are not augmented, but only normalized to the same size as the training set.
[0037] The data enhancement includes Mosaic stitching enhancement and HSV (Value Saturation Hue) color space perturbation;
[0038] The Mosaic stitching enhancement involves randomly scaling four different labeled time-frequency images and stitching them together into a single training image to enrich the context of small target samples and increase the number of effective samples within a batch. The HSV color space perturbation is achieved by applying a random perturbation of ±10% to the brightness, saturation, and hue channels of the time-frequency diagram to simulate the signal intensity changes caused by different acquisition gains and underwater sound propagation attenuation.
[0039] Furthermore, in step S2, the multi-resolution detection network includes a backbone network, a neck network, and three decoupled detection heads; as shown... Figure 3 As shown;
[0040] The backbone network extracts hierarchical multi-scale feature maps through layer-by-layer downsampling;
[0041] The neck network performs bidirectional cross-scale feature fusion on the multi-scale feature map through a feature pyramid network and a path aggregation network to generate an enhanced multi-scale feature representation.
[0042] The three decoupled detection heads predict the enhanced multi-scale feature representation through independent classification and regression branches; the detection results of the three decoupled detection heads are combined to form the final output of the multi-resolution detection network.
[0043] Furthermore, the backbone network consists of five stacked convolutional stages. Downsampling is performed through convolutional layers with a stride of 2, and feature extraction is performed by combining four cross-stage local modules. Three layers of feature maps are derived in the second, fourth, and fifth convolutional stages. At the end of the fourth stage, a spatial pyramid pooling module is connected, which maximizes the expansion of the receptive field by connecting three max pooling layers with different kernel sizes without introducing additional parameters. Finally, the backbone network outputs three feature maps of different scales: shallow feature map P2, medium feature map P4, and deep feature map P5. The feature maps P2, P4, and P5 are distributed in a hierarchical manner from fine to coarse in terms of spatial resolution and semantic abstraction.
[0044] Furthermore, the neck network receives shallow feature map P2, mid-level feature map P4, and deep feature map P5 output by the backbone network, and generates an enhanced multi-scale feature representation through bidirectional cross-scale feature fusion.
[0045] The shallow feature map P2, the middle feature map P4, and the deep feature map P5 are each connected to an independent decoupled detection head;
[0046] The bidirectional cross-scale feature fusion includes a top-down fusion path and a bottom-up fusion path;
[0047] The top-down fusion path involves upsampling to progressively amplify the deep semantic features of the deep feature map P5 and then concatenating and fusion them with the corresponding shallow feature map P2 along the channel dimension to enhance the semantic expression of the shallow features.
[0048] The bottom-up fusion path: After top-down fusion, the high-resolution localization features of the shallow feature map P2 are compressed step by step through downsampling operation, and then spliced and convolved with the corresponding deep feature map P5 to enhance the localization capability of the deep features.
[0049] After bidirectional cross-scale feature fusion, the neck network outputs three enhanced multi-scale feature maps P2', P4', and P5', which are then fed into the corresponding decoupled detection heads P2'', P4'', and P5'', respectively.
[0050] Furthermore, the three decoupled detection heads are three different detection scales, namely decoupled detection heads P2'', P4'', and P5'', with downsampling magnification... The numbers are 4, 16, and 32 respectively, and each decoupled detection head contains independent classification and regression branches; the decoupled detection head P2'' is a high-resolution detection head.
[0051] The classification branch and regression branch have the same structure, consisting of stacked 3×3 standard convolutions and 1×1 pointwise convolutions, but with independent parameters. The classification branch extracts semantic discriminative features and outputs the original classification probability of each grid cell belonging to each category of acoustic event (earthquake, whale call, air gun pulse). The regression branch extracts spatial positioning features and uses an anchorless detection method to directly predict the offset of the target bounding box center point relative to the grid and the width and height of the bounding box without the need for preset fixed anchor boxes.
[0052] Furthermore, in step S3, the process of constructing the frequency domain physical prior mask matrix is as follows:
[0053] Step S3.1: Establish the mapping relationship between the vertical dimension of the time-frequency diagram and the physical frequency:
[0054] First, for the time-frequency map generated in step S1, calculate the number of pixels per unit frequency in the high-frequency and low-frequency bands. and :
[0055] (2);
[0056] (3);
[0057] in, This refers to the pixel row where the high-frequency band and the low-frequency band meet. The height of the time-frequency graph; and These are the cutoff frequencies for the low-frequency and high-frequency bands, respectively.
[0058] The high-frequency band corresponds to 150-1000Hz; the low-frequency band corresponds to 0-150Hz.
[0059] Then, the row of any pixel in the time-frequency graph is calculated according to formula (4). corresponding physical frequency :
[0060] (4);
[0061] Finally, the rate lookup table row2freq corresponding to the pixel row is calculated using formula (4), and the one-to-one correspondence between each row number and the corresponding frequency is stored.
[0062] Step S3.2: Establish the mapping relationship between the vertical dimension of the multi-scale feature map and the physical frequency:
[0063] For downsampling factor The feature map of the first rows of pixels, The downsampling factor is The height of the feature map, corresponding to the row number in the time-frequency plot. for: An offset of 0.5 is used to map the pixel centers of the feature map to the time-frequency map. This indicates rounding to the nearest integer.
[0064] The downsampling factor is obtained by querying the row2freq table corresponding to the frequency of the pixel row established in step S3.1. The feature map of the first physical frequency value corresponding to the row ,Right now:
[0065] (5);
[0066] The height of the time-frequency map generated in step S1, when When outside the signal range, Marked as invalid;
[0067] Step S3.3, define category-specific frequency constraint rules:
[0068] The different acoustic events include three categories: vibration, whale calls, and air gun pulses.
[0069] Based on prior knowledge of underwater acoustic physics, different types of acoustic events have distinctly different distribution characteristics in the frequency domain, which is used to determine frequency constraints.
[0070] For whale vocalizations: whale vocalizations are highly concentrated around 40Hz, with an extremely narrow bandwidth, representing a typical narrowband single-frequency signal. On a time-frequency graph, this appears as a very short texture spanning only 1 to 2 pixels along the frequency axis. Therefore, traversing adjacent rows of the feature map... ,like That is, 40Hz is located between the corresponding frequencies in the two rows, then it will be... All rows are marked as possible frequency bands for whales, with a mask value set to 1; the mask values for the remaining frequency rows are set to a minimal penalty factor. ; ;
[0071] For air gun pulse acoustic events: When the air gun source is excited, it generates a broadband pulse with energy distributed across the entire frequency range (0-1000Hz) from low to high frequencies. On the time-frequency graph, this is represented by a broadband strong energy texture covering multiple vertical rows. Therefore, air gun pulses have the same probability of occurrence in all frequency rows within the signal region, and the mask value is set to 1 across the entire domain.
[0072] For seismic acoustic events: the spectral energy of natural earthquake signals is concentrated below 100Hz, and the energy decays rapidly as the frequency increases, which is manifested as an exponentially decaying low-frequency strong texture on the time-frequency graph; based on statistical analysis of the spectrum of actual earthquake events, the effective frequency upper limit of earthquake signals is 65Hz; therefore, earthquakes only have a non-zero probability of occurrence at frequencies below 65Hz, i.e., when 65Hz, the mask value is set to 1; when 65Hz... At that time, the mask value is set to the smallest penalty factor. ;
[0073] Step S3.4: Construct a multi-scale frequency domain physical prior mask matrix:
[0074] The three decoupled detection heads correspond to feature maps at three scales. Each of the three scale feature maps is assigned a frequency domain physical prior mask matrix of shape according to the constraint rules in step S3.3. ; where superscript For each target category, earthquake is denoted as "earth," whale call as "whale," and airgun pulse as "airgun." The frequency domain physical prior mask matrix is then... :
[0075] ;
[0076] After the three-scale frequency domain physical prior mask matrices are constructed, they are saved as prior probability mapping files.
[0077] Furthermore, in step S4, the process of embedding the frequency domain physical prior mask matrix constructed in step 3 into the multi-resolution detection network constructed in step 2 is as follows: the downsampling factor is... The detection head generates a classification prediction probability map as follows ,in, For the set of real numbers To predict the height and width of the probability map, the shape is determined by the downsampling factor. The feature maps are the same.
[0078] The frequency domain physical prior mask matrix is broadcast. Expanding the width dimensionally yields the broadcast-extended frequency-domain physical prior mask matrix. The frequency domain physical prior mask matrix is expanded by dimension. With classification prediction probability map The shapes are consistent; perform element-wise product operation to obtain the final detection score after physical correction. .
[0079] Furthermore, in step S5, the training process for the physical prior-guided multi-resolution detection network is as follows:
[0080] The physical prior-guided multi-resolution detection network was trained using the training set and validation set. The training convergence curve and index changes are shown in Figure 4.
[0081] The training was performed using a stochastic gradient descent optimizer.
[0082] The optimizer hyperparameters are configured as follows:
[0083] Initial learning rate ,momentum Weight decay Batch size The total number of training rounds is set to 300.
[0084] First, confidence threshold filtering is used to remove candidate detection boxes with classification scores below the threshold, resulting in filtered detection results.
[0085] Next, non-maximum suppression (NMS) within the category is applied to the retained candidate boxes with an IoU threshold of 0.7 to remove redundant overlapping detections for the same acoustic event;
[0086] Then, the filtered detection results are converted The format is a six-tuple, where confidence is the detection confidence after physical correction;
[0087] Finally, the output includes the time-frequency bounding boxes of all detected acoustic events in the time-frequency plot and their category labels.
[0088] Furthermore, in step S6, the evaluation process for the trained physical prior-guided multi-resolution detection network is as follows:
[0089] The trained physical prior-guided multi-resolution detection network was evaluated using a test set. Based on the detection results, the recognition accuracy, recall, and mean average precision (mAP) were calculated to provide a comprehensive performance evaluation of the physical prior-guided multi-resolution detection network.
[0090] Step S6.1: Input the test set samples into the physical prior-guided multi-resolution detection network, and obtain the detection results through forward propagation. The detection results are shown in Figure 6.
[0091] Step S6.2, Calculation of the mean metrics for precision, recall, and average precision:
[0092] For a given confidence threshold and IOU matching threshold, each prediction result of the physical prior-guided multi-resolution detection network includes True Positive (TP), False Positive (FP), False Negative (FN), and True Negative (TN).
[0093] True positive TP: A true acoustic event that is correctly detected, whose predicted category is consistent with the true category, and whose intersection-union ratio (IOU) between the predicted bounding box and the true bounding box is greater than 0.5;
[0094] The false positive (FP) refers to the incorrect identification of background noise or other categories as the target acoustic event.
[0095] The false negative (FN) refers to a real acoustic event that was not detected.
[0096] The true negative TN: correctly identifies the background region as a non-target region;
[0097] The precision is:
[0098] (6);
[0099] The recall rate is:
[0100] (7);
[0101] Among them, Precision measures the proportion of truly positive samples among those with positive predictions, reflecting the precision capability of the physical prior-guided multi-resolution detection network; Recall measures the proportion of all truly positive samples that are correctly detected, reflecting the recall capability of the physical prior-guided multi-resolution detection network.
[0102] The mean accuracy mAP is:
[0103] Average precision (AP) for a single class is defined as the area under the precision-recall curve (PR curve) for a single class at different confidence thresholds.
[0104] The arithmetic mean of the AP values for all target categories is taken to obtain the mean precision mAP:
[0105] (8);
[0106] in, This represents the total number of target categories.
[0107] mAP50 refers to... The average precision under the threshold, in this invention, mAP50 is:
[0108] (9);
[0109] in, , and These represent earthquakes, whale calls, and airgun pulses, respectively. Average accuracy at the threshold.
[0110] Compared with the prior art, the technical effects of the present invention are as follows:
[0111] This invention achieves high-precision simultaneous identification of multiple categories of acoustic events. It unifies the detection task into a joint detection framework encompassing earthquakes, whale calls, and airgun pulses, and accurately distinguishes between various events even in complex scenarios with multiple targets coexisting. This effectively overcomes the limitations of existing single-class identification methods in simultaneous multi-target monitoring. It can detect acoustic events at multiple scales. By introducing a high-resolution shallow detection branch, the detection network is able to capture acoustic events with durations as short as 2 seconds and frequency bandwidths less than 5 Hz (such as whale calls). Experiments show that the recall rate for whale calls is significantly improved after introducing the P2 high-resolution detection head.
[0112] This invention overcomes the false alarm problem caused by deep-sea noise by addressing the underlying physical logic. It proposes a physical prior-guided detection mechanism that utilizes the deterministic mapping between pixel rows in a time-frequency graph and frequencies to transform the frequency domain distribution patterns of various acoustic events into a lightweight frequency domain physical prior mask matrix. This matrix is then embedded into the classification output layer of the detection network using an element-wise multiplication approach. This mechanism endows the multi-resolution detection network with "frequency domain filtering" capabilities. For acoustic categories that are physically impossible to appear in a certain frequency band, their classification scores are suppressed to near zero by a minimal penalty factor in the prior mask, thus eliminating false alarm responses from non-physical frequency bands at the decision-making level. Experiments show that this mechanism effectively suppresses false alarms caused by low-frequency background noise. Attached Figure Description
[0113] Figure 1 This is a flowchart illustrating the underwater acoustic event recognition method described in this invention.
[0114] Figure 2 is a comparison of the time-frequency characteristics and dimensions of different acoustic events (whale calls, earthquakes, and air gun pulses) constructed in this invention.
[0115] In Figure 2, Figure 2(a) shows the time-frequency characteristics of air gun pulses and whale calls; Figure 2(b) shows the time-frequency characteristics of earthquake events.
[0116] Figure 3 This is a schematic diagram of the physical prior-guided multi-resolution detection network structure of the present invention;
[0117] Figure 4 is a convergence curve of the training process of the physical prior-guided multi-resolution detection network of the present invention.
[0118] In Figure 4, Figure 4(a) is the training loss convergence curve; Figure 4(b) is the mAP50 index curve; Figure 4(c) is the recall index curve; and Figure 4(d) is the precision index curve.
[0119] Figure 5 is an example of the comparison of recognition effects before and after applying physical prior and multi-scale optimization in this invention;
[0120] In Figure 5, Figure 5(a) shows the recognition effect of whale call events without the introduction of the P2 high-resolution detector head; Figure 5(b) shows the recognition effect of whale call events after the introduction of the P2 high-resolution detector head; Figure 5(c) shows the recognition effect of earthquake events without the use of the physical prior mask matrix; and Figure 5(d) shows the recognition effect of earthquake events after the use of the physical prior mask matrix.
[0121] Figure 6 shows the final identification results of the present invention for multiple types of underwater acoustic events;
[0122] In Figure 6, Figure 6(a) shows the recognition effect of the physical prior-guided multi-resolution detection network on air gun pulse events of different intensities; Figure 6(b) shows the recognition effect of the physical prior-guided multi-resolution detection network on whale call events of different intensities; and Figure 6(c) shows the recognition effect of the physical prior-guided multi-resolution detection network on earthquake events of different intensities. Detailed Implementation
[0123] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
[0124] Based on the dataset construction mentioned above, time-frequency samples are input into a physical prior-guided multi-resolution detection network for training. The effectiveness of the method and the model can be further verified by analyzing the training process and the performance of the final model on the test set.
[0125] Figure 4 illustrates the training process for the three types of acoustic event recognition tasks, while Figures 5 and 6 show the recognition results. A comparison of Figures 5(a) and 5(b) shows that no whale call events were detected without the P2 high-resolution detector. This is because the downsampling of features was too large without the P2 high-resolution detector, leading to missed detections. The introduction of the P2 high-resolution detector enabled the correct detection of whale call events, and the confidence level was also improved.
[0126] A comparison of Figures 5(c) and 5(d) shows that false detections of earthquakes occurred without using the physical prior mask matrix (5(c)). This is because the characteristics of earthquakes are similar to strong noise textures. However, by using the physical prior mask matrix, false detections can be eliminated through detection suppression of erroneous frequency bands (5(d)). Figure 6 also shows the final identification results, demonstrating that the proposed method can effectively identify three types of acoustic events and clearly outline the frequency range and start and end times of the acoustic events on the time-frequency plot.
[0127] It should be noted that the frequency-domain physical prior mask matrix mechanism proposed in this invention is essentially a general method decoupled from specific network architectures. Its core is to utilize the deterministic mapping relationship between the vertical dimension of the time-frequency graph and physical frequencies to transform the frequency-domain distribution prior of acoustic events into a mask matrix acting on the classification output layer. This method does not depend on specific backbone networks or detector head designs and can be embedded into the classification branches of various single-stage (such as SSD, RetinaNet) and two-stage (such as Faster R-CNN) object detection frameworks, providing physically constrained classification decision-making capabilities for any time-frequency graph-oriented multi-resolution object detection network.
Claims
1. A method for detecting underwater acoustic events using physical prior guidance and multi-resolution, characterized in that, Includes the following steps: Step S1, underwater acoustic data preprocessing: Raw acoustic audio data from continuous underwater observations were collected, and the raw acoustic audio data were preprocessed to obtain a time-frequency diagram; Step S2, construct a multi-resolution detection network: The multi-resolution detection network consists of a backbone network, a neck network, and three decoupled detection heads; Step S3, construct the frequency domain physical prior mask matrix: For the time-frequency map generated in step S1, a multi-scale frequency domain physical prior mask matrix is constructed according to the frequency constraint rules of different acoustic events. Step S4: Embed the frequency domain physical prior mask matrix constructed in step 3 into the multi-resolution detection network constructed in step 2 to obtain the physical prior guided multi-resolution detection network. Step S5: Train the physics-prior-guided multi-resolution detection network to obtain the trained physics-prior-guided multi-resolution detection network. Step S6: Evaluate the trained physical prior-guided multi-resolution detection network.
2. The physical prior guidance and multi-resolution underwater acoustic event detection method according to claim 1, characterized in that, In step S1, the underwater acoustic data preprocessing process is as follows: Step S1.1, Acquisition of raw underwater acoustic signals and parameter configuration: Continuous acoustic signal acquisition was conducted at underwater observation stations using hydrophones to obtain raw acoustic audio data; During acquisition, the sampling rate was set to 24 kHz to ensure complete capture of acoustic events within the target frequency band of 0-1 kHz; the raw acoustic audio data acquired each time were in discrete time series format. Stored in the form of; Step S1.2: Perform STFT on the raw acoustic audio data to generate a time-frequency representation; The mathematical definition of STFT is: for discrete time series The STFT value at time frame index m and frequency point index k for: (1)； in, It is a sliding window. For the intra-frame index position, , For frame length, For sampling point index; For time frame indexing, The imaginary unit, For the first The frame shift size is , For frame shift, For frequency point index, It is a constant. Pi; The frequency range of the time-frequency representation is 0-12kHz; Step S1.3: Perform frequency segment rendering on the time-frequency representation to obtain the time-frequency diagram; First, based on the spectral distribution characteristics of three types of acoustic events—earthquakes, whale calls, and air gun pulses—0-1000Hz was selected as the effective analysis bandwidth. Then, a frequency-band differentiated rendering strategy is used to render the time-frequency representation, generating a time-frequency map with an aspect ratio of 1:3: The frequency axis is divided into two intervals: a low-frequency band and a high-frequency band. The low-frequency band is 0-150Hz, and the high-frequency band is 150-1000Hz. The low-frequency band occupies the lower 1 / 3 of the image height, and the high-frequency band occupies the upper 2 / 3 of the image height, ultimately generating a time-frequency graph with an aspect ratio of 1:
3. Step S1.4: Annotate the time-frequency graph: On the time-frequency plot image, rectangular bounding boxes are used to label the three types of acoustic events. The bounding boxes enclose the time-frequency energy profile of the acoustic events, containing the complete start and end times of the events on the time axis and covering the effective frequency band of the events on the frequency axis. The rectangular bounding box annotation adopts a common annotation format for object detection, and each bounding box records five-tuple information. ;in, 0, 1, and 2 represent earthquakes, whale calls, and air gun pulse acoustic events, respectively; and These are the normalized coordinates of the bounding box center point in the time-frequency plot; and The normalized width and height of the bounding box; Step S1.5, Dataset Partitioning and Data Augmentation: The labeled time-frequency graph data were randomly divided into training set, validation set and test set in a ratio of 7:2:
1. Data augmentation is performed on the training set samples to improve generalization ability; the validation and test sets are not augmented, but only normalized to the same size as the training set. The data enhancements include Mosaic stitching enhancement and HSV color space perturbation; The Mosaic stitching enhancement involves randomly scaling four different labeled time-frequency images and stitching them together into a single training image to enrich the context of small target samples and increase the number of effective samples within a batch. The HSV color space perturbation is achieved by applying a random perturbation of ±10% to the brightness, saturation, and hue channels of the time-frequency diagram to simulate the signal intensity changes caused by different acquisition gains and underwater sound propagation attenuation.
3. The physical prior guidance and multi-resolution underwater acoustic event detection method according to claim 1, characterized in that, In step S2, the multi-resolution detection network includes a backbone network, a neck network, and three decoupled detection heads; The backbone network extracts hierarchical multi-scale feature maps through layer-by-layer downsampling; The neck network performs bidirectional cross-scale feature fusion on the multi-scale feature map through a feature pyramid network and a path aggregation network to generate an enhanced multi-scale feature representation. The three decoupled detection heads predict the enhanced multi-scale feature representation through independent classification and regression branches; The combined detection results from the three decoupled detection heads constitute the final output of the multi-resolution detection network.
4. The physical prior guidance and multi-resolution underwater acoustic event detection method according to claim 3, characterized in that, The backbone network consists of five stacked convolutional stages. It performs downsampling through convolutional layers with a stride of 2, combines four cross-stage local modules for feature extraction, and extracts three layers of feature maps in the second, fourth, and fifth convolutional stages. At the end of the fourth stage, a spatial pyramid pooling module is connected, which maximizes the expansion of the receptive field by connecting three max pooling layers with different kernel sizes without introducing additional parameters. Finally, the backbone network outputs three feature maps of different scales: shallow feature map P2, medium feature map P4, and deep feature map P5.
5. The physical prior guidance and multi-resolution underwater acoustic event detection method according to claim 3, characterized in that, The neck network receives shallow feature map P2, middle feature map P4 and deep feature map P5 output by the backbone network, and generates an enhanced multi-scale feature representation through bidirectional cross-scale feature fusion. The shallow feature map P2, the middle feature map P4, and the deep feature map P5 are each connected to an independent decoupled detection head; The bidirectional cross-scale feature fusion includes a top-down fusion path and a bottom-up fusion path; The top-down fusion path involves upsampling to progressively amplify the deep semantic features of the deep feature map P5 and then concatenating and fusion them with the corresponding shallow feature map P2 along the channel dimension to enhance the semantic expression of the shallow features. The bottom-up fusion path: After top-down fusion, the high-resolution localization features of the shallow feature map P2 are compressed step by step through downsampling operation, and then spliced and convolved with the corresponding deep feature map P5 to enhance the localization capability of the deep features. After bidirectional cross-scale feature fusion, the neck network outputs three enhanced multi-scale feature maps P2', P4', and P5', which are then fed into the corresponding decoupled detection heads P2'', P4'', and P5'', respectively.
6. The physical prior guidance and multi-resolution underwater acoustic event detection method according to claim 3, characterized in that, The three decoupled detection heads are three different detection scales, namely decoupled detection heads P2'', P4'', and P5'', with downsampling magnification. The numbers are 4, 16, and 32 respectively, and each decoupled detection head contains independent classification and regression branches; the decoupled detection head P2'' is a high-resolution detection head. The classification branch and regression branch have the same structure, consisting of stacked 3×3 standard convolutions and 1×1 pointwise convolutions, but with independent parameters. The classification branch extracts semantic discriminative features and outputs the original classification probability of each grid cell belonging to each category of acoustic event. The regression branch extracts spatial localization features and uses an anchorless detection method to directly predict the offset of the target bounding box center point relative to the grid and the width and height of the bounding box, without the need for preset fixed anchor boxes.
7. The physical prior guidance and multi-resolution underwater acoustic event detection method according to claim 1, characterized in that, In step S3, the process of constructing the frequency domain physical prior mask matrix is as follows: Step S3.1: Establish the mapping relationship between the vertical dimension of the time-frequency diagram and the physical frequency: First, for the time-frequency map generated in step S1, calculate the number of pixels per unit frequency in the high-frequency and low-frequency bands. and : (2)； (3)； in, This refers to the pixel row where the high-frequency band and the low-frequency band meet. The height of the time-frequency graph; and These are the cutoff frequencies for the low-frequency and high-frequency bands, respectively. The high-frequency band corresponds to 150-1000Hz; the low-frequency band corresponds to 0-150Hz. Then, the row of any pixel in the time-frequency graph is calculated according to formula (4). corresponding physical frequency : (4)； Finally, the rate lookup table row2freq corresponding to the pixel row is calculated using formula (4), and the one-to-one correspondence between each row number and the corresponding frequency is stored. Step S3.2: Establish the mapping relationship between the vertical dimension of the multi-scale feature map and the physical frequency: For downsampling factor The feature map of the first rows of pixels, , The downsampling factor is The height of the feature map, corresponding to the row number in the time-frequency plot. for: An offset of 0.5 is used to map the pixel centers of the feature map to the time-frequency map. This indicates rounding to the nearest integer. The downsampling factor is obtained by querying the row2freq table corresponding to the frequency of the pixel row established in step S3.
1. The feature map of the first physical frequency value corresponding to the row ,Right now: (5)； The height of the time-frequency map generated in step S1, when When outside the signal range, Marked as invalid; Step S3.3, define category-specific frequency constraint rules: The different acoustic events include three categories: vibration, whale calls, and air gun pulses. Based on prior knowledge of underwater acoustic physics, different types of acoustic events have distinctly different distribution characteristics in the frequency domain, which is used to determine frequency constraints. For whale vocalizations: whale vocalizations are highly concentrated around 40Hz, with an extremely narrow bandwidth, representing a typical narrowband single-frequency signal. On a time-frequency graph, this appears as a very short texture spanning only 1 to 2 pixels along the frequency axis. Therefore, traversing adjacent rows of the feature map... ,like That is, 40Hz is located between the corresponding frequencies of the two rows, then... and All rows are marked as possible frequency bands for whales, with a mask value set to 1; the mask values for the remaining frequency rows are set to a minimal penalty factor. ; For air gun pulse acoustic events: When the air gun source is excited, it generates a broadband pulse with energy distributed across the entire frequency range from low to high frequencies. On the time-frequency graph, this is represented by a broadband strong energy texture covering multiple rows in the vertical direction. Therefore, air gun pulses have the same probability of occurrence in all frequency rows within the signal region, and the mask value is set to 1 across the entire domain. For seismic acoustic events: the spectral energy of natural earthquake signals is concentrated below 100Hz, and the energy decays rapidly as the frequency increases, which is manifested as an exponentially decaying low-frequency strong texture on the time-frequency graph; based on statistical analysis of the spectrum of actual earthquake events, the effective frequency upper limit of earthquake signals is 65Hz; therefore, earthquakes only have a non-zero probability of occurrence at frequencies below 65Hz, i.e., when 65Hz, the mask value is set to 1; when 65Hz... At that time, the mask value is set to the smallest penalty factor. ; Step S3.4: Construct a multi-scale frequency domain physical prior mask matrix: The three decoupled detection heads correspond to feature maps at three scales. Each of the three scale feature maps is assigned a shape according to the constraint rules in step S3.
3. Frequency domain physical prior mask matrix ; where superscript For each target category, earthquake is denoted as "earth," whale call as "whale," and airgun pulse as "airgun." The frequency domain physical prior mask matrix is then... : After the three-scale frequency domain physical prior mask matrices are constructed, they are saved as prior probability mapping files.
8. The physical prior guidance and multi-resolution underwater acoustic event detection method according to claim 1, characterized in that, In step S4, the process of embedding the frequency domain physical prior mask matrix constructed in step 3 into the multi-resolution detection network constructed in step 2 is as follows: downsampling factor The detection head generates a classification prediction probability map as follows ,in, For the set of real numbers To predict the height and width of the probability map, the shape is determined by the downsampling factor. The feature maps are the same; The frequency domain physical prior mask matrix is broadcast. Expanding the width dimensionally yields the broadcast-extended frequency-domain physical prior mask matrix. The frequency domain physical prior mask matrix is expanded by dimension. With classification prediction probability map The shapes are consistent; Perform element-wise product operation to obtain the final detection score after physical correction. .
9. The physical prior guidance and multi-resolution underwater acoustic event detection method according to claim 1, characterized in that, In step S5, the training process for the physical prior-guided multi-resolution detection network is as follows: The physics-guided multi-resolution detection network was trained using training and validation sets; The training was performed using a stochastic gradient descent optimizer. First, confidence threshold filtering is used to remove candidate detection boxes with classification scores below the threshold, resulting in filtered detection results. Next, intra-class nonmaximum suppression is applied to the retained candidate boxes to remove redundant overlapping detections for the same acoustic event; Then, the filtered detection results are converted The format is a six-tuple, where confidence is the detection confidence after physical correction; Finally, the output includes the time-frequency bounding boxes of all detected acoustic events in the time-frequency plot and their category labels.
10. The physical prior guidance and multi-resolution underwater acoustic event detection method according to claim 1, characterized in that, In step S6, the evaluation process for the trained physical prior-guided multi-resolution detection network is as follows: The trained physical prior-guided multi-resolution detection network was evaluated using a test set. Based on the detection results, the recognition accuracy, recall, and mean precision (mAP) were calculated to provide a comprehensive performance evaluation of the physical prior-guided multi-resolution detection network. Step S6.1: Input the test set samples into the physical prior-guided multi-resolution detection network, and obtain the detection results through forward propagation; Step S6.2, Calculation of the mean metrics for precision, recall, and average precision: For a given confidence threshold and IOU matching threshold, each prediction result of the physical prior-guided multi-resolution detection network includes true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN). True positive TP: A true acoustic event that is correctly detected, whose predicted category is consistent with the true category, and whose intersection-union ratio (IOU) between the predicted bounding box and the true bounding box is greater than 0.5; The false positive (FP) refers to the incorrect identification of background noise or other categories as the target acoustic event. The false negative (FN) refers to a real acoustic event that was not detected. The true negative TN: correctly identifies the background region as a non-target region; The precision is: (6)； The recall rate is: (7)； Among them, Precision measures the proportion of truly positive samples among those with positive predictions, reflecting the precision capability of the physical prior-guided multi-resolution detection network; Recall measures the proportion of all truly positive samples that are correctly detected, reflecting the recall capability of the physical prior-guided multi-resolution detection network. The mean accuracy mAP is: The average precision (AP) of a single class is defined as the area under the precision-recall curve for a single class at different confidence thresholds. The arithmetic mean of the AP values for all target categories is taken to obtain the mean precision mAP: (8)； in, This represents the total number of target categories.