Underwater acoustic target recognition method based on multi-modal fusion

Abstract

The application discloses a kind of based on multimodal fusion's underwater acoustic target identification method, comprising: collecting underwater acoustic target;Adopt the kernel density estimation function of adaptive bandwidth to process underwater acoustic target, fitting obtains the probability distribution function of underwater acoustic target, according to standard sampling rate to the probability distribution function is sampled, obtains optimization underwater acoustic target;After extracting multimodal feature to optimization underwater acoustic target, data enhancement is carried out, the feature before data enhancement and the feature after enhancement are merged, and feature data set is obtained;Using multiple canonical correlation analysis model selects and fuses feature data set;The output result of statistical model is fused and output using decision layer.Using the above technical scheme, higher recognition accuracy is obtained when identifying underwater acoustic target.

Description

Technical Field

[0001] This invention relates to the field of underwater acoustic target recognition technology, and in particular to an underwater acoustic target recognition method based on multimodal fusion. Background Technology

[0002] The sound composition in the ocean is quite complex, including acoustic signals from seabed organisms, noise from ship and submarine engines, and sounds transmitted from land into the water. The sound collected by hydrophones may contain several of these types of sounds at the same time. These various sounds overlap in the frequency spectrum, reducing the detectability of target features, which makes underwater acoustic signal identification very difficult.

[0003] Classifying underwater acoustic signals using deep learning methods requires a large amount of data for training. The most challenging task in underwater acoustic signal target recognition is extracting more inclusive feature vectors and constructing a high-precision and efficient target recognition model with limited samples. Currently, most underwater acoustic signal recognition systems are optimized under specific conditions to achieve optimal performance, which can lead to a risk of generalization when applied to new environments. Summary of the Invention

[0004] Purpose of the invention: This invention provides a method for underwater acoustic target recognition based on multimodal fusion, aiming to achieve higher recognition accuracy when recognizing underwater acoustic targets.

[0005] Technical Solution: This invention provides a method for underwater acoustic target recognition based on multimodal fusion, comprising: acquiring underwater acoustic targets; processing the underwater acoustic targets using an adaptive bandwidth kernel density estimation function to fit the probability distribution function of the underwater acoustic targets, sampling the probability distribution function according to a standard sampling rate to obtain optimized underwater acoustic targets; extracting multimodal features from the optimized underwater acoustic targets and performing data augmentation, merging the features before and after data augmentation to obtain a feature dataset; selecting and fusing the feature dataset using multiple canonical correlation analysis models; and using a decision layer to fuse the output results of the statistical models.

[0006] Specifically, the kernel density estimation function for the adaptive bandwidth is:

[0007]

[0008]

[0009] The adaptive bandwidth h at the j-th point is calculated based on the mean square integral error function. j ω is the bandwidth parameter, n is the variable dimension, M is the number of sampling points, K(x) is the Gaussian kernel function, S is the covariance matrix of the optimized underwater acoustic target, and k(x) is the bandwidth h. jThe kernel density estimation function.

[0010] Specifically, the optimal bandwidth parameter of the Gaussian kernel function is:

[0011]

[0012] Where α is the sensitivity factor and f is the distribution function.

[0013] Specifically, after obtaining the optimized underwater acoustic target, the process includes: performing 56ms frame processing on the optimized underwater acoustic target, with an overlap length of 28ms, and setting the window shape to a Hanning window.

[0014] Specifically, the multimodal features include: spectral analysis features, chromaticity features, and Mel frequency cepstral features.

[0015] Specifically, the spectral analysis feature extraction process includes: performing a short-time Fourier transform on the nth frame audio signal.

[0016] Specifically, the chromaticity feature extraction process includes: converting the optimized underwater acoustic target into frames, recording the energy of each pitch within each frame to obtain a pitch spectrum; and superimposing the energy of notes at the same time, the same pitch level, but different octaves onto the corresponding pitch level elements in the chromaticity vector to obtain a chromaticity spectrum.

[0017] Specifically, the Mel frequency cepstral feature extraction process includes: converting the actual frequency scale to a Mel frequency scale; and calculating the output of the triangular filter configured on the Mel frequency axis based on the amplitude spectrum of the audio signal.

[0018]

[0019]

[0020] Where m(l) is the Mel frequency coefficient of the l-th filter, L is the total number of filters, and W l (k) is a triangular filter, X n (k) represents the amplitude spectrum of the audio signal, and o(l), c(l), and h(l) are the lower limit, center, and upper limit frequencies of the triangular filter, respectively. Logarithmic operations are performed on all filter outputs, followed by discrete cosine transform.

[0021] Specifically, the selection and fusion of feature datasets using multiple canonical correlation analysis models includes: selecting two types of multimodal features from the canonical correlation analysis models for fusion; and training the fused features and the unfused features using different models respectively.

[0022] Specifically, the output of the statistical model is fused and output using the decision layer, including:

[0023]

[0024] Wherein, model i outputs a binary vector x i N is the total number of models, containing D elements, j is the position of the element in the vector, and w i Let y(j) be the weight coefficients of the i-th model, and y(j) be a row vector of length d. At time k, the decision layer output d(k) satisfies

[0025] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: (1) The adaptive kernel density estimation method is used to re-optimize the sampled data and enhance the internal features of the original data; (2) For the multimodal fusion method, two decision-making methods are adopted: feature layer fusion and decision layer fusion. The feature layer fusion part uses canonical correlation analysis to determine the features to be fused, so as to avoid mutual interference between features of different modalities. Each feature is trained using a different network model, and then the weighted voting mechanism is used in the decision layer to output the final result, thereby increasing the credibility of the final result. Attached Figure Description

[0026] Figure 1 Here is a flowchart for network identification;

[0027] Figure 2 This represents the relationship between adjacent Mel filters. Detailed Implementation

[0028] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0029] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0030] This invention provides a multi-feature fusion underwater target recognition method, such as... Figure 1 As shown, a specific implementation is as follows: The acquired underwater acoustic target signal is processed using an adaptive bandwidth kernel density estimation method to obtain a probability density function. The distribution function corresponding to this function is then sampled according to the original sampling rate to obtain an optimized underwater acoustic target signal. In the adaptive bandwidth kernel density estimation, a Gaussian kernel is selected as the kernel function, and the adaptive bandwidth parameter h is calculated based on the mean square integral error function. j The function is defined as follows:

[0031]

[0032]

[0033] Where h jLet be the adaptive bandwidth at the j-th point, ω be the bandwidth parameter, n be the dimension of the problem variables, M be the number of sampling points, K(x) be the kernel function, S be the covariance matrix of the optimized underwater acoustic target, and k(x) be the bandwidth of h. j The optimal choice of the Gaussian kernel function bandwidth can be obtained by calculating the kernel density estimation function:

[0034]

[0035] Where α is the sensitivity factor, 0≤α≤1, and f is the distribution function.

[0036] In this embodiment of the invention, the underwater acoustic target is preprocessed. The engine speed of large ships is between 1600-2300 rpm, so the audio signal is processed in frames with a length of 56ms and an overlap length of 28ms. The window shape is set to Hanning window.

[0037] In this embodiment of the invention, multimodal feature extraction and data augmentation are performed on the preprocessed data. The augmented features are then merged with the original features to form a new feature dataset D, wherein the multimodal features are as follows:

[0038] In terms of spectral analysis, the audio signal x of the nth frame... n (m) Performing a Fourier transform (Discrete-Time Fourier Transform, DTFT) yields the Short-Time Fourier Transform, which is defined as follows:

[0039]

[0040] As defined, the short-time Fourier transform is essentially the standard Fourier transform of a windowed speech signal. Here, the window w(nm) is a sliding window that slides along the sequence x(m) as n changes. In this invention, the Hanning window is chosen for processing because it has high resolution across the frequency range, significant sidelobe attenuation, and low spectral leakage.

[0041] In terms of chromaticity features, it is a general term for chromaticity vectors and chromaticity spectra. A chromaticity vector is a vector containing 12 elements, which represent the energy of 12 pitches within a time period (e.g., one frame). The energy of the same pitch at different octaves is accumulated. The chromaticity spectra are a sequence of chromaticity vectors. Specifically, the absolute time is converted into frames according to the length of the selected window, and the energy of each pitch within each frame is recorded, becoming a pitch spectra. Based on the pitch spectra, the energy (measured by loudness) of notes at the same time, the same pitch, but at different octaves is superimposed onto the elements of that pitch in the chromaticity vector, thus creating the chromaticity spectra.

[0042] Experiments on human auditory perception show that human hearing focuses only on certain specific regions, rather than the entire spectral envelope. Mel frequency analysis is based on these experiments, allowing only signals in a specific frequency band to pass while ignoring signals in other frequency bands. However, these filters are not uniformly distributed on the frequency axis. There are many filters in the low-frequency region, with a relatively dense distribution, while the number of filters becomes fewer and the distribution sparser in the high-frequency region. The Mel scale is a non-linear frequency scale based on the human ear's sensory judgment of equidistant pitch changes, and its relationship with frequency in Hertz is as follows:

[0043] m = 2595 * log10(1 + f / 700) (5)

[0044] Therefore, if the graduations on the Mel scale are uniform, the distances between Hertz will become increasingly larger. Mel-scale filter banks have high resolution in the low-frequency range, which matches the auditory characteristics of the human ear, and this is the physical significance of the Mel scale.

[0045] The Mel-frequency cepstral characteristic is obtained by taking the weighted sum of the amplitudes of all signals within the frequency bandwidth of each triangle of the filter as the output of a certain bandpass filter, then performing a logarithmic operation on all filter outputs, and finally performing a discrete cosine transform (DCT) to obtain the Mel-frequency cepstral characteristic. The specific calculation process is as follows:

[0046] The actual frequency scale is converted to the Mel frequency scale according to equation (2).

[0047] A triangular filter bank with L channels is configured on the Mel frequency axis, where the number L is determined by the cutoff frequency of the signal. The center frequency c(l) of each triangular filter is evenly distributed on the Mel frequency axis. Let o(l), c(l), and h(l) be the lower limit, center, and upper limit frequencies of the l-th triangular filter, respectively. Then the lower limit, center, and upper limit frequencies between adjacent triangular filters are as follows: Figure 2 As shown, the following relationship holds:

[0048] c(l)=h(l-1)=o(l+1) (6)

[0049] Based on the amplitude spectrum of the audio signal |X n (k)| Calculate the output of each triangular filter:

[0050]

[0051]

[0052] Performing a logarithmic operation on all filter outputs, followed by a Discrete Cosine Transform (DCT), yields the MFCC.

[0053]

[0054] Where m(l) is the Mel frequency coefficient of the l-th filter, L is the total number of filters, and W l (k) is a triangular filter, X n (k) represents the amplitude spectrum of the audio signal, o(l), c(l), and h(l) are the lower, center, and upper frequencies of the triangular filter, respectively, and N is the sequence length. Canonical correlation analysis is used to extract the two most representative features and fuse them. Different models are then used to train the fused and unfused features. Suitable classifiers include convolutional networks, long short-term memory networks, and residual networks.

[0055] In this embodiment of the invention, all model results are statistically analyzed, and finally, a decision-level fusion process is performed, parameter tuning is applied, and the output is determined using a weighted voting mechanism. Specifically, each classifier i outputs a binary vector x. i The array X contains D elements, where D is the number of hypothetical output decisions. The binary classification vector at position j indicates that the data is identified by the classifier as belonging to class j. The classification vector from the i-th classifier becomes the i-th row of array X, which is passed to the weighted decision fusion structure. Each row is then weighted using prior assumptions about the classifier's reliability. Subsequently, the elements of the array are summed along each column. This process can be described mathematically as follows:

[0056]

[0057] Wherein, model i outputs a binary vector x i N is the total number of models, containing D elements, j is the position of the element in the vector, and w i Let be the weight coefficients of the i-th model, and let be a row vector of length d along the output y(j), where each element represents the confidence level of the input data from multiple classifier sets in belonging to a specific class. At time k, the output decision d(k) satisfies:

[0058]

[0059] The weighted decision fusion approach, which prioritizes the category with the highest confidence, allows for future expansion in two ways. First, it provides a pathway to use confidence as input to each classifier; second, it can incorporate an adaptive mechanism that modifies the classifier weights as the system processes data.

Claims

1. A method for underwater acoustic target recognition based on multimodal fusion, characterized in that, include: Acoustic target acquisition; An adaptive bandwidth kernel density estimation function is used to process the underwater acoustic target, and the probability distribution function of the underwater acoustic target is obtained by fitting. The probability distribution function is then sampled according to the standard sampling rate to obtain the optimized underwater acoustic target. The optimized underwater acoustic target is processed by framing for 56ms, with an overlap length of 28ms, and the window shape is set to Hanning window; After extracting multimodal features from the optimized underwater acoustic target, data augmentation is performed. The features before and after data augmentation are then merged to obtain a feature dataset. The multimodal features include: spectral analysis features, chromaticity features, and Mel frequency cepstral features; Multiple canonical correlation analysis models were used to select and fuse the feature dataset; The output of the statistical model is fused and output using the decision layer.

2. The underwater acoustic target recognition method based on multimodal fusion according to claim 1, characterized in that, The kernel density estimation function for the adaptive bandwidth is: The adaptive bandwidth h at the j-th point is calculated based on the mean square integral error function. j ω is the bandwidth parameter, n is the variable dimension, M is the number of sampling points, K(x) is the Gaussian kernel function, S is the covariance matrix of the optimized underwater acoustic target, and k(x) is the bandwidth h. j The kernel density estimation function.

3. The underwater acoustic target recognition method based on multimodal fusion according to claim 2, characterized in that, The optimal bandwidth parameter of the Gaussian kernel function is: Where α is the sensitivity factor and f is the distribution function.

4. The underwater acoustic target recognition method based on multimodal fusion according to claim 3, characterized in that, The spectral analysis features extraction process includes: Perform a short-time Fourier transform on the nth frame of audio signal.

5. The underwater acoustic target recognition method based on multimodal fusion according to claim 3, characterized in that, The extraction process for the chromaticity features includes: The optimized underwater acoustic target is converted into frames, and the energy of each pitch within each frame is recorded to obtain a pitch spectrum; The energies of notes at the same time, of the same pitch, but at different octaves are superimposed onto the elements of the corresponding pitch in the chromaticity vector to obtain the chromaticity spectrum.

6. The underwater acoustic target recognition method based on multimodal fusion according to claim 3, characterized in that, The extraction process of the Mel frequency cepstral features includes: Convert the actual frequency scale to the Mel frequency scale; Based on the amplitude spectrum of the audio signal, the output of the triangular filter configured on the Mel frequency axis is calculated: Where m(l) is the Mel frequency coefficient of the l-th filter, L is the total number of filters, and W l (k) is a triangular filter, X n (k) represents the amplitude spectrum of the audio signal, and o(l), c(l) and h(l) are the lower limit, center and upper limit frequencies of the triangular filter, respectively; Perform a logarithmic operation on all filter outputs, and then perform a discrete cosine transform.

7. The underwater acoustic target recognition method based on multimodal fusion according to claim 3, characterized in that, The selection and fusion of feature datasets using multiple canonical correlation analysis models includes: The canonical correlation analysis model selects two of the multimodal features for fusion; different models are used to train the fused features and the unfused features respectively.

8. The underwater acoustic target recognition method based on multimodal fusion according to claim 7, characterized in that, The output of the statistical model is fused using a decision layer, including: Wherein, model i outputs a binary vector x i N is the total number of models, containing D elements, j is the position of the element in the vector, and w i Let y(j) be the weight coefficients of the i-th model, and y(j) be a row vector of length d. At time k, the decision layer output d(k) satisfies

Images