Small sample ship soundprint data enhancement and high-precision identification method

Abstract

The present application belongs to the technical field of underwater acoustic target recognition and machine learning, and discloses a small sample ship acoustic print data enhancement and high-precision recognition method. The method divides effective audio slices into small sample data sets, large sample data sets and overall data sets based on the sample amount of each category. Then, the effective audio slices are subjected to short-time Fourier transform and Mel filtering processing to generate fixed-size log Mel spectrograms. Next, a convolutional variational autoencoder is constructed and trained to form a reconstructed small sample data set. After that, the small sample data set, the reconstructed small sample data set and the large sample data set are combined, and the merged data set is subjected to feature extraction by a BEATs encoder with frozen parameters to obtain an acoustic print feature vector. A LightGBM classifier is trained and called for classification and recognition to output the recognition result of the ship category. The method significantly alleviates the overfitting problem of the classifier and can improve the precision and stability of ship acoustic print data recognition.

Description

Technical Field

[0001] This invention belongs to the field of underwater acoustic target recognition and machine learning technology, specifically relating to a method for enhancing and high-precision recognition of small sample ship acoustic data. Background Technology

[0002] Long-range underwater target detection primarily relies on acoustic signals. Compared to spatial electromagnetic sensing, underwater acoustic propagation is significantly affected by sea conditions, including wave intensity, seawater temperature and salinity distribution, and acoustic refraction effects. It is also subject to interference from platform motion and environmental noise, such as wave noise, wind noise, mechanical noise, and other ship-radiated noise. These factors lead to significant differences in acoustic observation results for the same target at different times and in different sea areas, manifesting as domain shift or distribution drift. Existing publicly available ship acoustic signature datasets generally suffer from deficiencies in sample size, annotation density, and inter-class separability. Directly employing end-to-end depth models under small sample conditions easily leads to overfitting and decreased generalization performance.

[0003] In 2020, the Belgian imec-IDLab proposed the ECAPA-TDNN method, which introduces SE-Res2Block and channel-aware statistical pooling into the x-vector framework. It expands the temporal receptive field through multi-scale dilated convolutions and adaptively weights channel features, reducing the equal error rate (EER) to 0.87% in VoxSRC 2019, but with a parameter count of 14.7M and an inference real-time factor (RTF) of 0.033. In 2023, Alibaba DAMO Academy proposed the CAM++ method, using a densely connected TDNN as its backbone and introducing a lightweight context masking module. It generates frame-level masks through multi-granularity pooling for feature correction, achieving EERs of 0.73% and 6.78% on the VoxCeleb-O and CN-Celeb datasets, respectively, while reducing the parameter count to 7.18M and the RTF to 0.013.

[0004] While the aforementioned methods perform well in large-sample speaker recognition tasks, they still exhibit the following problems in small-sample underwater voiceprint recognition scenarios such as the ShipsEar dataset: First, the parameter-sample ratio is severely imbalanced under small sample conditions. The ECAPA-TDNN and CAM++ methods have parameter sizes of 14.7M and 7.18M respectively, both requiring hundreds of thousands of training samples to achieve stable convergence. However, the ShipsEar dataset, after slicing, contains only about 8000 data segments, with fewer than 100 segments per class, far below the safe parameter-sample ratio threshold. This leads to the typical overfitting phenomenon of increased training performance followed by decreased validation performance in the early stages of training. Second, the fixed high-dimensional feature space is highly sensitive to environmental disturbances in small-sample scenarios. Both methods use 80-dimensional Fbank features as input. When samples are insufficient, random disturbances such as sea state changes and self-noise are easily mislearned as class discrimination features. On the ShipsEar dataset, the confusion rate between easily confused categories such as fishing boats and motorboats increases significantly, resulting in insufficient overall recognition stability. Third, there is a lack of effective generative data augmentation mechanisms. The CAM++ method primarily relies on the SpecAug masking strategy, while the ECAPA-TDNN method introduces speech rate perturbation on top of this. However, both are online perturbations and fail to expand the sample distribution. Under small sample conditions, excessively high masking probabilities reduce prediction stability, while low masking probabilities offer limited enhancement.

[0005] In summary, the large-parameter, large-sample design paradigms represented by ECAPA-TDNN and CAM++ are prone to parameter redundancy and generalization performance degradation in small-sample underwater voiceprint recognition scenarios such as the ShipsEar dataset. It is necessary to introduce a recognition framework that is oriented towards small-sample conditions and takes into account both model lightweighting and generative data augmentation capabilities. Summary of the Invention

[0006] The purpose of this invention is to propose a method for enhancing and recognizing small-sample ship acoustic data. This method can expand small-sample datasets without requiring additional manual annotation, and can ensure that small samples achieve the intra-class dispersion of large samples, significantly alleviating the overfitting problem of classifiers and improving the accuracy and stability of ship acoustic data recognition.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A method for enhancing and high-precision identifying small-sample ship acoustic signature data includes the following steps:

[0009] Step 1. Resample, slide slice, and clean the energy threshold of the original audio of the ship's acoustic signature to obtain effective audio slices. Based on the sample size of each category, the effective audio slices are divided into small sample datasets, large sample datasets, and the overall dataset.

[0010] Step 2. Perform short-time Fourier transform and Mel filtering on the small sample dataset and the effective audio slices in the overall dataset to generate a fixed-size log-Mel spectrogram;

[0011] Step 3. Construct and train the Convolutional Variational Autoencoder (Conv-VAE). Train the Conv-VAE using the log-Mel spectrogram of the entire dataset. After training, use the trained Conv-VAE to reconstruct and generate the log-Mel spectrogram of the small sample dataset to obtain the reconstructed spectrogram. Further generate reconstructed audio slices to form the reconstructed small sample dataset.

[0012] Step 4. Merge the small sample dataset, the reconstructed small sample dataset, and the large sample dataset. Use the BEATs encoder with frozen parameters to extract features from the merged dataset to obtain the voiceprint feature vector.

[0013] Step 5. Using the voiceprint feature vector as input, train and call the LightGBM classifier to perform classification and recognition, and output the recognition result of the ship category.

[0014] The present invention has the following advantages:

[0015] First, the method of this invention improves the accuracy and stability of ship acoustic data recognition. Compared with the current mainstream underwater acoustic model, the recognition accuracy is significantly improved, and the standard deviation of the 5-fold cross-validation is less than 0.3%, indicating that the method of this invention is stable and reproducible.

[0016] Secondly, the method of this invention has the advantage of being friendly to small samples. Both ECAPA-TDNN and CAM++ methods use full network fine-tuning, which requires more than 2,000 samples per class to converge. However, the method of this invention uses a BEATs encoder with frozen parameters to extract features from the merged dataset, which only requires more than 500 samples per class to approach asymptotic accuracy, reducing the data requirement by more than 75%.

[0017] Third, the method of this invention improves efficiency. It uses frozen feature extraction and lightweight tree model for training and inference. Under a general CPU environment, single-fold cross-validation can be completed in minutes. After extracting a 768-dimensional vector in a single forward pass during the inference stage, the tree model traversal can output the category in milliseconds, without the need for repeated GPU calculations, which significantly shortens the end-to-end latency.

[0018] Fourth, the method of this invention also has advantages in terms of storage and energy consumption. The method proposed in this invention only contains the quantified BEATs encoder model weights and LightGBM classifier model files, and its total size is less than 500MB, which is far lower than that of mainstream end-to-end deep networks. The tree model inference process is completed in a CPU environment, and the terminal power consumption is significantly reduced.

[0019] Fifth, the method of this invention is simple to operate. In the Conv-VAE convolutional variational autoencoder used in this invention, VAE data augmentation is completed in the latent space without introducing additional noise. The model mask pre-training of the BEATs encoder has absorbed two million hours of general voiceprints, and the feature generalization is strong. The leaf split gain of the LightGBM classifier automatically captures high-dimensional nonlinear boundaries without the need for manual design of convolution kernels or attention patterns, thereby reducing the difficulty of operation while ensuring accuracy. Attached Figure Description

[0020] Figure 1 This is a flowchart of a method for enhancing and high-precision identifying small sample ship acoustic signature data in an embodiment of the present invention.

[0021] Figure 2 This is a flowchart illustrating the small-sample ship acoustic signature enhancement and recognition based on the ConvVAE-BEATsLGB method in this embodiment of the invention.

[0022] Figure 3 This is a network structure diagram of the Convolutional Variational Autoencoder (Conv-VAE) in an embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram of the signal processing flow in the BEATs encoder in an embodiment of the present invention.

[0024] Figure 5 This is a schematic diagram of the signal processing flow in the LightGBM classifier in an embodiment of the present invention.

[0025] Figure 6 This is the confusion matrix obtained by using the ConvVAE-BEATsLGB method for ship acoustic signature data recognition.

[0026] Figure 7 This is the confusion matrix obtained by using the CAM++ method for ship acoustic signature data recognition.

[0027] Figure 8 This is the confusion matrix obtained by using the ECAPA-TDNN method for ship acoustic signature data recognition. Detailed Implementation

[0028] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0029] Example 1

[0030] The proposed method for enhancing and high-precision recognition of small-sample ship acoustic data in this embodiment specifically involves an underwater ship acoustic data enhancement, feature extraction, and classification recognition method under small-sample conditions. This method designs an end-to-end processing flow based on Conv-VAE (Convolutional Variational Autoencoder) to generate enhanced small-sample datasets, that is, using Conv-VAE to generate reconstructed small-sample datasets; using a BEATs encoder with frozen parameters for feature extraction; and calling a LightGBM classifier for classification recognition, namely the ConvVAE-BEATsLGB framework.

[0031] The proposed method for enhancing and accurately recognizing small-sample ship acoustic signature data based on the ConvVAE-BEATsLGB framework consists of three main parts: first, an adaptive effective sound segment detection and slicing (Adaptive VoiceActivity Detection-Cut, AdaVAD-Cut); second, a convolutional variational autoencoder (Conv-VAE); and third, a feature extraction and classification (Bidirectional Encoder representation from Audio Transformers and Light Gradient Boosting, BEATsLGB). The general approach is as follows:

[0032] First, we will introduce the raw audio slices and data cleaning process.

[0033] Considering that the subsequent BEATs encoder uses time-frequency feature maps as input, the original underwater audio is uniformly resampled to 64kHz. Slices are obtained by sliding along the time axis with a 5s window and a 1s step, resulting in fixed-length audio segments. The energy of each segment is calculated, and an adaptive threshold is used to determine valid segments. Silence and low-energy segments are removed, and the remaining qualified segments, i.e., valid audio slices, enter the subsequent dataset construction and feature extraction process. The fixed-length slice design and the subsequent feature extraction stage uniformly use a 128×128 log-Mel spectrogram, matching the input specifications. The 128 frequency dimensions correspond to the number of Mel triangular filters, and the 128 time dimensions correspond to a fixed number of frames, thus ensuring consistency between the audio slices, spectral representation, and downstream model input dimensions throughout the end-to-end process.

[0034] Next, we will introduce the dataset construction and preprocessing.

[0035] The first step is dataset construction.

[0036] In this embodiment, clean audio slices (i.e., valid audio slices) are aggregated by category based on the ShipsEar dataset annotations, and then divided into three datasets based on a sample size threshold for each category: small sample dataset, large sample dataset, and overall dataset. Subsequent reconstructive data augmentation is performed only on the small sample dataset. The large sample dataset is used to merge with the small sample dataset and the regenerated data (i.e., the reconstructed small sample dataset) to participate in classification and recognition. Subsequently, the same set of encoder-decoder parameters is used to compress any voiceprint to a 1024-dimensional latent space before restoring it to avoid overfitting, and all data are used to construct the overall dataset.

[0037] Second, data preprocessing.

[0038] In this embodiment, stratified sampling is used to divide 80% of the samples into a training set and 20% into a validation set. A fixed random seed is used, and the output samples are uniformly processed on a 128×128 log-Mel spectrum, consistent with the subsequent Conv-VAE and BEATs encoders and LightGBM classifier.

[0039] Next, we introduce the construction of a Convolutional Variational Autoencoder (Conv-VAE) to generate new audio data. Conv-VAE is used to reconstruct audio data, which involves four parts: original audio to Mel spectrogram, training, Mel spectrogram reconstruction, and audio reconstruction.

[0040] First, the original audio is converted into a Mel spectrogram.

[0041] In this embodiment, a short-time Fourier transform is performed on each audio segment of the training set obtained from the overall dataset to obtain the power spectrum. Then, a Mel triangular filter bank is used to convert the frequency axis to a Mel scale, and the logarithmic power spectrum is calculated. Next, min-max normalization is applied to the entire Mel spectrogram, compressing all values ​​to between 0 and 1. Finally, the time axis is padded or truncated to 128 frames, forming a fixed-size 128×128 Mel spectrogram, i.e., a fixed-size logarithmic Mel spectrogram.

[0042] Second, train the Convolutional Variational Autoencoder (Conv-VAE).

[0043] Conv-VAE replaces the fully connected encoder-decoder structure used for image modeling in traditional VAE models with convolutional and deconvolutional structures, while maintaining the overall three-part architecture of encoder-latent space-decoder. The encoder consists of several two-dimensional convolutional layers (Conv2d) and ReLU activation functions, compressing the input Mel-frequency spectrogram into a compact high-level feature representation through layer-by-layer downsampling. At the end of the encoder, the output feature map of the fourth convolutional layer is flattened and subjected to an affine transformation to obtain a one-dimensional vector. This one-dimensional vector is used as the input to two parallel fully connected layers, which output the mean vector and log-variance vector of the latent distribution, respectively.

[0044] The decoder first uses a 2D transposed convolutional layer (ConvTranspose2d, i.e., a deconvolutional layer) to reshape the latent vectors into a low-resolution feature map. Then, it progressively upsamples through multiple layers of ConvTranspose2d and ReLU activation functions to restore spatial resolution. At each upsampling stage, the features cached in the decoder are concatenated with the corresponding scale features in the encoder via skip connections to preserve the detailed structural information of the spectrum. The final layer of the decoder uses a Sigmoid activation function to normalize the output spectrum amplitude to the [0,1] interval.

[0045] During the training of the Conv-VAE model, a joint loss function consisting of reconstruction loss and KL divergence regularization term is used. The reconstruction loss constrains the difference between the decoder output spectrum and the original input spectrum, and can be expressed as mean squared error (MSE). The KL divergence term constrains the consistency between the latent spatial distribution and the standard normal distribution, thereby achieving stable and high-quality spectrum reconstruction and generation. A convolutional variational autoencoder with skip connections is trained using this method, compressing the input 128×128 log-Mel spectrogram to a 1024-dimensional latent vector, and reconstructing it into a spectrogram of the same size during the decoding stage. An early stopping strategy is incorporated during training to save the model parameters that achieve the best performance on the validation set. Finally, a weight file containing the complete encoder and decoder parameters is obtained, which is used for subsequent reconstruction of the Mel spectrogram and data augmentation.

[0046] Third, a new Mel spectrogram is generated using the Conv-VAE model.

[0047] The trained Conv-VAE model is used as a reconstruction machine to read all 128×128 Mel spectrum image segments (Mell segments) in batches, compress them into 1024-dimensional latent variables, and then decode them into reconstructed spectra, thereby completing image reconstruction.

[0048] Fourth, reconstruct the audio.

[0049] The generated Mel spectrum is restored to a 64kHz audible waveform using the reverse process. First, inverse normalization is used to bring the [0,1] values ​​of the network output back to the original decibel range. Then, an inverse Mel transform is performed to convert the Mel scale back to a linear scale to obtain the true energy spectrum. Finally, an inverse Mel transform is performed to convert the energy on the Mel scale back to a normal linear frequency scale, resulting in reconstructed audio slices with the same number as the original audio slices. The number of reconstructed slices is the same as the original small sample dataset, but the content is newly generated. This part of the data constitutes the reconstructed small sample dataset, thus achieving data augmentation.

[0050] Next, we introduce the construction of the BEATsLGB model, which includes a BEATs encoder for feature extraction and a LightGBM classifier for classification and recognition.

[0051] The first step is to merge datasets.

[0052] The method of this invention only performs reconstruction enhancement on small sample datasets. If the number of samples in a certain category is still below the threshold, reconstruction can be repeated until the threshold is reached. (The small sample dataset...) Reconstructing small sample datasets and large sample datasets The data is merged to form a new dataset, i.e., the merged dataset. .

[0053] Second, feature extraction.

[0054] The feature extraction process is based on the BEATs encoder, which can utilize the official weight file BEATs_iter3_plus_cpt2.pt. The BEATs encoder employs a self-supervised pre-trained audio Transformer model, consisting of two stages: an encoder and a predictor. The encoder includes patch embedding layers, learnable positional encodings, and 12 Transformer encoding blocks. The predictor performs mean pooling followed by a linear classifier. The raw score vector obtained from the final linear transformation, before Softmax normalization, can be used for audio classification, quality assessment, and multimodal fusion.

[0055] The BEATs encoder model is used only as a frozen feature extractor to extract features from the merged dataset. The data is transformed into fixed-dimensional, high-discrimination embeddings for direct classification by the subsequent LGB model, i.e., the LightGBM classifier.

[0056] Thirdly, model training and classification recognition.

[0057] The LightGBM classifier is a gradient boosting decision tree framework that uses a leaf-wise strategy and histogram algorithm to quickly build deep trees. It excels at achieving high-precision, low-latency regression and classification on small- to medium-sized structured data. The recognition model uses the LightGBM classifier, with a gradient boosting decision tree at its core. It fits the residuals of the previous ensemble by building new trees round by round, and automatically captures nonlinear boundaries using hierarchical sampling and leaf splitting gain. The extracted high-dimensional BEATs (voiceprint vectors), i.e., the voiceprint feature vectors, are mapped to 12 ship probabilities, thus achieving high-accuracy underwater audio classification with small samples. During the training phase, a five-fold hierarchical cross-validation is used. The 768-dimensional feature matrix is ​​divided fold by fold in an 8:2 ratio. Multi-class softmax loss is used as the objective function. Up to 800 decision trees are built on each fold, and early stopping is based on the logarithmic loss of the validation set. The process terminates after 50 rounds if there is no decrease in performance, and the optimal result is saved.

[0058] Finally, we will introduce the part on calculating the confusion matrix.

[0059] Based on the merged dataset, its confusion matrix is ​​calculated. The confusion matrix can quickly show the number of correct recognitions, the number of incorrect recognitions, and the classification accuracy for each category.

[0060] The following is a detailed introduction to the method for enhancing and high-precision recognizing small-sample ship acoustic data proposed in this invention.

[0061] like Figures 1 to 2 As shown, the method for enhancing and high-precision identifying small-sample ship acoustic signature data includes the following steps:

[0062] Step 1. Resample, slide slice, and clean the energy threshold of the original audio of the ship's acoustic signature to obtain effective audio slices. Based on the sample size of each category, the effective audio slices are divided into small sample datasets, large sample datasets, and the overall dataset.

[0063] In step 1, the process of resampling, sliding slicing, and energy threshold cleaning of the original underwater audio of the ship's acoustic signature to obtain the effective audio slice, namely the AdaVAD-Cut slice, is as follows:

[0064] Reading and resampling: All continuous waveforms in the original ship acoustic print audio are uniformly resampled to the target sampling rate to obtain a resampled discrete sequence, which is used for subsequent fixed-length slicing and feature calculation.

[0065] (1)

[0066] in, This represents the discrete signal after resampling. This represents the original continuous waveform. This indicates the target sampling rate, which is 64kHz in this embodiment. . This represents the total number of sample points, i.e., the original audio of the ship's acoustic signature at the target sampling rate. The total number of sampling points.

[0067] Total sample points Represented as:

[0068] .

[0069] in, The original sampling rate, This indicates the original audio of the ship's acoustic signature at the original sampling rate. The total number of sampling points.

[0070] Calculate the average power, i.e., the global energy, of the resampled discrete sequence. Global energy This will serve as the global energy benchmark for determining the validity of subsequent segments:

[0071] (2)

[0072] Among them, global energy This is used to characterize the overall energy level of the entire audio segment and serves as the energy scale reference for subsequent segmented energy statistics and threshold adaptive calculations. All subsequent segmented energy calculations and statistics are performed under this energy scale.

[0073] Define slicing parameters: Preset slicing parameters include window length. and step length And convert it into sample length:

[0074] (3)

[0075] In this embodiment, the window length is... 5s, step size For 1 second, Indicates the length of a slice window. This indicates the step size of each window slide.

[0076] Sliding window slicing: This process involves sliding a window along the time axis to extract 5-second continuous segments, covering the entire audio segment. The resulting 5-second audio segment is then sliced. have:

[0077] (4)

[0078] in, For fragment index, , This represents the maximum number of segments. , Represents the discrete signal after resampling From the middle Each sampling point starts at a length of [number] points. A continuous sampling sequence.

[0079] Maximum number of segments Defined as:

[0080] .

[0081] Valid segment detection and energy filtering: Remove silent segments with too low energy (invalid audio segments), retaining only the valid signals (valid audio segments). Calculate segment energy. for:

[0082] (5)

[0083] in, Used to represent audio segments The segmented energy value.

[0084] Determine the energy threshold according to formula (6). for:

[0085] (6)

[0086] in, It is an adaptive threshold automatically estimated by the data's own statistics. This represents the set of segmented energy values, which consists of the segmented energy values ​​of all audio segments. . Represents a set of segmented energy The mean, Represents a set of segmented energy The standard deviation.

[0087] Energy thresholding is performed on all audio segments obtained from the sliding slice.

[0088] Based on the segment energy value of the current segment Does it meet the requirements? Invalid audio slices are identified.

[0089] Specifically, if the segment energy value of the current segment Below global energy of If the energy value of the current segment is multiple, then the current segment is determined to be an invalid audio segment and discarded. Not lower than global energy of If the value is multiplied by 1, the current segment is determined to be a valid audio slice and is retained.

[0090] Saving qualified segments: In this embodiment, the selected segments, i.e., valid audio slices, are saved at a sampling rate of 64kHz as input for subsequent dataset construction and feature extraction. They are also mapped to a 128×128 log-Mel spectrum to match the input size of the downstream model of the system.

[0091] The specific process of dataset construction and preprocessing is as follows:

[0092] First, let's introduce the dataset construction part.

[0093] Based on the ShipsEar dataset annotations, the clean segments (i.e., valid audio segments) obtained in step 1 are summarized by category, and a threshold is set based on the number of valid segments in each category. In this embodiment, the data samples of eight categories—dredgers, fishing boats, ocean-going liner ships, pilot boats, roll-on / roll-off ships, sailboats, trawlers, and tugboats—are less than 500 segments each, constituting a small sample dataset. The dataset comprises four categories of data samples, ranging from 500 to 4000, including motorboats, mussel boats, environmental noise, and passenger ships, forming a large sample dataset. Total set For including small sample datasets and large sample datasets ,Right now .

[0094] Next, we will introduce the data preprocessing section.

[0095] Training and validation are performed using 8:2 stratified sampling with a fixed random seed to construct the training and validation sets.

[0096] Step 2. Perform short-time Fourier transform and Mel filtering on the effective audio slices from the small sample dataset and the overall dataset to generate fixed-size log-Mel spectrograms. The log-Mel spectrogram generated from the overall dataset is used for subsequent training of the Conv-VAE, while the log-Mel spectrogram generated from the small sample dataset is used for subsequent reconstruction and generation processes.

[0097] Step 2 is as follows:

[0098] Transforming raw audio into Mel spectrograms: converting small sample datasets With the overall dataset Perform a short-time Fourier transform on the effective audio slices in the signal to obtain the signal. With a Hann window added, the m-th frame after windowing is represented as:

[0099] (7)

[0100] in, This represents the complex matrix output by the short-time Fourier transform. This is the frame sequence number, i.e., the time frame index. For frequency Serial number , Frequency representing a single-sided spectrum number. For intra-frame sampling point index, Let the length of the Hann window be _____. . Represents the discrete signal after resampling in an effective audio slice. In the Temporal sampled values ​​of the frame, This is the frame shift length. This is the Hann window function. j represents the imaginary unit. In this embodiment... , , The time frame is 128.

[0101] For complex matrices The power spectral matrix is ​​obtained by taking the modulus of each element and squaring them. for:

[0102] (8)

[0103] Mapping linear frequencies to Mel frequencies:

[0104] (9)

[0105] in, Indicates the linear frequency Nonlinear transformation function mapped to the Mel frequency scale.

[0106] 128 Mel triangular filter centers are placed at equal intervals on the Mel axis, and a weight matrix is ​​constructed.

[0107] For the power spectrum matrix Weighted summation on the Mel frequency scale yields the... The Mel-triangle filter in the first... The output power value over each time frame is denoted as... :

[0108] (10)

[0109] in, This indicates the sequence number of the Mel-triangle filter. , This represents the total number of Mel-triangle filters, in this embodiment. . Indicates the first A Mel-triangle filter at frequency The weighting coefficient at the location.

[0110] The global maximum power value of the Mel power spectrum across all time frames and all Mel triangular filters. For reference, the power spectrum is relativized and converted to decibels:

[0111] (11)

[0112] (12)

[0113] in, As a relativistic reference in formula (11), the value of is made relative to... Centered on. In the Mel-decibel spectral matrix The Middle The Mel filter and the first Mel spectrum decibel values ​​at each time frame. For the first The Mel-triangle filter in the first... Output power over time frames. This represents a stable term used to prevent the denominator from being zero in logarithmic operations. , It is an extremely small amount. This indicates the operation of taking the global maximum power value of the Mel power spectrum across all time frames and all Mel triangular filters.

[0114] Find the minimum and maximum values ​​over the entire Mel-decibel spectrum matrix, linearly stretch it to [0,1], and add a protective bias to obtain:

[0115] (13)

[0116] in, This represents the characteristics of the Mel-decibel spectrum after linear normalization. and Representing the Mel-decibel spectral matrix respectively Maximum and minimum values ​​across all time frames and across all Mel triangular filter dimensions. This represents a stabilizing term, or protective bias, used to prevent the denominator from being zero. .

[0117] To standardize the Mel-spectral feature size for different audio samples, the effective time frame number is determined after feature extraction of the current audio sample. When the number of frames is less than the preset number, zero-padding is performed on the linearly normalized Mel-decibel spectrum features at the end of the time dimension, as defined in formula (14):

[0118] (14)

[0119] in, Indicates the size is The Mel spectrogram units; when the number of effective frames is less than 128, zero padding is performed at the end of the time dimension to unify the feature map size, thereby obtaining the overall dataset. and small sample datasets Mel spectrum.

[0120] Step 3. Conv-VAE Latent Space Reconstruction: Conv-VAE is constructed and trained using the log-Mel spectrogram of the entire dataset. After training, the trained Conv-VAE is used to reconstruct and generate the log-Mel spectrogram of the small sample dataset, resulting in a reconstructed spectrogram. Furthermore, reconstructed audio slices are generated to form the reconstructed small sample dataset.

[0121] In step 3, Conv-VAE includes an encoder section and a decoder section.

[0122] The encoder part of Conv-VAE includes four convolutional layers, which are defined as the first convolutional layer, the second convolutional layer, the third convolutional layer, and the fourth convolutional layer, respectively. The decoder part of Conv-VAE includes four deconvolutional layers, which are defined as the first deconvolutional layer, the second deconvolutional layer, the third deconvolutional layer, and the fourth deconvolutional layer, respectively.

[0123] like Figure 3 As shown, the signal processing flow in Conv-VAE is as follows:

[0124] Read the fixed-size log-Mel spectrogram generated in step 2, and randomly index 16 Mel segments with shapes (1, 128, 128). Then, add a single-channel dimension to each Mel segment and encapsulate the 16 Mel segments in the batch processing dimension. At the same time, normalize the feature values ​​to the range of [0, 1] to obtain a Mel spectrogram feature map with shape (16, 1, 128, 128).

[0125] The encoder part of Conv-VAE takes a single-channel Mel-spectral feature map of shape (16, 1, 128, 128) as input and performs downsampling layer by layer using four convolutional layers. The kernel size of the four convolutional layers is uniformly set to 3×3. The output feature map of each stage in the encoder part of Conv-VAE is buffered for skip connections in the decoder part.

[0126] The first convolutional layer's single-channel Mel-frequency spectral feature map of the input. The output feature map of the first convolutional layer is obtained by processing the data using two-dimensional convolution and downsampling with a stride of 2. The calculation process is expressed as follows:

[0127] (15)

[0128] in, This represents the output feature map of the first convolutional layer. In spatial location The value at that location. This represents the modified linear unit activation function, also known as the ReLU activation function. This represents the weight parameters of the two-dimensional convolution kernel in the first convolutional layer. This represents the spatial index of the convolution kernel in the frequency and time dimensions. This represents the feature map input to the first convolutional layer. In spatial location The value at that location. The first convolutional layer takes the input feature map coordinates. The mapping achieves downsampling with a step size of 2.

[0129] The second convolutional layer processes the input feature map. The second convolutional layer uses the same convolutional structure and downsampling method as the first convolutional layer, namely, two-dimensional convolution and downsampling with a stride of 2, to obtain the output feature map of the second convolutional layer. The calculation process is expressed as follows:

[0130] (16)

[0131] in, This represents the output feature map of the second convolutional layer. In spatial location The value at that location. This indicates the input channel index of the second convolutional layer. In this embodiment, the number of output channels of the second convolutional layer is 32. Indicates the second convolutional layer and the first The two-dimensional convolution kernel weight parameters corresponding to each input channel. This represents the feature map input to the second convolutional layer. In spatial location The value at that location.

[0132] The third convolutional layer processes the input feature map. The output feature map of the third convolutional layer is obtained by processing the data using two-dimensional convolution and downsampling with a stride of 2. The calculation process is expressed as follows:

[0133] (17)

[0134] in, This represents the output feature map of the third convolutional layer. In spatial location The value at that location. This indicates the input channel index of the third convolutional layer. In this embodiment, the number of output channels of the second convolutional layer is 64. Indicates the relationship between the third convolutional layer and the first convolutional layer. The two-dimensional convolution kernel weight parameters corresponding to each input channel. This represents the feature map input to the third convolutional layer. In spatial location The value at that location.

[0135] The fourth convolutional layer processes the input feature map The output feature map of the fourth convolutional layer is obtained by processing the data using two-dimensional convolution and downsampling with a stride of 2. The calculation process is expressed as follows:

[0136] (18)

[0137] The tensor shape at this point is (B,256,8,8). This represents the output feature map of the fourth convolutional layer. In spatial location The value at that location. This indicates the input channel index of the fourth convolutional layer. In this embodiment, the number of output channels of the second convolutional layer is 128. Indicates the fourth convolutional layer and the first The two-dimensional convolution kernel weight parameters corresponding to each input channel. This represents the feature map input to the fourth convolutional layer. In spatial location The value at that location.

[0138] Skip features in the encoder partial buffer of Conv-VAE Including the output feature maps of each convolutional layer , , , .

[0139] Feature flattening: flattening the feature map By flattening and affine transformation, a one-dimensional vector is obtained. .

[0140] Latent distribution mean estimation and latent distribution variance estimation: estimating a one-dimensional vector... As input to two parallel fully connected layers, the output mean and log-variance are:

[0141] (19)

[0142] in, Represents the potential mean vector. This represents the latent log-variance vector. and This represents the weight of the fully connected component, which is (1024, 16384) in this embodiment. and For bias, in this embodiment it is (1024).

[0143] By reparameter sampling, noise of the same shape is extracted from the standard normal distribution:

[0144] (20)

[0145] in, This represents a random noise vector sampled from a standard normal distribution. This indicates that the mean is zero and the covariance is... The 1024-dimensional standard normal distribution, It is a 1024-dimensional unit covariance matrix.

[0146] Based on the reparameterization technique, the potential mean vector With the latent log-variance vector Decoupling is achieved by introducing standard normal noise. To achieve random sampling, while ensuring randomness, backpropagation becomes differentiable, thereby obtaining the latent vector. :

[0147] (twenty one)

[0148] in, , This represents the batch size, which is the number of Mel spectrum samples simultaneously input into the convolutional neural network during one forward propagation. This is the standard deviation vector. This is element-wise multiplication.

[0149] Latent vector feature mapping: latent vector After mapping by a fully connected layer, it is reshaped into a feature map of size 8×8. .

[0150] feature map With the output feature map of the fourth convolutional layer In channel-dimensional splicing, the process is represented as follows:

[0151] (twenty two)

[0152] in, and These represent the bias parameters and weight parameters of the fully connected layer, respectively, in this embodiment. The value is (16384, 1024). This represents the operation of reshaping a vector into a feature map of shape (B,256,8,8). This represents the initial feature map obtained after fully connected mapping and reshaping. This indicates the Skip concatenation operation, which is a feature concatenation operation in the channel dimension. Indicated by feature map and The intermediate feature map obtained after the Skip stitching operation.

[0153] The input to the first deconvolutional layer is the intermediate feature map. The signal processing in the first deconvolution layer is represented as follows:

[0154] (twenty three)

[0155] in, This represents the output feature map of the first deconvolutional layer. In spatial location The value at that location. This represents the input channel index of the first deconvolution layer. In this embodiment, the number of output channels of the first deconvolution layer is 256. Indicates the first deconvolution layer and the... The deconvolution kernel weight parameters corresponding to each input channel. This represents the feature map input to the first deconvolution layer. In spatial location The value at that location.

[0156] The input to the second deconvolutional layer is the intermediate feature map. intermediate feature map It is the output feature map of the first deconvolution layer and the output feature map of the third convolutional layer The processing steps obtained after the Skip concatenation operation are as follows:

[0157] (twenty four)

[0158] The output shape is (B, 64, 32, 32). This represents the deconvolution operation of the second deconvolution layer. This represents the output feature map of the second deconvolution layer after deconvolution and ReLU activation.

[0159] The input to the third deconvolutional layer is the intermediate feature map. intermediate feature map It is the output feature map of the second deconvolution layer Output feature map of the second convolutional layer The processing steps obtained after the Skip concatenation operation are as follows:

[0160] (25)

[0161] The output shape is (B,32,64,64). This represents the deconvolution operation of the third deconvolution layer. This represents the output feature map of the third deconvolution layer after deconvolution and ReLU activation.

[0162] The input to the fourth deconvolutional layer is the intermediate feature map. intermediate feature map It is the output feature map of the third deconvolution layer and the output feature map of the first convolutional layer The processing steps obtained after the Skip concatenation operation are as follows:

[0163] (26)

[0164] The output shape is (B,1,128,128). This represents the deconvolution operation of the fourth deconvolution layer, used to map the feature map to the input Mel spectrum. Figure 1 The resulting spatial resolution and channel dimensions. This represents the Sigmoid activation function, used to map output features to... Interval. This represents the output feature map of the fourth deconvolution layer after deconvolution and Sigmoid activation, i.e., the reconstructed spectrogram.

[0165] Step 3, specifically the process of training Conv-VAE using the log-Mel spectrogram of the entire dataset, is as follows:

[0166] Based on the overall dataset Building the training set and verification set During the training process of Conv-VAE, each complete training iteration, i.e., on the training set... After completing a parameter update, in the validation set The validation loss is calculated above and used to monitor the convergence status of the model.

[0167] Validation set Total of Mel spectrum samples, or Mel segments, are randomly selected. The Mel-ray spectral samples are batch-packaged to obtain the verification batch input. :

[0168] (27)

[0169] Keeping the network parameters frozen, i.e., not updating them during the validation phase, allows the forward propagation to obtain the reconstructed latent log-variance of the validation set. With potential distribution parameters.

[0170] Calculate the batch loss, i.e., the mean square error of the validation set reconstruction. for:

[0171] (28)

[0172] in, , , This indicates the first input in the batch verification process. Mel spectrum samples in frequency dimension index Time-dimensional index The original Mel-spectral eigenvalues ​​at that location. Indicates the first element in the Conv-VAE output reconstruction spectrum. Mel fragment in frequency dimension index Time-dimensional index The reconstructed Mel spectrum eigenvalues ​​at the location.

[0173] Calculate the KL term, i.e., the KL divergence of the latent space of the validation set. for:

[0174] (29)

[0175] in, , and Representing the validation set The Middle Mel spectrum samples in the first The latent mean and standard deviation over the latent space are given. In this embodiment, the latent mean of the verification set is (B, 1024), and the logarithmic variance of the latent set is (B, 1024).

[0176] Calculate the joint loss of the validation set. for:

[0177] (30)

[0178] in, These are the weighting coefficients for the KL term.

[0179] Calculate the joint loss of the training set for:

[0180] (31)

[0181] in, To reconstruct the mean squared error for the training set, Let KL divergence be the latent space of the training set.

[0182] During the training of Conv-VAE, the joint loss of the validation set is continuously monitored. When the condition shown in formula (32) is met, that is, the joint loss of the validation set in the current training cycle is not less than the minimum value of the joint loss of the validation set in the previous 5 training cycles, it is determined that the performance of Conv-VAE on the validation set has not continued to improve, and the training adjustment strategy is executed.

[0183] (32)

[0184] in, Indicates the index of the current training cycle. Indicates the index of historical training cycles. . Indicates the first The joint loss value calculated on the validation set at the end of each training cycle. Indicates the first The joint loss value calculated on the validation set at the end of each training cycle.

[0185] The specific training and adjustment strategies are as follows:

[0186] The learning rate is decayed and Conv-VAE is continued to be trained.

[0187] If the condition shown in formula (32) is still satisfied after the learning rate decays, that is, in the current training cycle Joint loss of the validation set If the score does not drop, the current training process will be terminated early.

[0188] The final trained Conv-VAE is denoted as conv_vae1024.pth.

[0189] In step 3, the log-Mel spectrogram of the small sample dataset is reconstructed and generated using the trained Conv-VAE. The specific process of obtaining the reconstructed spectrogram is as follows:

[0190] By scanning the Mel spectrogram sequence corresponding to the same audio segment in the small sample dataset one by one, a single Mel spectrogram segment is obtained, and its overall tensor shape is ( ,128,128), of which This indicates the number of Mel spectrogram frames obtained by dividing the same audio segment.

[0191] The single-segment Mel spectrogram is fed into the encoder section of the Conv-VAE to obtain the latent spatial statistics:

[0192] (33)

[0193] in, The input is a single-segment Mel spectrogram sample with a size of 128×128. and It is a fully connected layer.

[0194] Then, reparameter sampling is performed to obtain the latent vector. :

[0195] (34)

[0196] Conv-VAE decoder section Using the latent vector z and the skip features partially cached by the encoder Reconstruct the Mel spectrogram of the small sample dataset to obtain the reconstructed spectrogram. :

[0197] (35)

[0198] The output shape strictly matches the input, and the numerical range is... Thus, the Mel spectrogram of the small sample dataset was reconstructed.

[0199] In step 3, reconstructed audio slices are generated based on the reconstructed spectrogram output by Conv-VAE, thus forming a reconstructed small sample dataset. Specifically, the log-Mel spectrograms corresponding to the small sample dataset are reconstructed using the trained Conv-VAE to obtain the reconstructed Mel spectrograms. The process of reconstructing the audio signal based on this is as follows:

[0200] Reconstructed spectrogram based on the decoder output in Conv-VAE The normalized reconstructed Mel spectrum magnitude matrix is ​​obtained. Its numerical range is For the reconstructed Mel spectrum amplitude matrix Perform inverse normalization to map it back to the decibel scale, and you will get the inverse normalized Mel-spectrum decibel matrix. :

[0201] (36)

[0202] in, and In this embodiment, the preset linear mapping coefficients are used. , .

[0203] Converting the logarithmic scale to linear power yields the Mel power spectrum. :

[0204] (37)

[0205] in, In the Mel-decibel spectral matrix The Middle The Mel filter and the first Mel-frequency spectral decibel values ​​at each time frame. The decibel values ​​are expressed in power decibel form, and their conversion to linear power is as follows: . This is a preset constant bias term used to avoid numerical instability issues arising from logarithmic or exponential operations during numerical calculations. It is an extremely small amount.

[0206] Using the inverse Mel filter matrix Mel power spectrum Perform weighted summation to reconstruct the linear short-time Fourier transform (STFT) power spectrum. :

[0207] (38)

[0208] in, For Fourier transform index, , This indicates the number of Fast Fourier Transform (FFT) points used in the Short-Time Fourier Transform. In this embodiment... Then the number of single-sided frequency points is Inverse Mel filter matrix Its shape is 513×128.

[0209] Known STFT power spectrum However, the phase is missing. The phase is estimated using the Griffin-Lim algorithm after 64 iterations.

[0210] Initialize the random phase matrix .

[0211] Each iteration constructs a complex spectrum, performs an inverse STFT to obtain the time-domain signal, and then obtains a new phase through a forward transform:

[0212] (39)

[0213] in, It is a linear frequency power spectrum.

[0214] Known reconstructed power spectrum However, the phase is missing, so the Griffin-Lim algorithm is used to estimate the phase:

[0215] Initialize the random phase matrix And iteratively update:

[0216] (40)

[0217] (41)

[0218] (42)

[0219] in, , Indicates the first The complex spectrum of STFT obtained from the next iteration. For amplitude spectrum, . For the first The phase obtained in the next iteration For the first The phase obtained in the next iteration. For the first The time-domain signal obtained in the next iteration. This represents the inverse short-time Fourier transform operator. This represents the Hann window function. This represents the operation of extracting phase from the complex spectrum of an STFT.

[0220] When the phase converges or the number of iterations reaches 64, data augmentation is completed, and the time-domain signal obtained from the final iteration is used as a reconstructed audio slice to reconstruct the small sample dataset. It consists of several reconstructed audio slices.

[0221] Step 4. BEATs Freeze Feature Extraction: Merge the small sample dataset, the reconstructed small sample dataset, and the large sample dataset. Use the BEATs encoder with frozen parameters to extract features from the merged dataset to obtain the voiceprint feature vector.

[0222] The method of this invention only performs reconstruction enhancement on small sample datasets. If the number of samples of a certain category in the reconstructed small sample dataset obtained in step 3 is still lower than the threshold, reconstruction can be repeated once until the threshold is reached. The reconstructed small sample dataset with the number of samples of each category reaching the threshold is then used to merge with the small sample dataset and the large sample dataset.

[0223] In this embodiment, BEATs are only used as frozen feature extractors, and their linear classification heads are not used. The output 768-dimensional global embedding, i.e., the voiceprint feature vector, is used for subsequent classification. Step 4 is as follows:

[0224] small sample dataset Reconstructing small sample datasets and large sample datasets Merge the datasets to obtain the merged dataset. :

[0225] (43)

[0226] The union operation represents merging three types of data at the sample level. The merged dataset... The samples in the dataset undergo uniform preprocessing, converting them into log-Mel spectrograms with a sampling rate of 64kHz and a size of 128×128. The preprocessing process includes short-time Fourier transform and Mel spectrum mapping.

[0227] Merged dataset Perform a short-time Fourier transform on the effective audio slices in the signal, and then process the signal obtained from the short-time Fourier transform. Add a Hann window, the first window added after adding the window A frame is represented as:

[0228] (44)

[0229] in, Indicates the first The time-domain frame signal after windowing. , This represents the total number of frames. Indicates signal In the The temporal sample value of the frame. In this embodiment, during the framing process shown in formula (44), the frame length is 1024 and the frame shift is 512.

[0230] Total frames Represented as:

[0231] (45)

[0232] in, This represents the total number of samples in the resampled signal.

[0233] Perform a Discrete Fourier Transform (DFT) on each frame of the signal to obtain the complex spectrum. for:

[0234] (46)

[0235] By splicing together all frames, a two-dimensional spectral matrix is ​​obtained. :

[0236] (47)

[0237] in, It represents 513 frequencies. and A complex matrix space consisting of time frames.

[0238] For complex spectrum Square the modulus to obtain the power spectrum. :

[0239] (48)

[0240] in, Indicates the first Frame in frequency index The power spectral density at that location.

[0241] By concatenating the power spectra of all time frames, we obtain the linear frequency power spectrum matrix, which is the overall power spectrum matrix. :

[0242] .

[0243] Construct 128 Mel-triangle filters, with center frequencies uniformly distributed according to the Mel scale:

[0244] (49)

[0245] (50)

[0246] in, Indicates the first A Mel triangular filter at frequency index The weighting coefficient at the location. Indicated in frequency index The linear frequency at that point. For the first The center frequency of each filter, This indicates the interval between the center frequencies of adjacent filters.

[0247] All filter weights The matrix of Mel triangular filter banks , .

[0248] The overall power spectral matrix With Mel triangular filter bank matrix Performing matrix multiplication and taking the logarithm yields the final log-Mel spectrum matrix, i.e., the log-Mel spectrum diagram. Represented as:

[0249] (51)

[0250] Through matrix multiplication, i.e. We perform weighted convergence of the linear frequency power spectrum on the Mel frequency scale to obtain the Mel frequency power spectrum, then take the natural logarithm and add a very small constant. To avoid numerical instability, the final log-Mel spectrum matrix is ​​obtained. In this embodiment, in formula (51) M can be directly input into the patch embedding layer of the BEATs encoder model.

[0251] like Figure 4 As shown, the signal processing flow in the BEATs encoder is as follows:

[0252] The final log-Mel spectrum matrix Input to the patch embedding layer of the BEATs encoder.

[0253] The official weight file BEATs_iter3_plus_cpt2.pt is injected, and the original prediction header parameters are removed during loading, retaining only the encoder part.

[0254] Time-frequency block feature embedding: the final log-Mel spectrum matrix Treating it as a single-channel image, after processing by the patch embedding layer of the BEATs encoder, two-dimensional sequence features are obtained:

[0255] (52)

[0256] in, This represents the token sequence output by the patch embedding layer. In this embodiment, its size is 16×16, the step size is 16, and the channel dimension is 768. This represents the patch embedding operation, which divides the input feature map into parts of size 1. A patch with a step size of 16 is used, and each patch is projected to a 768-dimensional channel space through a linear mapping.

[0257] Location information encoding: for Add learnable location coding.

[0258] Multi-layer self-attention feature modeling: adding learnable positional encodings It is then fed into a 12-layer bidirectional Transformer encoding block for processing:

[0259] (53)

[0260] in, , Each Transformer encoding block contains a multi-head self-attention module and a feedforward network, with the hidden dimension maintained at 768. Indicates the first The output of the layer Transformer coded block, Indicates the first The output of the layer Transformer encoded block.

[0261] Sequence Feature Aggregation: Output of the 12th Layer Transformer Encoding Block The final hidden state is then averaged along the sequence dimension to obtain the merged dataset. 768-dimensional global voiceprint vector :

[0262] (54)

[0263] in, This indicates the operation of averaging along the time series dimension. The frozen global voiceprint vector, i.e., the voiceprint feature vector, is the audio slice. It will no longer participate in gradient updates.

[0264] All parameters of the BEATs encoder are frozen, including the weights of the patch embedding layer, positional encoding, and the 12 Transformer encoding blocks. They do not participate in gradient updates and are executed only once in the forward pass.

[0265] The merged dataset is processed by the BEATs encoder with frozen parameters. Feature extraction yields a 768-dimensional voiceprint feature vector for each sample. And composed of all voiceprint feature vectors This forms the feature matrix X. In this embodiment, the final dataset is obtained. 768-dimensional universal voiceprint vector and The feature matrix is ​​768 × 10⁸, which can be directly loaded into the subsequent LightGBM classifier model.

[0266] Step 5. LightGBM hierarchical cross-validation and output: Using the voiceprint feature vector as input, train and call the LightGBM classifier to perform classification and recognition, and output the recognition result of the ship category.

[0267] The voiceprint feature vector g can be input into a linear classifier to obtain 12 classes of logits, i.e., 12 classes of ship labels:

[0268] .

[0269] in, Represents 12 types of logits; For the weights of the linear classifier, ; For classifier bias, ; using the argmax function to The process is performed to determine the ship category.

[0270] In this embodiment, step 5 specifically includes:

[0271] The dataset used for classification and recognition loads a feature matrix X and its label y extracted by the BEATs encoder, where:

[0272] (55)

[0273] in, This represents the total number of samples in the dataset used for classification and recognition. The sample size is greater than 6000, and the sample size of each class is greater than 500.

[0274] The dataset used for classification and recognition is divided into training and testing sets, and 5-fold stratified cross-validation is used to ensure that the proportion of each class in each fold is consistent with the proportion of classes in the overall dataset. Fold to obtain training index With validation index The corresponding datasets are denoted as follows: and .

[0275] like Figure 5 As shown, the specific processing flow of the signal in the LightGBM classifier is as follows:

[0276] The LightGBM classifier takes voiceprint feature vectors as input and uses an additive model to build a decision tree ensemble.

[0277] (56)

[0278] in, , , Indicates from the previous The weighted result of the decision trees The predicted score vector of the i-th sample, i.e. Additive model for the first The predicted score vector for each sample. For the first The prediction function of a decision tree. Indicates the first Voiceprint feature vectors extracted from each sample by the BEATs encoder.

[0279] For multi-class classification tasks, multi-class softmax loss is used:

[0280] (57)

[0281] in, The cross-entropy loss function represents the multi-class classification task. , Indicates category index, This indicates the total number of categories. The predicted score vector is represented at the th... The softmax probability output on the class is a normalization function that maps the real-valued score vector of the model output to a probability distribution. It can convert the predicted scores of each class into probability values ​​belonging to each class. Its output components are non-negative and sum to 1. This indicates a conditional indicator function, which is defined if and only if the first... The true label of the sample is the first... Time-like functions The value is 1, otherwise the function The value is 0, condition indicator function This is used to ensure that cross-entropy loss only penalizes the predicted probability corresponding to the true class.

[0282] Calculate the first and second gradients before each boost round:

[0283] (58)

[0284] in, It is the first-order gradient. It is a second-order gradient. This represents the current predicted probability.

[0285] For any candidate split point s, the first The dimensional feature is divided into a left subset L and a right subset R, and the split gain is defined. for:

[0286] (59)

[0287] in, and This represents the regularization coefficient of the LightGBM classifier. and These represent the left child node at the [number]th position. The sum of the first and second gradients on the class. and These represent the right child node at the [number]th position. The sum of the first and second gradients on the class.

[0288] Calculate the gradient statistic of the subset:

[0289] (60)

[0290] in, and They represent the first The cumulative statistics of the first and second gradients of the class at the current node.

[0291] like If so, then the split will be executed.

[0292] The weights of the new leaf are given by the closed-form solution:

[0293] (61)

[0294] in, and These are the weights of the left leaf and the right leaf, respectively.

[0295] For any sample, the feature threshold is determined layer by layer according to the current tree partitioning rule, and the sample is routed to a certain leaf node. The leaf node has a pre-stored vector of leaf weights. Once a sample falls into a leaf node, the leaf weight is adjusted. The output of this tree is exhausted.

[0296] Record No. The integrated output after the wheel upgrade is Then a single-round update is represented as:

[0297] (62)

[0298] in, For the first Additive model for the first The predicted score of each sample, i.e., the updated predicted score. Indicates the first Additive model for the first The predicted score of each sample, which is the output of the previous additive model. Indicates the first The leaf weight of each sample falling into a leaf node.

[0299] In this embodiment, the maximum number of trees during the training of the LightGBM classifier is 800; the multi-class log loss on the validation set stops early if it fails to decrease (converges) after 50 rounds. The regularization coefficient of the LightGBM classifier... and The learning rate is 0.1, the subsampling and column sampling rates are both 0.8, the learning rate is 0.05, and the number of leaves is no more than 63.

[0300] For any 768-dimensional input voiceprint feature vector ,through Wheel lift (total) After (trees), the score vectors for each category are:

[0301] (63)

[0302] in, This indicates that the input voiceprint feature vector is at the th... Predicted scores on the class For the first Decision trees for samples in the th Prediction functions on classes.

[0303] The class probabilities are obtained by normalization using the Softmax function. :

[0304] (64)

[0305] in, Indicates the sample at the 1st Predicted scores on the class .

[0306] The final decision, i.e., the predicted class, is obtained from the LightGBM classifier. for:

[0307] (65)

[0308] in, This indicates the operation of retrieving the category index corresponding to the highest category probability.

[0309] Prediction Category This refers to the ship category identification result output by the LightGBM classifier.

[0310] After training is complete, the last fold or weighted average classifier weight file object is serialized and saved; during the deployment phase, it can be loaded to reproduce the complete inference process of the formula without retraining.

[0311] Based on the general representation obtained by freezing the BEATs encoder, the method of this invention uses the LightGBM classifier to complete the high-precision discrimination of 12 types of original ship audio, with an average accuracy of over 95.5%.

[0312] In step 5 of this embodiment, the calculation of the confusion matrix is ​​further included, and the specific process is as follows:

[0313] Statistical analysis is performed on all samples in the test set obtained from the dataset partitioned for classification and recognition. The samples are then paired and counted according to their true and predicted classes to form... square array That is, the confusion matrix, and the elements in the confusion matrix. Represented as:

[0314] (66)

[0315] in, , This represents the total number of samples in the test set obtained from the dataset partitioning used for classification and recognition. Indicates the first The true label of each sample Indicates the first Predicted labels for each sample. It is an indicator function if and only if the first... The true label of the sample is the first... The class and the real label is the first When class, function The value is 1, otherwise the function The value is 0. Indicates the true category is the first Class and predicted category is the first The number of samples for each class, the number of correct identifications for each class corresponding to the diagonal elements in the confusion matrix, and the number of incorrect identifications for each class corresponding to the off-diagonal elements.

[0316] Figure 6 , Figure 7 and Figure 8 The confusion matrices obtained using the ConvVAE-BEATsLGB method, CAM++ method, and ECAPA-TDNN method of this invention were compared respectively. After weighting the diagonal recall rate with the number of samples in each class, the prediction accuracies were 95.56%, 92.3%, and 90.2%, respectively. Prediction accuracy is the weighted accuracy. The specific calculation method is as follows:

[0317] (67)

[0318] in, For the first Class recall rate For the first The number of samples per class. It can be seen that the ConvVAE-BEATsLGB method proposed in this invention significantly outperforms the existing mainstream ECAPA-TDNN and CAM++ methods on the same underwater ship audio evaluation dataset. Using the same five-fold hierarchical cross-validation configuration on the ShipsEar dataset, the average classification accuracy of this invention is improved by approximately 5.3% compared to the ECAPA-TDNN method and by approximately 3.3% compared to the CAM++ method. Furthermore, the single-line inference latency of the ConvVAE-BEATsLGB method proposed in this invention is less than 1ms, and the model storage size is less than 500MB, both significantly smaller than the comparative models.

[0319] Compared with existing methods, the method of the present invention has the following advantages:

[0320] First, the method enhances the generated model data. It introduces Conv-VAE skip connections at the Mel spectrum level for latent space reconstruction, compressing the 128×128 spectrum to a 1024-dimensional heavy parameter distribution. This is followed by symmetric deconvolution and Skip-Concat decoding of the same-level encoded features, generating new samples of the same size as the original spectrum. Unlike CAM++ and ECAPA-TDNN methods, which only mask perturb the waveform or spectrum, this method performs distribution sampling on the latent manifold, generating hundreds of new spectra at once. The sample size can be expanded to approximately twice the original small sample set without additional manual annotation. This allows for the achievement of intra-class dispersion with a small sample size, significantly alleviating the overfitting problem of subsequent classifiers.

[0321] Secondly, regarding data preprocessing, this invention addresses the issues of low signal-to-noise ratio and inconsistent segment lengths in the ShipsEar underwater audio dataset by proposing the AdaVAD-Cut strategy for data cleaning. First, a global energy threshold is used for initial screening. Then, segments are cut in 5-second increments and sliding 1-second intervals. Segments shorter than 5 seconds are padded with zeros in the latent space, while excessively long segments are truncated. Finally, a uniform 128×128 Mel spectrum is output. Compared to existing technologies that directly discard short segments or pad with zeros, this invention avoids energy abrupt changes during the slicing stage and prevents silent segments from being mixed into the training set. This ensures that subsequent Conv-VAE and BEATs encoder models operate on clean data, thereby improving the consistency between enhancement and recognition.

[0322] Third, the recognition model is faster and less resource-intensive. Compared with conventional end-to-end fine-tuning, this invention freezes the official weights of the BEATs model as a whole and uses it only as a 768-dimensional universal voiceprint extractor. The back end is connected to the gradient boosting tree of the LightGBM classifier model. The number of trainable parameters is less than 0.1M. While improving the accuracy by 3% to 5%, it achieves minute-level training and millisecond-level inference.

[0323] Of course, the above description is only a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. It should be noted that any equivalent substitutions or obvious modifications made by those skilled in the art under the guidance of this specification fall within the scope of this specification and should be protected by the present invention.

Claims

1. A method for enhancing and high-precision identifying small sample ship acoustic signature data, characterized in that, Includes the following steps: Step 1. Resample, slide slice, and clean the energy threshold of the original audio of the ship's acoustic signature to obtain effective audio slices. Based on the sample size of each category, the effective audio slices are divided into small sample datasets, large sample datasets, and the overall dataset. Step 2. Perform short-time Fourier transform and Mel filtering on the small sample dataset and the effective audio slices in the overall dataset to generate a fixed-size log-Mel spectrogram; Step 3. Construct and train the Convolutional Variational Autoencoder (Conv-VAE). Train the Conv-VAE using the log-Mel spectrogram of the entire dataset. After training, use the trained Conv-VAE to reconstruct and generate the log-Mel spectrogram of the small sample dataset to obtain the reconstructed spectrogram. Further generate reconstructed audio slices to form the reconstructed small sample dataset. Step 4. Merge the small sample dataset, the reconstructed small sample dataset, and the large sample dataset. Use the BEATs encoder with frozen parameters to extract features from the merged dataset to obtain the voiceprint feature vector. Step 5. Using the voiceprint feature vector as input, train and call the LightGBM classifier to perform classification and recognition, and output the recognition result of the ship category; In step 1, the process of resampling, sliding slicing, and energy threshold cleaning of the original ship acoustic signature to obtain an effective audio slice is as follows: All continuous waveforms in the original ship acoustic print audio are uniformly resampled to the target sampling rate to obtain the resampled discrete sequence: (1) in, This represents the discrete signal after resampling; Represents the original continuous waveform; Indicates the target sampling rate; ; This represents the total number of sample points, i.e., the original audio of the ship's acoustic signature at the target sampling rate. The total number of sampling points; Total sample points Represented as: ； in, The original sampling rate, This indicates the original audio of the ship's acoustic signature at the original sampling rate. The total number of sampling points; Calculate the average power, i.e., the global energy, of the resampled discrete sequence. for: (2) Among them, global energy Used to characterize the overall energy level of the entire audio segment; Preset slicing parameters, including window length and step length And convert it into sample length: (3) in, Indicates the length of the slice window. Indicates the step size of each window slide; The resampled discrete sequence is processed by sliding slicing, extracting continuous segments along the time axis to cover the entire audio segment. The audio segments obtained from the sliding slices... have: (4) in, For fragment index, , This represents the maximum number of segments. , Represents the discrete signal after resampling From the middle Each sampling point starts at a length of [number] points. A continuous sampling sequence; Maximum number of segments Defined as: ； Calculate fragment energy for: (5) in, Used to represent audio segments The segmented energy value; Determine the energy threshold according to formula (6). for: (6) in, This represents the set of segmented energy values, which consists of the segmented energy values ​​of all audio segments. ; Represents a set of segmented energy The mean, Represents a set of segmented energy Standard deviation; Energy thresholding is performed on all audio segments obtained from the sliding slice. If the segment energy value of the current segment Below global energy of If the energy value of the current segment is multiple, then the current segment is determined to be an invalid audio segment and discarded; if the segment energy value of the current segment is... Not lower than global energy of If the value is multiplied by 1, the current segment is determined to be a valid audio slice and is retained.

2. The method for enhancing and high-precision identifying small-sample ship acoustic signature data according to claim 1, characterized in that, Step 2 specifically involves: small sample dataset With the overall dataset Perform a short-time Fourier transform on the effective audio slices in the signal to obtain the signal. With a Hann window added, the m-th frame after windowing is represented as: (7) in, This represents the complex matrix output by the short-time Fourier transform. The frame number is the time frame index; For frequency Serial number , Frequency representing a single-sided spectrum number; For intra-frame sampling point index, Let the length of the Hann window be _____. ; Represents the discrete signal after resampling in an effective audio slice. In the Temporal sampled values ​​of the frame, This is the frame shift length; Here, j represents the Hann window function; j represents the imaginary unit. For complex matrices The power spectral matrix is ​​obtained by taking the modulus of each element and squaring them. for: (8) Mapping linear frequencies to Mel frequencies: (9) in, Indicates the linear frequency Nonlinear transformation function mapped to the Mel frequency scale; Place 128 Mel triangular filter centers at equal intervals along the Mel axis and construct a weight matrix; For the power spectrum matrix Weighted summation on the Mel frequency scale yields the... The Mel-triangle filter in the first... The output power value over each time frame is denoted as... : (10) in, This indicates the sequence number of the Mel-triangle filter. , This represents the total number of Mel-triangle filters; Indicates the first A Mel-triangle filter at frequency Weighting coefficients at each location; The global maximum power value of the Mel power spectrum across all time frames and all Mel triangular filters. For reference, the power spectrum is relativized and converted to decibels: (11) (12) in, In the Mel-decibel spectral matrix The Middle The Mel filter and the first Mel spectrum decibel values ​​at each time frame; For the first The Mel-triangle filter in the first... Output power over each time frame; This represents a stable term used to prevent the denominator from being zero in logarithmic operations. ; This represents the operation of taking the global maximum power value of the Mel power spectrum across all time frames and all Mel triangular filters; Find the minimum and maximum values ​​over the entire Mel-decibel spectrum matrix, linearly stretch it to [0,1], and add a protective bias to obtain: (13) in, This represents the characteristics of the Mel-decibel spectrum after linear normalization; and Representing the Mel-decibel spectral matrix respectively Maximum and minimum values ​​across all time frames and all dimensions of the Mel triangular filter; This represents a stabilizing term, or protective bias, used to prevent the denominator from being zero. ; To standardize the Mel-spectral feature size for different audio samples, the effective time frame number is determined after feature extraction of the current audio sample. When the number of frames is less than the preset number, zero-padding is performed on the linearly normalized Mel-decibel spectrum features at the end of the time dimension, as defined in formula (14): (14) in, Indicates the size is Mel spectrogram units, i.e., the generated small sample dataset With the overall dataset The fixed-size log-Mel spectrum.

3. The method for enhancing and high-precision identifying small-sample ship acoustic signature data according to claim 2, characterized in that, In step 3, Conv-VAE includes an encoder section and a decoder section; The encoder part of Conv-VAE includes four convolutional layers, which are defined as the first convolutional layer, the second convolutional layer, the third convolutional layer, and the fourth convolutional layer, respectively. The decoder part of Conv-VAE includes four deconvolutional layers, which are defined as the first deconvolutional layer, the second deconvolutional layer, the third deconvolutional layer, and the fourth deconvolutional layer, respectively. The specific processing flow of signals in Conv-VAE is as follows: Read the fixed-size log-Mel spectrogram generated in step 2, and randomly index 16 Mel segments with shapes (1, 128, 128). Then add a single-channel dimension to each Mel segment, and encapsulate the 16 Mel segments in the batch processing dimension. At the same time, normalize the feature values ​​to the range of [0, 1] to obtain a Mel spectrogram feature map with shape (16, 1, 128, 128). The encoder part of Conv-VAE takes a Mel spectrum feature map of shape (16,1,128,128), i.e. a single-channel Mel spectrum feature map, as input, and uses four convolutional layers to perform downsampling layer by layer. The kernel size of the four convolutional layers is uniformly set to 3×3. The first convolutional layer's single-channel Mel-frequency spectral feature map of the input. The output feature map of the first convolutional layer is obtained by processing the data using two-dimensional convolution and downsampling with a stride of 2. The calculation process is expressed as follows: (15) in, This represents the output feature map of the first convolutional layer. In spatial location The value at; This represents the modified linear unit activation function, i.e., the ReLU activation function; This represents the two-dimensional convolution kernel weight parameters of the first convolutional layer; This represents the spatial index of the convolution kernel in the frequency and time dimensions; This represents the feature map input to the first convolutional layer. In spatial location The value at; The second convolutional layer processes the input feature map. The output feature map of the second convolutional layer is obtained by processing the data using two-dimensional convolution and downsampling with a stride of 2. The calculation process is expressed as follows: (16) in, This represents the output feature map of the second convolutional layer. In spatial location The value at; Indicates the input channel index of the second convolutional layer; Indicates the second convolutional layer and the first Two-dimensional convolution kernel weight parameters corresponding to each input channel; This represents the feature map input to the second convolutional layer. In spatial location The value at; The third convolutional layer processes the input feature map. The output feature map of the third convolutional layer is obtained by processing the data using two-dimensional convolution and downsampling with a stride of 2. The calculation process is expressed as follows: (17) in, This represents the output feature map of the third convolutional layer. In spatial location The value at; Indicates the input channel index of the third convolutional layer; Indicates the relationship between the third convolutional layer and the first convolutional layer. Two-dimensional convolution kernel weight parameters corresponding to each input channel; This represents the feature map input to the third convolutional layer. In spatial location The value at; The fourth convolutional layer processes the input feature map The output feature map of the fourth convolutional layer is obtained by processing the data using two-dimensional convolution and downsampling with a stride of 2. The calculation process is expressed as follows: (18) in, This represents the output feature map of the fourth convolutional layer. In spatial location The value at; This represents the input channel index of the fourth convolutional layer; Indicates the fourth convolutional layer and the first Two-dimensional convolution kernel weight parameters corresponding to each input channel; This represents the feature map input to the fourth convolutional layer. In spatial location The value at; Skip features in the encoder partial buffer of Conv-VAE Including the output feature maps of each convolutional layer , , , ; Output feature map of the fourth convolutional layer By flattening and affine transformation, a one-dimensional vector is obtained. ; a one-dimensional vector As input to two parallel fully connected layers, the output mean and log-variance are: (19) in, Represents the potential mean vector. Represents the latent log-variance vector; and Indicates the weights of the fully connected layer. and For bias; By reparameter sampling, noise of the same shape is extracted from the standard normal distribution: (20) in, This represents a random noise vector sampled from a standard normal distribution. This indicates that the mean is zero and the covariance is... The 1024-dimensional standard normal distribution, It is a 1024-dimensional unit covariance matrix; Based on the reparameterization technique, the potential mean vector With the latent log-variance vector Decoupling is achieved by introducing standard normal noise. Random sampling is performed to obtain the latent vector. : (21) in, , Indicates batch size; The standard deviation vector; This is element-wise multiplication; latent vector After being mapped by a fully connected layer, it is reshaped into a feature map of size 8×8. ; feature map With the output feature map of the fourth convolutional layer In channel-dimensional splicing, the process is represented as follows: (22) in, and These represent the bias parameters and weight parameters of the fully connected layer, respectively. This represents the operation of reshaping a vector into a feature map of shape (B,256,8,8); This represents the initial feature map obtained after fully connected mapping and reshaping. This indicates the Skip concatenation operation, which is a feature concatenation operation in the channel dimension; Indicated by feature map and The intermediate feature map obtained after the Skip stitching operation; The input to the first deconvolutional layer is the intermediate feature map. The signal processing in the first deconvolution layer is represented as follows: (23) in, This represents the output feature map of the first deconvolutional layer. In spatial location The value at; This represents the input channel index of the first deconvolutional layer; Indicates the first deconvolution layer and the... The deconvolution kernel weight parameters corresponding to each input channel; This represents the feature map input to the first deconvolution layer. In spatial location The value at; The input to the second deconvolutional layer is the intermediate feature map. intermediate feature map It is the output feature map of the first deconvolution layer and the output feature map of the third convolutional layer The processing steps obtained after the Skip concatenation operation are as follows: (24) in, This represents the deconvolution operation of the second deconvolution layer. This represents the output feature map of the second deconvolution layer after processing with the deconvolution operation and the ReLU activation function. The input to the third deconvolutional layer is the intermediate feature map. intermediate feature map It is the output feature map of the second deconvolution layer Output feature map of the second convolutional layer The processing steps obtained after the Skip concatenation operation are as follows: (25) in, This represents the deconvolution operation of the third deconvolution layer. This represents the output feature map of the third deconvolution layer after processing with the deconvolution operation and the ReLU activation function. The input to the fourth deconvolutional layer is the intermediate feature map. intermediate feature map It is the output feature map of the third deconvolution layer and the output feature map of the first convolutional layer The processing steps obtained after the Skip concatenation operation are as follows: (26) in, This represents the deconvolution operation of the fourth deconvolution layer; This represents the Sigmoid activation function, used to map output features to... interval; This represents the output feature map of the fourth deconvolution layer after deconvolution and Sigmoid activation, i.e., the reconstructed spectrogram.

4. The method for enhancing and high-precision identifying small-sample ship acoustic signature data according to claim 3, characterized in that, In step 3, the process of training Conv-VAE using the log-Mel spectrogram of the entire dataset is as follows: Based on the overall dataset Building the training set and verification set During the training process of Conv-VAE, each complete training iteration, i.e., on the training set... After completing a parameter update, in the validation set The verification loss calculated above; Validation set Total of Mel spectrum samples, or Mel segments, are randomly selected. The Mel-ray spectral samples are batch-packaged to obtain the verification batch input. : (27) Keeping the network parameters frozen, forward propagation yields the reconstructed latent log-variance of the validation set. With potential distribution parameters; Calculate the batch loss, i.e., the mean square error of the validation set reconstruction. for: (28) in, , , This indicates the first input in the batch verification process. Mel spectrum samples in frequency dimension index Time-dimensional index The original Mel-spectral eigenvalues ​​at the location; Indicates the first element in the Conv-VAE output reconstruction spectrum. Mel fragment in frequency dimension index Time-dimensional index Reconstructed Mel-spectral eigenvalues ​​at the location; Calculate the KL term, i.e., the KL divergence of the latent space of the validation set. for: (29) in, , and Representing the validation set The Middle Mel spectrum samples in the first Mean and standard deviation over the latent space; Calculate the joint loss of the validation set. for: (30) in, These are the weighting coefficients for the KL terms; Calculate the joint loss of the training set for: (31) in, To reconstruct the mean squared error for the training set, The KL divergence of the latent space for the training set; During the training of Conv-VAE, the joint loss of the validation set is continuously monitored. When the condition shown in formula (32) is met, that is, the joint loss of the validation set in the current training cycle is not less than the minimum value of the joint loss of the validation set in the previous 5 training cycles, it is determined that the performance of Conv-VAE on the validation set has not continued to improve, and the training adjustment strategy is executed. (32) in, Indicates the index of the current training cycle. Indicates the index of historical training cycles. ; Indicates the first The joint loss value calculated on the validation set at the end of each training cycle; Indicates the first The joint loss value calculated on the validation set at the end of each training cycle; The specific training and adjustment strategies are as follows: Decrease the learning rate and continue training the Conv-VAE; If the condition shown in formula (32) is still satisfied after the learning rate decays, that is, in the current training cycle Joint loss of the validation set If the score does not drop, the current training process will be terminated early. Finally, a well-trained Conv-VAE is obtained.

5. The method for enhancing and high-precision identifying small-sample ship acoustic signature data according to claim 4, characterized in that, In step 3, the process of reconstructing and generating the log-Mel spectrogram of the small sample dataset using the trained Conv-VAE to obtain the reconstructed spectrogram is as follows: By scanning the Mel spectrogram sequence corresponding to the same audio segment in the small sample dataset one by one, a single Mel spectrogram segment is obtained, and its overall tensor shape is ( ,128,128), of which This indicates the number of Mel spectrogram frames obtained by dividing the same audio segment; The single-segment Mel spectrogram is fed into the encoder section of the Conv-VAE to obtain the latent spatial statistics: (33) in, The input is a single-segment Mel spectrogram sample with a size of 128×128; and It is a fully connected layer; Then, reparameter sampling is performed to obtain the latent vector. : (34) Conv-VAE decoder section Using the latent vector z and the skip features partially cached by the encoder Reconstruct the Mel spectrogram of the small sample dataset to obtain the reconstructed spectrogram. : (35)。 6. The method for enhancing and high-precision identifying small-sample ship acoustic signature data according to claim 5, characterized in that, In step 3, the process of generating reconstructed audio slices based on the reconstructed spectrogram output by Conv-VAE, and thus forming a reconstructed small sample dataset, is as follows: Reconstructed spectrogram based on the decoder output in Conv-VAE The normalized reconstructed Mel spectrum magnitude matrix is ​​obtained. Its numerical range is ; For reconstructing the Mel spectrum amplitude matrix Perform inverse normalization to map it back to the decibel scale, and you will get the inverse normalized Mel-spectrum decibel matrix. : (36) in, and These are the preset linear mapping coefficients; Converting the logarithmic scale to linear power yields the Mel power spectrum. : (37) in, In the Mel-decibel spectral matrix The Middle The Mel filter and the first Mel spectrum decibel values ​​at each time frame; This is a preset constant bias term; Using the inverse Mel filter matrix Mel power spectrum Perform weighted summation to reconstruct the linear short-time Fourier transform (STFT) power spectrum. : (38) in, For Fourier transform index, , This indicates the number of Fast Fourier Transform points used in the Short Time Fourier Transform; Known STFT power spectrum However, the phase is missing, so the Griffin-Lim algorithm is used to estimate the phase: Initialize the random phase matrix ; Each iteration constructs a complex spectrum, performs an inverse STFT to obtain the time-domain signal, and then obtains a new phase through a forward transform: (39) in, The power spectrum is linear frequency. Known reconstructed power spectrum However, the phase is missing, so the Griffin-Lim algorithm is used to estimate the phase: Initialize the random phase matrix And iteratively update: (40) (41) (42) in, , Indicates the first The complex spectrum of STFT obtained in the next iteration; For amplitude spectrum, ; For the first The phase obtained in the next iteration For the first The phase obtained in the next iteration; For the first The time-domain signal obtained in the next iteration; This represents the inverse short-time Fourier transform operator; Represents the Hann window function; This represents the operation of extracting phase from the complex spectrum of an STFT; When the phase converges or the number of iterations reaches 64, data augmentation is completed, and the time-domain signal obtained from the final iteration is used as a reconstructed audio slice to reconstruct the small sample dataset. It consists of several reconstructed audio slices.

7. The method for enhancing and high-precision identifying small-sample ship acoustic signature data according to claim 6, characterized in that, Step 4 specifically involves: small sample dataset Reconstructing small sample datasets and large sample datasets Merge the datasets to obtain the merged dataset. : (43) For the merged dataset The samples in the dataset undergo uniform preprocessing, converting them into log-Mel spectrograms with a sampling rate of 64kHz and a size of 128×128. The preprocessing process includes short-time Fourier transform and Mel spectrum mapping. Merged dataset Perform a short-time Fourier transform on the effective audio slices in the signal, and then process the signal obtained from the short-time Fourier transform. Add a Hann window, the first window added after adding the window A frame is represented as: (44) in, Indicates the first The time-domain frame signal after windowing. , Total number of frames; Indicates signal In the Temporal sampled values ​​of the frame; Total frames Represented as: (45) in, This represents the total number of samples in the resampled signal. Perform a Discrete Fourier Transform (DFT) on each frame of the signal to obtain the complex spectrum. for: (46) By splicing together all frames, a two-dimensional spectral matrix is ​​obtained. : (47) in, It represents 513 frequencies. and A complex matrix space consisting of time frames; For complex spectrum Square the modulus to obtain the power spectrum. : (48) in, Indicates the first Frame in frequency index Power spectral density at; By concatenating the power spectra of all time frames, we obtain the linear frequency power spectrum matrix, which is the overall power spectrum matrix. : ； Construct 128 Mel-triangle filters, with center frequencies uniformly distributed according to the Mel scale: (49) (50) in, Indicates the first A Mel triangular filter at frequency index Weighting coefficients at each location; Indicated in frequency index The linear frequency at that point; For the first The center frequency of each filter, Indicates the interval between the center frequencies of adjacent filters; All filter weights The matrix of Mel triangular filter banks , ; The overall power spectral matrix With Mel triangular filter bank matrix Performing matrix multiplication and taking the logarithm yields the final log-Mel spectrum matrix, i.e., the log-Mel spectrum diagram. Represented as: (51) The specific signal processing flow in the BEATs encoder is as follows: The final log-Mel spectrum matrix Input to the patch embedding layer of the BEATs encoder; Inject the weight file BEATs_iter3_plus_cpt2.pt, remove the original prediction header parameters during loading, and only retain the encoder part; The final log-Mel spectrum matrix Treating it as a single-channel image, after processing by the patch embedding layer of the BEATs encoder, two-dimensional sequence features are obtained: (52) in, This represents the token sequence output by the patch embedding layer; This represents the patch embedding operation, which divides the input feature map into parts of size 1. A patch with a step size of 16 is used, and each patch is projected to a 768-dimensional channel space through a linear mapping; right After adding learnable positional encoding, it is fed into a 12-layer bidirectional Transformer encoding block for processing: (53) in, , Each Transformer coding block contains a multi-head self-attention module and a feedforward network. Indicates the first The output of the layer Transformer coded block, Indicates the first The output of the layer Transformer coded block; Output of the 12th layer Transformer coded block The final hidden state is then averaged along the sequence dimension to obtain the merged dataset. 768-dimensional global voiceprint vector : (54) in, This represents the operation of averaging along the time series dimension; All parameters of the BEATs encoder are frozen, including the weights of the patch embedding layer, positional encoding, and the 12 Transformer encoding blocks. They do not participate in gradient updates and are executed only once in the forward pass. The merged dataset is processed by the BEATs encoder with frozen parameters. Feature extraction yields a 768-dimensional voiceprint feature vector for each sample. And composed of all voiceprint feature vectors This forms the characteristic matrix X.

8. The method for enhancing and high-precision identifying small-sample ship acoustic signature data according to claim 7, characterized in that, Step 5 specifically involves: The dataset used for classification and recognition loads a feature matrix X and its label y extracted by the BEATs encoder, where: (55) in, This represents the total number of samples in the dataset used for classification and recognition. The number of samples is greater than 6000, and the sample size of each class is greater than 500. The dataset used for classification and recognition is divided into training and testing sets, and 5-fold stratified cross-validation is used to ensure that the proportion of each class in each fold is consistent with the proportion of classes in the overall dataset. Fold to obtain training index With validation index The corresponding datasets are denoted as follows: and ; The specific processing flow of the signal in the LightGBM classifier is as follows: The LightGBM classifier takes voiceprint feature vectors as input and uses an additive model to build a decision tree ensemble. (56) in, , , Indicates from the previous The weighted result of the decision trees The predicted score vector of the i-th sample, i.e. Additive model for the first The predicted score vector of each sample; For the first The prediction function of a decision tree. Indicates the first Voiceprint feature vectors extracted from each sample by the BEATs encoder; For multi-class classification tasks, multi-class softmax loss is used: (57) in, The cross-entropy loss function represents the multi-class classification task. , Indicates category index, Indicates the total number of categories; The predicted score vector is represented at the th... The softmax probability output on the class; This indicates a conditional indicator function, which is defined if and only if the first... The true label of the sample is the first... Time-like functions The value is 1, otherwise the function The value is 0; Calculate the first and second gradients before each boost round: (58) in, It is the first-order gradient; It is a second-order gradient; This represents the current predicted probability. For any candidate split point s, the first The dimensional feature is divided into a left subset L and a right subset R, and the split gain is defined. for: (59) in, and Represents the regularization coefficient of the LightGBM classifier; and These represent the left child node at the [number]th position. The sum of the first and second gradients on the class; and These represent the right child node at the [number]th position. The sum of the first and second gradients on the class; Calculate the gradient statistic of the subset: (60) in, and They represent the first The cumulative statistics of the first and second gradients of the class at the current node; like If so, then the split will be executed; The weight of the new leaf is shown in formula (61): (61) in, and These are the weights of the left leaf and the right leaf, respectively. Record No. The integrated output after the wheel upgrade is Then a single-round update is represented as: (62) in, For the first Additive model for the first The predicted score of each sample, i.e., the updated predicted score; Indicates the first Additive model for the first The predicted score of each sample, which is the output of the previous additive model; Indicates the first The leaf weight of each sample falling into a leaf node; For any 768-dimensional input voiceprint feature vector ,through After the round of improvement, the score vectors for each category are: (63) in, This indicates that the input voiceprint feature vector is at the th... Predicted scores on the class For the first Decision trees for samples in the th Prediction functions on classes; The class probabilities are obtained by normalization using the Softmax function. : (64) in, Indicates the sample at the 1st Predicted scores on the class ; The final decision, i.e., the predicted class, is obtained from the LightGBM classifier. for: (65) in, This represents the operation of retrieving the category index corresponding to the highest category probability; Prediction Category This refers to the ship category identification result output by the LightGBM classifier.

9. The method for enhancing and high-precision identifying small-sample ship acoustic signature data according to claim 8, characterized in that, Step 5 also includes calculating the confusion matrix, the specific process of which is as follows: Statistical analysis is performed on all samples in the test set obtained from the dataset partitioned for classification and recognition. The samples are then paired and counted according to their true and predicted classes to form... square array That is, the confusion matrix, and the elements in the confusion matrix. Represented as: (66) in, , This represents the total number of samples in the test set obtained by partitioning the dataset for classification and recognition. Indicates the first The true label of each sample Indicates the first Predicted labels for each sample; It is an indicator function if and only if the first... The true label of the sample is the first... The class and the real label is the first When class, function The value is 1, otherwise the function The value is 0; Indicates the true category is the first Class and predicted category is the first The number of samples in a class.

Images