Underwater target identification method based on MutiHead-Jamba large model

Abstract

The invention belongs to the field of pattern recognition, and discloses an underwater target recognition method based on a MutiHead-Jamba large model, which comprises the following steps of: preprocessing audio data; extracting representative features in the underwater target, and fusing the obtained representative features to form a data set of the text; dividing the data set into a training set, a verification set and a test set according to a proportion of 8: 1: 1; building a MutiHead-Jamba network model, and sending the training set and the verification set to the network model for training and tuning; the test set is input into the trained MutiHead-Jamba model for prediction, and the type of the underwater target can be obtained; according to the method, the nearest Jamba model is used for deep feature extraction work, and a multi-head fusion idea in a multi-head attention mechanism is combined to form the MutiHead-Jamba network model provided by the invention, so that the recognition precision and robustness of the model can be improved.

Description

Technical Field

[0001] This invention belongs to the field of pattern recognition, specifically relating to an underwater target recognition method based on the MutiHead-Jamba large model. Technical Background

[0002] The underwater environment is filled with sounds from propellers, deep-sea resources, and mammals. When passive sonar receives acoustic signals, it uses specific feature extraction methods to extract useful features. Then, pattern recognition technology is used to classify these extracted features, allowing it to distinguish the surrounding environment. Feature extraction and pattern recognition are crucial components of underwater automatic target identification technology. They are applicable not only to research on underwater equipment and weapons but also to ship navigation, marine animal research, and deep-sea exploration, holding significant importance for national defense, science and technology, and ecology. However, underwater identification scenarios are characterized by environmental complexity and target diversity. The similarity between different targets limits the model's recognition capacity. Although machine learning and deep learning methods have achieved significant results, they still have the following limitations.

[0003] Machine learning methods exhibit poor tolerance, poor robustness, and weak anti-interference ability in underwater environments. As a result, the accuracy of underwater automatic identification will gradually decrease, weakening the applicability of the model.

[0004] To address the category similarity problem, deep learning methods extract abstract semantic features to improve the model's recognition accuracy. However, this introduces the risk of overfitting, resulting in poor generalization ability and affecting the model's recognition accuracy.

[0005] Therefore, providing a method for achieving high-precision target recognition in complex underwater environments is an urgent problem to be solved. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies and to develop an underwater target identification method based on the MutiHead-Jamba large model.

[0007] To achieve the above objectives, the present invention employs the following technical methods.

[0008] The underwater target identification method based on the MutiHead-Jamba large model includes the following steps:

[0009] Step 1: Audio data preprocessing;

[0010] Step 2: Extract representative features from underwater targets and fuse the obtained representative features to form the dataset of this invention;

[0011] Step 3: Divide the dataset into training set, validation set and test set in a ratio of 8:1:1;

[0012] Step 4: Build the MultiHead-Jamba network model and feed the training and validation sets into the network model for training and optimization;

[0013] Step 5: Input the test set into the trained MultiHead-Jamba model for prediction to obtain the type of underwater target;

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0015] (1) The underwater target recognition method based on the MultiHead-Jamba large model of the present invention significantly improves the classification accuracy of the MultiHead-Jamba large model compared with the classic neural network without increasing the computational load of the model. It can learn rich features while ensuring the capacity of the network and has good applicability.

[0016] (2) The present invention is based on the underwater target recognition method of the MutiHead-Jamba large model. The architecture of the MutiHead-Jamba large model does not depend on the temporal order in the audio and has high computational efficiency.

[0017] (3) The underwater target recognition method based on the MultiHead-Jamba large model of the present invention adopts parallel operation for training the large model, which greatly improves the convergence speed of the model.

[0018] (4) An underwater target recognition method based on the MultiHead-Jamba large model uses the Attention mechanism in the Transformer module to focus on important information in the feature map.

[0019] (5) The underwater target recognition method based on the MultiHead-Jamba large model uses the Mamba module in the Mamba module to extract depth feature information, which can capture the long-term dependencies in the audio signal better.

[0020] (6) The underwater target recognition method based on the MultiHead-Jamba large model uses the MoE module to replace the multilayer perceptron for outputting results, which can increase the model capacity without increasing the computational load.

[0021] (7) The underwater target recognition method based on the MultiHead-Jamba large model is to form a Jamba module by serial residual connection of the Transformer module, Mamba module and MoE module. By interleaving the Transformer module and Mamba module, the advantages of the two modules are combined, which can improve the recognition accuracy of the underwater model with low memory usage.

[0022] (8) The underwater target recognition method based on the MultiHead-Jamba large model combines the multi-head connection in MultiHead attention and the Jama module to form the MultiHead-Jamba network. The multi-head connection can integrate the input and output of different heads, which greatly improves the robustness and generalization of the model. Attached Figure Description

[0023] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0024] Figure 1 This is a flowchart of an underwater target identification method based on the MutiHead-Jamba large model;

[0025] Figure 2 This is a diagram of the Jamba module structure;

[0026] Figure 3 This is a structural diagram of the MutiHead-Jamba network;

[0027] Figure 4 It is the loss curve during the training process;

[0028] Figure 5 This is a comparison chart of the recognition accuracy of the MutiHead-Jamba large model; Detailed Implementation

[0029] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0030] like Figure 1 As shown, the underwater target recognition method based on the MultiHead-Jamba large model includes the following steps:

[0031] Step 1: Audio data preprocessing;

[0032] Sub-step 1.1: Select two public datasets, ShipsFar and DeepShip, as the datasets for this experiment.

[0033] Sub-step 1.2 involves using frame segmentation technology to divide each audio file into multiple sub-datasets based on 5s and 3s, and then merging these datasets to expand the sample size.

[0034] Sub-step 1.3 involves performing Min-Max standardization on the augmented data, using the following formula:

[0035]

[0036] Where X is the original dataset, X min X is the minimum value in the original dataset. max X is the maximum value in the original dataset. n For the preprocessed dataset, the Min-Max preprocessing operation can map the original values ​​X to the range [0, 1], making each data point have the same dimensions.

[0037] Step 2: Extract representative features from underwater targets and fuse the obtained representative features to form the dataset of this invention.

[0038] Sub-step 2.1 involves extracting MFCC features that are highly sensitive to the human auditory system. The formula for MFCC features is divided into the following sub-steps:

[0039] Sub-step 2.1.1 converts the audio time-domain signal into a frequency-domain signal using a Fast Fourier Transform:

[0040]

[0041] Where X(n) is the time-domain signal, X(i) is the frequency-domain signal, N is the number of sampling points, and i is the spectral line in the spectrum.

[0042] Sub-step 2.1.2: Calculate the energy spectrum of the frequency domain signal, and output the logarithmic energy by passing the energy spectrum through multiple Mel filter banks.

[0043]

[0044] Among them, H l (i) is the frequency domain response of the l-th Mel filter bank, and E(l) is the logarithmic energy.

[0045] Sub-step 2.1.3 involves performing a discrete cosine transform on the obtained energy to obtain the MFCC coefficients:

[0046]

[0047] Where M(n) is the nth MFCC coefficient, which is a representative feature of the underwater target type. Then, the different MFCC coefficients are fused to obtain the dataset of this invention.

[0048] Sub-step 2.2 involves manually labeling the feature set obtained in sub-step 2.1. The labels are in one-hot encoding form. Since both public datasets used in this invention have four target types, the one-hot encoding form for the label of the first target type is [1, 0, 0, 0], the label of the second target type is [0, 1, 0, 0], and so on.

[0049] Step 3: Build the MutiiHead-Jamba network model and send the training and validation sets to the network model for training and optimization.

[0050] In sub-step 3.1, we first connect the modules sequentially in the order of Norm-Attention-Norm, which allows us to focus on important information in the feature map through the attention mechanism. Then, we introduce residual connections on the basis of the serial connections to reduce the computational cost of the model and accelerate the convergence speed of the network. Then, the output of the serial-parallel connections is output through a multilayer perceptron. Finally, we combine the above strategies to form the Transformer module.

[0051] Sub-step 3.2 follows the sequence of sub-step 3.1, performing serial-parallel connections, but using the Mamba module instead of the Attention mechanism in step 3.1 to extract deep feature information. This effectively captures long-term dependencies in the audio signal, and combining the above strategies results in... Figure 3 The Mamba module is shown. The core of the Mamba module is the State Space Model (SSM) module, which draws on classical control theory, establishing a graph mapping between input and output through a latent state. Therefore, the SSM can be represented by linear ordinary differential equations:

[0052]

[0053] Y(t) = C·H(t)

[0054] Where X(t) and Y(t) are the input and output, respectively, H(t) is the potential state, and A, B, and C are the control variables.

[0055] Sub-step 3.3 follows the sequence of sub-step 3.1, performing serial-parallel connections, but using the Mixture of Expert (MoE) module instead of the multilayer perceptron in step 3.1 for output. This increases the model's capacity without increasing computational load, and combining these strategies forms the Mamba-MoE module.

[0056] Sub-step 3.4: Combine the Transformer module, Mamba module, and Mamba-MoE module obtained in steps 3.1 to 3.3 according to... Figure 2 The order of the modules is sequentially connected to form Jamba modules. By interleaving the use of Transformer and Mamba modules, the advantages of both modules are combined, which can improve the recognition accuracy of underwater models with low memory usage.

[0057] In sub-step 3.5, to improve the robustness of the model, this invention borrows multi-head connections from Multi-Head attention and combines them with the Jama module from 3.4 to form the proposed MultiHead-Jamba network. Multi-head connections enable each head to perform deep feature extraction and dependency capture in different subspaces, better capturing complex relationships and patterns in underwater audio signals compared to a single Jama module. Furthermore, multi-head connections integrate the inputs and outputs of different heads, significantly improving the model's robustness and generalization ability. Therefore, the proposed MultiHead-Jamba network enhances the model's expressive power, improves its recognition accuracy, and enhances its robustness.

[0058] Step 4: Input the test set into the trained MutiHead-Jamba model for prediction to obtain the type of underwater target.

[0059] Sub-step 4.1 involves shuffling the test set by setting different random seeds and selecting the test set with replacement, then inputting the test set into the MultiHead-Jamba network for classification and recognition.

[0060] Sub-step 4.2 uses a five-fold cross-validation method to average and weight the results of the five trials to prevent the randomness of the experiment, and the average weighted result is used as the recognition accuracy of underwater targets.

[0061] The effects of the present invention will be further described below with reference to experiments.

[0062] 1. Experimental conditions

[0063] The network was built using Python, acoustic feature extraction was performed using the librosa library, and the proposed model was built using Keras 2.6.0. Adam was used as the optimizer. The model was trained on an NVIDIA GeForce GTX 2080 Ti. Experiments were conducted using two authoritative public datasets, ShipsEar and DeepShip.

[0064] 2. Simulation Content and Result Analysis

[0065] The training set, validation set, and test set used in the simulation experiments of this invention are datasets generated according to step 3 of this invention. The ShipsEar dataset and the DeepShip dataset contain 6570 samples and 56821 samples respectively. The datasets are divided into training set, validation set, and test set in a ratio of 8:1:1.

[0066] The MultiHead-Jamba model is trained and optimized using the training and validation sets, then input into the test set to identify underwater target types and output classification results.

[0067] The loss function for underwater target recognition is as follows: Figure 4 As shown in the comparison experiment of recognition performance, Figure 5 As shown.

[0068] from Figure 4 As can be seen, the training error (loss) gradually decreases with the increase of the number of iterations, indicating that the training process of the MutiHead-Jamba model is relatively stable. The final loss approaches zero and stabilizes, indicating a low model error. The difference between the losses of the training set and the validation set is approximately zero, indicating a low model variance. These phenomena suggest that the MutiHead-Jamba model is in a good fit state.

[0069] from Figure 5 As can be seen, this invention includes comparative experiments with different classifiers to verify the effectiveness of the proposed model. The MultiHead-Jamba model proposed in this invention achieves higher recognition accuracy than traditional machine learning and deep learning methods for underwater target identification.

[0070] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An underwater target identification method based on the MultiHead-Jamba large model, comprising the following steps: Step 1: Audio data preprocessing; Step 2: Extract representative features from underwater targets and fuse the obtained representative features to form the dataset of this invention; Step 3: Divide the dataset into training set, validation set and test set in a ratio of 8:1:1; Step 4: Build the MultiHead-Jamba network model and feed the training and validation sets into the network model for training and optimization; Step 5: Input the test set into the trained MutiHead-Jamba model for prediction to obtain the type of underwater target.

2. The underwater target method based on the MultiHead-Jamba model according to claim 1, characterized in that, Step 1 includes the following sub-steps: Sub-step 1.1: Select two public datasets, ShipsEar and DeepShip, as the datasets for this experiment. Sub-step 1.2 involves using frame segmentation technology to divide each audio file into multiple sub-datasets based on 5s and 3s, and then merging these datasets to expand the sample size. Sub-step 1.3 involves performing Min-Max standardization on the augmented data, using the following formula: Where X is the original dataset, X min X is the minimum value in the original dataset. max X is the maximum value in the original dataset. n For the preprocessed dataset, the Min-Max preprocessing operation can map the original values ​​X to the range [0, 1], making each data point have the same dimensions.

3. The underwater target method based on the MultiHead-Jamba model according to claim 2, characterized in that, Step 2 includes the following sub-steps: Sub-step 2.1 involves extracting MFCC features that are highly sensitive to the human auditory system. The formula for MFCC features is divided into the following sub-steps: Sub-step 2.1.1 converts the audio time-domain signal into a frequency-domain signal using a Fast Fourier Transform: Where X(n) is the time-domain signal, X(i) is the frequency-domain signal, N is the number of sampling points, and i is the spectral line in the spectrum. Sub-step 2.1.2: Calculate the energy spectrum of the frequency domain signal, and output the logarithmic energy by passing the energy spectrum through multiple Mel filter banks. Among them, H l (i) is the frequency domain response of the l-th Mel filter bank, and E(l) is the logarithmic energy. Sub-step 2.1.3 involves performing a discrete cosine transform on the obtained energy to obtain the MFCC coefficients: Where M(n) is the nth MFCC coefficient, which is a representative feature of the underwater target type. Then, the different MFCC coefficients are fused to obtain the dataset of this invention. Sub-step 2.2 involves manually labeling the feature set obtained in sub-step 2.

1. The labels are in one-hot encoding form. Since both public datasets used in this invention have four target types, the one-hot encoding form for the label of the first target type is [1, 0, 0, 0], the label of the second target type is [0, 1, 0, 0], and so on.

4. The underwater target method based on the MultiHead-Jamba model according to claim 2, characterized in that, Step 4 includes the following sub-steps: In sub-step 3.1, we first connect the modules sequentially in the order of Norm-Attention-Norm, which allows us to focus on important information in the feature map through the attention mechanism. Then, we introduce residual connections on the basis of the serial connections to reduce the computational cost of the model and accelerate the convergence speed of the network. Then, the output of the serial-parallel connections is output through a multilayer perceptron. Finally, we combine the above strategies to form the Transformer module. Sub-step 3.2 follows the sequence of sub-step 3.1, performing serial-parallel connections, but using the Mamba module instead of the attention mechanism in step 3.1 to extract deep feature information. This effectively captures long-term dependencies in the audio signal. Combining the above strategies forms the Mamba module shown in Figure 3. The core of the Mamba module is the State Space Model (SSM) module, which draws on classical control theory, establishing a graph mapping between input and output through a latent state. Therefore, the SSM can be represented by linear ordinary differential equations: Y(t) = C·H(t) Where X(t) and Y(t) are the input and output, respectively, H(t) is the potential state, and A, B, and C are the control variables. Sub-step 3.3 follows the sequence of sub-step 3.1, performing serial-parallel connections, but using the Mixture of Expert (MoE) module instead of the multilayer perceptron in step 3.1 for output. This increases the model's capacity without increasing computational load, and combining these strategies forms the Mamba-MoE module. In sub-step 3.4, the Transformer module, Mamba module, and Mamba-MoE module obtained in steps 3.1 to 3.3 are connected in series according to the order shown in Figure 2 to form a Jamba module. By using the Transformer module and Mamba module alternately, the advantages of the two modules are combined, which can improve the recognition accuracy of underwater models with low memory usage. In sub-step 3.5, to improve the robustness of the model, this invention borrows multi-head connections from Multi-Head attention and combines them with the Jama module from 3.4 to form the proposed MultiHead-Jamba network. Multi-head connections enable each head to perform deep feature extraction and dependency capture in different subspaces, better capturing complex relationships and patterns in underwater audio signals compared to a single Jama module. Furthermore, multi-head connections integrate the inputs and outputs of different heads, significantly improving the model's robustness and generalization ability. Therefore, the proposed MultiHead-Jamba network enhances the model's expressive power, improves its recognition accuracy, and enhances its robustness.

5. The underwater target method based on the MultiHead-Jamba model according to claim 2, characterized in that, Step 5 includes the following sub-steps: Sub-step 4.1 involves shuffling the test set by setting different random seeds and selecting the test set with replacement, then inputting the test set into the MultiHead-Jamba network for classification and recognition. Sub-step 4.2 uses a five-fold cross-validation method to average and weight the results of the five trials to prevent the randomness of the experiment, and the average weighted result is used as the recognition accuracy of the underwater target.

Images