Underwater acoustic target and AIS ship dynamic adaptive correlation method based on multi-modal deep learning

Abstract

The application relates to a multi-modal deep learning-based underwater acoustic target and AIS ship dynamic adaptive correlation method, which comprises the following steps: step S1: constructing a multi-modal correlation sample library, collecting and spatiotemporally aligning underwater acoustic data and automatic identification system (AIS) data; using reliable AIS signals to label corresponding AIS identification codes for underwater acoustic data segments, forming a labeled sample pair of underwater acoustic features and AIS identities, and constructing a negative sample set containing AIS signal loss or fraud scenarios. The application fundamentally improves the performance of the model by adopting a multi-modal deep learning network.

Description

Technical Field

[0001] This invention belongs to the field of underwater acoustic signal processing and multi-source information fusion technology, specifically involving a dynamic adaptive association method between underwater acoustic targets and AIS ships based on multimodal deep learning. Background Technology

[0002] In fields such as marine monitoring, maritime supervision, underwater security, and marine scientific research, achieving comprehensive and accurate perception of surface vessels and underwater targets is of great significance. Currently, Automatic Identification Systems (AIS) and seabed / underwater acoustic monitoring systems are two main sources of information.

[0003] The AIS system can provide high-precision, high-frequency information on the identity, position, speed, heading, and static attributes of surface vessels, but its effectiveness depends entirely on the active transmission of signals by the vessels. Therefore, AIS has inherent limitations: it cannot identify unauthorized or stealthy vessels that have not installed, turned off, are damaged, or are deliberately spoofing their AIS signals, and it completely lacks underwater target detection capabilities. On the other hand, underwater acoustic monitoring systems (such as fixed seabed hydrophone arrays and towed linear arrays) passively receive underwater acoustic signals to detect and track underwater targets, making them a core means of acquiring underwater target information. However, these systems acquire limited target information (usually only bearing, intensity, and spectrum), have relatively low positioning accuracy, and are susceptible to interference from complex marine environmental noise (such as biological noise and shipping noise), resulting in high false alarm and missed alarm rates, and they cannot directly determine the target's identity.

[0004] Effectively fusing AIS data with underwater acoustic detection data allows for the interpretation and labeling of underwater acoustic targets using the high-precision identity and status information of AIS, while simultaneously using underwater acoustic detection to compensate for the blind spots of AIS in detecting non-cooperative targets. This is a key technological path to achieving integrated collaborative sensing between the sea surface and underwater environments. Early methods primarily relied on geometric track association, such as nearest neighbor algorithms and track similarity calculations. However, in scenarios with dense targets, intersecting tracks, or frequent maneuvers, the false association rate rises sharply. Subsequently, researchers proposed methods based on traditional machine learning, such as support vector machines and fuzzy neural networks, to perform matching through manually designed features (such as azimuth difference, velocity difference, and Doppler matching degree). While these methods offer some performance improvements, they also have significant limitations: First, feature engineering is complex and highly dependent on expert experience, resulting in insufficient robustness of the designed features in complex and ever-changing real-world marine environments. Second, the models are essentially shallow learning paradigms with limited learning and generalization capabilities, making it difficult to automatically extract deep-seated, high-order correlation features from the data. Third, the models have poor adaptability; when the application area or acoustic environment changes, it is necessary to redesign the features or even adjust the model structure.

[0005] In recent years, deep learning technology has begun to be introduced into this field. For example, some scholars have proposed target recognition methods based on matching AIS and passive sonar information, using fuzzy neural networks to process AIS tracks and passive sonar tracks. However, the input to these methods is still preprocessed track data (such as position, velocity, and bearing), rather than the original or low-level acoustic signals (such as time-spectrum maps and original waveforms), failing to fully utilize the rich structural information in underwater acoustic data. Essentially, they have not yet broken free from the framework of manual feature extraction + shallow discrimination, and the potential of deep learning in automatic representation learning has not yet been fully explored.

[0006] Therefore, there is an urgent need for an intelligent association method that can utilize raw underwater acoustic signals and multi-dimensional AIS information end-to-end, and has strong adaptability and high robustness. Summary of the Invention

[0007] The technical problem to be solved by this invention is to provide a dynamic adaptive association method between underwater acoustic targets and AIS ships based on multimodal deep learning. This method aims to solve the problems of insufficient information from a single data source in complex marine environments, poor robustness of traditional association methods, and fragile feature design with weak generalization ability.

[0008] The technical solution of this invention is to provide a method for dynamically adaptively associating underwater acoustic targets with AIS ships based on multimodal deep learning, including the following steps:

[0009] Step S1: Construct a multimodal association sample library

[0010] Collect and spatiotemporally align underwater acoustic data with Automatic Identification System (AIS) data; use reliable AIS signals to label underwater acoustic data segments with corresponding AIS identifiers, forming labeled sample pairs of underwater acoustic features and AIS identities, and construct a negative sample set containing scenarios of missing or spoofed AIS signals;

[0011] Step S2: Design and train a deep feature fusion network

[0012] The deep feature fusion network includes: an underwater acoustic feature subnetwork for automatically extracting deep acoustic fingerprint features from raw underwater acoustic data; an AIS feature subnetwork for extracting dynamic behavior pattern features and static attribute features of ships from AIS data; and a feature fusion and association decision layer for performing multimodal fusion of the deep acoustic fingerprint features with the dynamic behavior pattern features and static attribute features of the ships, and outputting an association confidence matrix representing the association probability between the underwater acoustic target and the AIS ship.

[0013] Step S3: Online Adaptive Association and Early Warning

[0014] The real-time acquired underwater acoustic features and AIS features are input into the trained deep feature fusion network to obtain a real-time association confidence matrix. The decision threshold is dynamically calculated based on the current environmental complexity, and the results of successful association, marked as a suspected non-cooperative target, or confirmed association failure are output according to the comparison between the real-time association confidence matrix and the dynamic decision threshold.

[0015] The method proposed in this invention changes from "manually designed features + shallow model decision" to "deep network automatically learns essential features + probabilistic output", thus solving the feature vulnerability problem.

[0016] In the problem of associating underwater acoustic data with AIS data, early methods mainly relied on geometric track association, which resulted in a high false association rate when targets were densely packed and tracks intersected. Subsequently, methods based on traditional machine learning emerged, using manually designed features for matching. While this provided some improvement, feature engineering was complex and generalization ability was weak. In recent years, deep learning has been introduced, capable of automatically learning features from raw data, bringing new breakthrough potential to association tasks. This invention fundamentally improves model performance by employing a multimodal deep learning network.

[0017] Preferably, the construction of the multimodal association sample library in step S1 specifically includes:

[0018] Data acquisition and synchronization: The data received by the underwater acoustic monitoring array and the AIS base station are synchronized in time and spatially registered using a unified time reference source;

[0019] Data preprocessing and fragmentation: The original underwater acoustic signal is denoised and time-frequency transformed to generate a time-frequency spectrum, and divided into underwater acoustic analysis segments with fixed durations; AIS data is decoded, cleaned, correlated with points and generated with tracks, and AIS tracks are strictly aligned with the underwater acoustic analysis segments in time windows through interpolation and resampling.

[0020] Sample labeling: Labeled samples are generated by combining automatic matching with manual verification, and challenging samples, including negative samples and adversarial samples, are actively constructed.

[0021] Preferably, in step S2,

[0022] The underwater acoustic feature subnetwork uses a convolutional neural network and an attention mechanism to automatically extract deep acoustic fingerprint features related to ship radiated noise, taking underwater acoustic time-spectrum maps as input.

[0023] The AIS feature subnetwork uses a sequence model to process the dynamic trajectory sequence of the ship to capture maneuvering patterns, and integrates the static attribute features of the ship with the contextual environment features to generate AIS semantic feature vectors.

[0024] The feature fusion and association decision layer adopts a cross-attention mechanism, using underwater acoustic features as queries and AIS features as keys and values, to calculate attention-weighted AIS features, which are then fused with underwater acoustic features. Finally, the association confidence matrix is ​​output through a fully connected layer.

[0025] Preferably, the backbone network of the underwater acoustic feature subnetwork is ResNet-18, followed by a convolutional block attention module (CBAM); the AIS feature subnetwork uses a gated recurrent unit (GRU) to process dynamic track sequences and uses an embedding layer (Embedding) to process the ship's category features.

[0026] Preferably, in step S2, when training the deep feature fusion network, Focal Loss is used as the loss function, and data augmentation operations such as time shifting, frequency shifting, and adding random noise are performed on the underwater acoustic spectrogram; the training strategy is to first pre-train the underwater acoustic sub-network and the AIS sub-network separately, and then perform joint fine-tuning of the entire network.

[0027] Preferably, the dynamic calculation of the decision threshold in step S3 is as follows: τ = τ_base + α * target density + β * environmental noise level, where τ_base is the baseline threshold and α and β are adjustment coefficients.

[0028] Preferably, step S3 further includes a continuous optimization and incremental learning step: the verification results of manually marked suspected non-cooperative targets are stored as new labeled samples in the sample library, and the incremental learning of the model is triggered; the incremental learning freezes the shallow structure of the deep feature fusion network, and only the fusion layer and decision layer are retrained and fine-tuned.

[0029] Preferably, the underwater acoustic data includes broadband spectrum, narrowband spectrum, bearing history, and depth information; the AIS data includes the ship's position, speed, heading, size, and type.

[0030] Compared with the prior art, the present invention has the following advantages:

[0031] This invention fundamentally improves model performance by employing a multimodal deep learning network. First, the deep learning model automatically learns high-dimensional essential features from raw data, directly overcoming the vulnerability of manually designed features in complex scenarios. Second, dual-channel fusion and cross-attention mechanisms achieve a deep understanding of underwater acoustic features and AIS behavioral semantics, making their association judgments more interpretable. Finally, the model can continuously optimize and incrementally learn, constantly expanding its application boundaries. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the overall workflow of the underwater acoustic target and AIS ship dynamic adaptive association method described in this embodiment of the invention.

[0033] Figure 2 This is a schematic diagram of the deep feature fusion network described in an embodiment of the present invention. Detailed Implementation

[0034] The present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0035] A method for adaptively associating underwater acoustic targets with AIS ship dynamics based on multimodal deep learning, such as... Figure 1 As shown, it includes the following steps:

[0036] (1) Constructing a multimodal association sample library, the main process is as follows:

[0037] (1.1) Data acquisition and synchronization, mainly including equipment deployment and spatiotemporal alignment:

[0038] (1.1.1) Equipment deployment: An underwater acoustic monitoring array will be deployed in the sea area of ​​interest, and its location will be precisely determined by GPS or BeiDou. At the same time, data from AIS base stations in the sea area will be accessed.

[0039] (1.1.2) Time synchronization: All underwater acoustic acquisition nodes and AIS receivers are connected to a unified time reference source to ensure that each data packet has a UTC timestamp with the same second-level precision.

[0040] (1.1.3) Spatial registration: The underwater acoustic data and the latitude and longitude coordinates provided by AIS are uniformly converted to the same geographic coordinate system (WGS-84 in this embodiment) and then to rectangular coordinates (x,y) for subsequent calculations.

[0041] (1.2) Data preprocessing and fragmentation

[0042] (1.2.1) Preprocessing of underwater acoustic data: The original underwater acoustic signal is denoised. The adaptive filtering LMS algorithm is used to suppress environmental noise and generate a time spectrum. A fixed duration (30 seconds in this embodiment) is used as an analysis segment to form a continuous acoustic snapshot.

[0043] (1.2.2) AIS Data Preprocessing: 1. First, the AIS data is decoded. AIS messages are transmitted via the NMEA 0183 protocol. AIVDM (received AIS messages) is used to represent messages received from other ships or base stations. 2. Then, the data is cleaned: ① Message integrity verification: All characters from the start character "!" to the checksum delimiter "*" are XORed sequentially. The result is converted to a hexadecimal value and compared with the value of the checksum field. If they match, the message is considered correctly received; otherwise, the message is discarded. ② Missing value cleaning: Data entries missing the critical field MMSI are ignored. Data entries missing important fields (length, width, name, IMO number, call sign) are filled by querying the existing database. If the missing fields cannot be filled, they are stored in the anomaly database for processing in subsequent cleaning cycles. ③ Content cleaning: If the content contains invalid characters (such as letters in the MMSI field), identify problematic data through database filtering and compare it with other valid fields in the same data source to fill or correct outliers. 3. Next, perform AIS track association: group tracks according to the unique identifier MMSI number in the AIS static information (if there is no MMSI, it can be based on IMO). If there is no MMSI, fuzzy classification can be performed based on ship dynamic information (longitude, latitude, heading, speed) and voyage data (departure point, destination). Then, sort the data in the same MMSI group in ascending order by timestamp, and finally segment them according to the anchorage status (speed ≈ 0 continuously exceeding the threshold). 4. Finally, after generating single track data through track data association, interpolate and resample to ensure strict alignment with each underwater acoustic data segment in the time window. The track information at a certain moment is only related to track points within a small range around that moment, so the cubic spline interpolation method in the local fitting method is used to interpolate the track data. Principle of cubic spline curve:

[0044] Suppose we have the following nodes

[0045]

[0046] spline curve It is a formula defined by a segment. Given n+1 data points and n intervals, the cubic spline equation satisfies the following conditions:

[0047] In each segment interval (i = 0, 1, …, n-1, x increments). Both are cubic polynomials.

[0048] satisfy (i = 0, 1, …, n)

[0049] derivative Second derivative The interval [a, b] is continuous, that is... The curve is smooth. Therefore, the piecewise division of n cubic polynomials can be written as:

[0050] , i = 0, 1, …, n-1, where ai, bi, ci, di represent 4n unknown coefficients.

[0051] (1.3) Sample labeling, including automatic initial labeling, manual verification and enhancement:

[0052] (1.3.1) Automatic tagging: During periods when the AIS signal is clear and there are few targets, AIS targets are automatically matched for underwater acoustic segments through nearest neighbor matching to generate preliminary tags.

[0053] (1.3.2) Manual Verification and Enhancement: The operator replays the acoustic spectrogram, bearing lines, and electronic nautical charts overlaid with AIS tracks on the computer to perform 100% verification and correction of the automatic annotation results. Simultaneously, challenging samples and negative samples are actively constructed: segments where AIS vessels are clearly present but not detected by underwater acoustics; or segments where underwater acoustics detects targets but there are no AIS vessels in the area (simulating AIS being off). Adversarial samples: collecting underwater acoustic segments corresponding to vessels for which AIS information is available (e.g., sudden speed changes, unreasonable headings).

[0054] (2) Design and training of deep feature fusion network

[0055] (2.1) Network input layer design: Channel 1 input, fixed-size time spectrum graph, fixed-size input is [300,256], representing the time dimension (300 frames) and frequency dimension (256 frequency points) respectively. The amplitude value of the spectrum is channel normalized so that its mean is 0 and its standard deviation is 1. Then the input tensor is [Batch_size,1,300,256];

[0056] The second input channel is a structured feature vector, including dynamic sequence features and static attribute features of the vessel within the aligned time period. The dynamic sequence features refer to AIS data extracted at five equally spaced time points aligned with the spectrogram. Each time point feature includes relative position (x, y), motion state (speed to ground, heading to ground, rate of change of speed), and the relative position is standardized using the formula x_normalized=(x-μ) / σ, where μ is the mean and σ is the standard deviation. The angle features are relativized to 0°, (sin(0),cos(0))=[0,1]. Static attributes include category features (fishing vessels, merchant ships, cargo ships, etc.) encoded into a 6-dimensional vector using an embedding layer, and vessel size features are normalized. Contextual features include average sea state level and total ambient noise power level. The dynamic sequence input vector is [Batch_size,5,7], and the static and contextual features are concatenated into a vector of [Batch_size,10].

[0057] (2.2) Dual-channel feature extraction network structure, such as Figure 2 As shown, the deep feature fusion network designed in this embodiment includes two feature extraction subnetworks and one feature fusion and decision layer, namely, an underwater acoustic feature subnetwork and an AIS / context feature subnetwork:

[0058] (2.2.1) Underwater acoustic feature subnetwork: The backbone adopts a lightweight convolutional neural network ResNet-18. An attention module is added after it to make the network focus on the time-frequency region in the acoustic spectrogram that is most relevant to the ship's radiated noise, thereby suppressing background noise interference.

[0059] (2.2.2) AIS / Context Feature Sub-Network: For dynamic trajectory sequences, a Gated Recurrent Unit (GRU) network is used to capture the ship's maneuvering patterns (such as turning and acceleration). The final hidden state output by the GRU is concatenated with static and context features to form a unified AIS semantic feature vector. The input is [Batch_size, 5, 7], the number of bidirectional GRU hidden layer units is 32, and the bidirectional hidden state of the last time step is taken. After concatenation, a 64-dimensional behavioral feature vector is obtained. Then, the static feature and environment feature vector [Batch_size, 10] are mapped to 16 dimensions through a fully connected layer. Finally, they are concatenated to form an AIS semantic feature vector of [Batch_size, 80].

[0060] (2.3) Feature Fusion and Association Decision Layer: The high-level feature vector output by the underwater acoustic sub-network is concatenated with the AIS semantic feature vector, and then deeply fused through a fully connected layer. The underwater acoustic feature vector is [Bach_size, 128], and the AIS semantic feature vector is [Bach_size, 80]. The two vectors are linearly transformed to generate Key, Query, and Value. The attention weights are calculated using the underwater acoustic features as the Query and the AIS features as the Key and Value. The attention-weighted AIS features and the underwater acoustic features are then added element-wise to obtain the deeply fused feature vector [Bach_size, 128].

[0061] The output layer is a many-to-many relationship. Assuming there are M underwater acoustic targets and N AIS vessels in the current scene, the network ultimately outputs an M×N association confidence matrix. The value of the (i, j)th element in the matrix represents the probability that the i-th underwater acoustic target and the j-th AIS vessel are the same entity (normalized to [0,1] by softmax). The structure is: a fully connected layer FC (128>64), followed by a ReLU activation function, then another fully connected layer FC (64>32), followed by another ReLU activation function, and finally a special fully connected layer FC (32>M*N), followed by a Sigmoid activation function.

[0062] (2.4) Model Training: A weighted loss function, Focal Loss, is used, FL(p) = -α*(1-p)^y*log(p), where p is the prediction probability. During training, the underwater acoustic spectrogram is slightly time-shifted, frequency-shifted, and random noise is added to enhance the robustness of the model. The underwater acoustic sub-network and the AIS network are pre-trained separately with a large amount of data, and then the entire network is jointly fine-tuned. The first few layers of the two sub-networks are frozen, and only the high-level layers and the fusion layer are trained to accelerate convergence and improve performance.

[0063] (3) Online adaptive association: the trained model is deployed in the actual system to form a closed loop from perception to decision.

[0064] (3.1) Real-time data stream processing: The online system receives underwater acoustic and AIS data in parallel, following (1.1) and (1.2), to produce real-time data segments with a unified format. The system maintains a sliding time window, and the data within the window is fed into the trained deep feature fusion network in real time for forward inference.

[0065] (3.2) Adaptive Association Decision: The association threshold is τ, which is dynamically calculated based on the current scene complexity: τ = τ_base + α* (target density) + β* (environmental noise level). Where τ_base is the baseline threshold, and α and β are adjustment coefficients. For any element in the confidence matrix that is greater than the dynamic threshold τ, the system determines that the association is successful and outputs a match. If for a certain underwater acoustic target, its confidence with all AIS ships is lower than τ, but the highest value exceeds a low warning threshold (set to 0.3 in this embodiment), then the target is marked as a suspected non-cooperative target.

[0066] (3.3) Continuous optimization and incremental learning: The suspected non-cooperative targets alerted by the system are manually verified. Target information can be obtained using radar, video, and other commercial AIS data. This information, along with underwater acoustic segments, is stored as new labeled samples in the database. After accumulating a certain number of new samples, incremental learning is initiated. This process freezes the shallow structure of the network, training and fine-tuning only the final fusion layer and decision layer.

[0067] Table 1 shows the comparison of the association accuracy of the method of the present invention with that of different existing methods on the same test dataset.

[0068]

[0069] As shown in Table 1, this method achieved the highest association accuracy (93.2%) compared to existing typical methods on the same test dataset, verifying the effectiveness and advancement of this invention.

[0070] The above description only illustrates preferred embodiments of the present invention and should not be construed as limiting the scope of the claims. Any equivalent procedural modifications made using this specification are included within the patent protection scope of this invention.

Claims

1. A method for dynamically adaptively associating underwater acoustic targets and AIS ships based on multimodal deep learning, characterized in that: Includes the following steps, Step S1: Construct a multimodal association sample library Collect and spatiotemporally align underwater acoustic data with Automatic Identification System (AIS) data; use reliable AIS signals to label underwater acoustic data segments with corresponding AIS identifiers, forming labeled sample pairs of underwater acoustic features and AIS identities, and construct a negative sample set containing scenarios of missing or spoofed AIS signals; Step S2: Design and train a deep feature fusion network The deep feature fusion network includes: an underwater acoustic feature subnetwork for automatically extracting deep acoustic fingerprint features from raw underwater acoustic data; an AIS feature subnetwork for extracting dynamic behavior pattern features and static attribute features of ships from AIS data; and a feature fusion and association decision layer for performing multimodal fusion of the deep acoustic fingerprint features with the dynamic behavior pattern features and static attribute features of the ships, and outputting an association confidence matrix representing the association probability between the underwater acoustic target and the AIS ship. Step S3: Online Adaptive Association and Early Warning The real-time acquired underwater acoustic features and AIS features are input into the trained deep feature fusion network to obtain a real-time association confidence matrix. The decision threshold is dynamically calculated based on the current environmental complexity, and the results of successful association, marked as a suspected non-cooperative target, or confirmed association failure are output according to the comparison between the real-time association confidence matrix and the dynamic decision threshold.

2. The method for dynamic adaptive association between underwater acoustic targets and AIS ships based on multimodal deep learning according to claim 1, characterized in that: The construction of the multimodal association sample library in step S1 specifically includes: Data acquisition and synchronization: The data received by the underwater acoustic monitoring array and the AIS base station are synchronized in time and spatially registered using a unified time reference source; Data preprocessing and fragmentation: The original underwater acoustic signal is denoised and time-frequency transformed to generate a time-frequency spectrum, and divided into underwater acoustic analysis segments with fixed durations; AIS data is decoded, cleaned, correlated with points and generated with tracks, and AIS tracks are strictly aligned with the underwater acoustic analysis segments in time windows through interpolation and resampling. Sample labeling: Labeled samples are generated by combining automatic matching with manual verification, and challenging samples, including negative samples and adversarial samples, are actively constructed.

3. The method for dynamic adaptive association between underwater acoustic targets and AIS ships based on multimodal deep learning according to claim 1, characterized in that: In step S2 The underwater acoustic feature subnetwork uses a convolutional neural network and an attention mechanism to automatically extract deep acoustic fingerprint features related to ship radiated noise, taking underwater acoustic time-spectrum maps as input. The AIS feature subnetwork uses a sequence model to process the dynamic trajectory sequence of the ship to capture maneuvering patterns, and integrates the static attribute features of the ship with the contextual environment features to generate AIS semantic feature vectors. The feature fusion and association decision layer adopts a cross-attention mechanism, using underwater acoustic features as queries and AIS features as keys and values, to calculate attention-weighted AIS features, which are then fused with underwater acoustic features. Finally, the association confidence matrix is ​​output through a fully connected layer.

4. The method for dynamic adaptive association between underwater acoustic targets and AIS ships based on multimodal deep learning according to claim 1, characterized in that: The backbone network of the underwater acoustic feature subnetwork is ResNet-18, followed by a convolutional block attention module (CBAM). The AIS feature subnetwork uses gated recurrent units (GRUs) to process dynamic track sequences and uses embedding layers to process ship category features.

5. The method for dynamic adaptive association between underwater acoustic targets and AIS ships based on multimodal deep learning according to claim 1, characterized in that: In step S2, when training the deep feature fusion network, Focal Loss is used as the loss function, and data augmentation operations such as time shifting, frequency shifting, and adding random noise are performed on the underwater acoustic spectrogram. The training strategy is to first pre-train the underwater acoustic sub-network and the AIS sub-network separately, and then perform joint fine-tuning of the entire network.

6. The method for dynamic adaptive association between underwater acoustic targets and AIS ships based on multimodal deep learning according to claim 1, characterized in that: The dynamic calculation of the decision threshold in step S3 is as follows: τ = τ_base + α * target density + β * ambient noise level, where τ_base is the baseline threshold and α and β are adjustment coefficients.

7. The method for dynamic adaptive association between underwater acoustic targets and AIS ships based on multimodal deep learning according to claim 1, characterized in that: Step S3 also includes continuous optimization and incremental learning steps: The verification results of manually marked suspected non-cooperative targets are stored as new labeled samples in the sample library, and the incremental learning of the model is triggered. The incremental learning freezes the shallow structure of the deep feature fusion network, and only the fusion layer and decision layer are retrained and fine-tuned.

8. The method for dynamic adaptive association between underwater acoustic targets and AIS ships based on multimodal deep learning according to claim 1, characterized in that: The underwater acoustic data includes broadband spectrum, narrowband spectrum, bearing history, and depth information; the AIS data includes the ship's position, speed, heading, size, and type.

Images