A ship traffic situation awareness method, device, equipment and storage medium

Abstract

The application provides a ship traffic situation awareness method, device, equipment and storage medium, the method comprises: obtaining the radar feature and AIS feature of the target ship, and determining the attention weight; constructing a time differential consistency constraint according to the time variation rate of the attention weight and the time differential error of the radar feature and the AIS feature, the time differential consistency constraint is used to force the time variation rate to be inversely proportional to the time differential error; generating a fusion feature according to the constructed time differential consistency constraint and the attention weight; generating a ship trajectory prediction of the target ship according to the fusion feature. The application introduces a time differential consistency constraint and a delay perception attention module, accurately aligns and anti-oscillation fuses the radar and AIS features, generates stable fusion features, and then realizes accurate ship trajectory prediction, effectively improves the situation awareness accuracy and navigation safety.

Description

Technical Field

[0001] This application belongs to the field of maritime traffic, and in particular relates to a method, device, equipment and storage medium for maritime traffic situational awareness. Background Technology

[0002] Existing ship traffic situational awareness methods based on multi-source data fusion typically include key steps such as data acquisition, spatiotemporal alignment, feature fusion, and situational awareness generation. Specifically, the system first synchronously collects target sea area data from devices such as radar sensors, AIS receivers, and hydro-meteorological sensors, and outputs a structured dataset after processing. Next, a spatiotemporal alignment engine processes the multi-source data, completing spatiotemporal alignment and data association, and outputting a spatiotemporally aligned associated dataset. Then, a multimodal fusion module constructs various feature vectors for the associated dataset, uses an attention mechanism to generate fused feature vectors, and obtains the deep fused features of each ship target. Finally, the situational awareness engine predicts ship trajectories based on the fused features, calculates the nearest encounter distance (CPA) and nearest encounter time (TCPA) between ships, and generates collision risk warnings when conditions are met.

[0003] However, in practical applications, existing technologies have revealed a series of problems, especially in the process of multimodal data fusion. Due to the difference in sampling frequencies between radar and sensors such as AIS, millisecond-level time delays can occur, causing drastic fluctuations in the weights assigned to features of the same target in adjacent time steps. This amplification effect of time sensitivity directly leads to oscillations in the feature fusion process, manifested as high-frequency jumps in the fused features across consecutive time steps. This manifests as significant "jitter" in ship trajectory prediction, increasing prediction errors; in conflict warning, it easily triggers false alarms, leading to unnecessary alerts and interfering with normal ship traffic management.

[0004] These problems are highly insidious, almost invisible when data synchronization is good, and only become apparent when asynchrony exceeds the system's tolerance threshold. Moreover, because their manifestations are similar to sensor noise, they are easily misjudged, thus delaying problem-solving and posing potential threats to ship navigation safety. These issues severely restrict the performance improvement of existing ship traffic situational awareness methods and fail to meet the ever-increasing demands for high precision and high reliability in maritime safety. Summary of the Invention

[0005] The purpose of this application is to overcome the deficiencies in the prior art and provide a method, apparatus, device and storage medium for ship traffic situational awareness.

[0006] This application provides a method for ship traffic situational awareness, including:

[0007] Acquire the radar and AIS signatures of the target vessel;

[0008] Based on the probability distributions of the radar features and the AIS features, the attention weights are determined;

[0009] Based on the temporal rate of change of the attention weight and the temporal differential error between the radar feature and the AIS feature, a temporal differential consistency constraint is constructed, which is used to force the temporal rate of change to be inversely proportional to the temporal differential error;

[0010] Based on the constructed temporal differential consistency constraint and the attention weight, a fusion feature is generated;

[0011] Based on the fusion features, a predicted ship trajectory for the target vessel is generated.

[0012] Optionally, attention weights are determined based on the probability distributions of the radar features and the AIS features, including:

[0013] The radar features and the AIS features are spliced ​​together to form a spliced ​​feature;

[0014] The splicing features are input into a linear transformation matrix to generate the linearly transformed result;

[0015] The result after the linear transformation is normalized using softmax to determine the attention weights.

[0016] Optionally, construct temporal differential consistency constraints, including:

[0017] Construct a loss term, which is the product of the norm of the temporal derivative of the attention weight and the square norm of the temporal differential error;

[0018] Construct temporal differential consistency constraints based on the loss term.

[0019] Optionally, the timing differential error includes: calculation through a first-order low-pass filter, specifically including the following steps:

[0020] Obtain the average feature value of the radar feature and the AIS feature within a past time window;

[0021] The timing differential error is determined by calculating the difference between the average value of the feature and the current value of the radar feature or the AIS feature.

[0022] Optionally, construct temporal differential consistency constraints, including:

[0023] Perform a Fast Fourier Transform on the attention weight sequence to obtain the spectrum;

[0024] A power suppression operation is performed on the high-frequency components of the spectrum to generate a corrected spectrum;

[0025] Perform an inverse fast Fourier transform on the modified spectrum to obtain the modified attention weight sequence;

[0026] The temporal differential consistency constraint is constructed based on the modified attention weight sequence.

[0027] Optionally, based on the constructed temporal differential consistency constraint and the attention weights, a fusion feature is generated, including:

[0028] The radar features and the AIS features are spliced ​​together to form a spliced ​​feature;

[0029] The attention weights are used to weight the spliced ​​features to generate fused features.

[0030] Optionally, the calculation steps for the timing differential error include:

[0031] The width of the integration window is set to the maximum delay time constant of the sensor, and the time constant is determined based on the sensor characteristics.

[0032] A first-order low-pass filter is performed based on the integral window width to calculate the timing differential error.

[0033] This application also provides a ship traffic situation awareness device, including:

[0034] The acquisition module acquires the radar and AIS characteristics of the target vessel.

[0035] The weighting module determines the attention weights based on the probability distributions of the radar features and the AIS features;

[0036] The construction module constructs a temporal differential consistency constraint based on the temporal change rate of the attention weight and the temporal differential error between the radar feature and the AIS feature. The temporal differential consistency constraint is used to force the temporal change rate to be inversely proportional to the temporal differential error.

[0037] The fusion module generates fusion features based on the constructed temporal differential consistency constraint and the attention weight;

[0038] The generation module generates a predicted ship trajectory for the target ship based on the fused features.

[0039] This application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0040] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the above-described method.

[0041] The beneficial effects of this application are:

[0042] This application provides a method for ship traffic situational awareness, comprising: acquiring radar features and AIS features of a target ship; determining attention weights based on the probability distributions of the radar features and the AIS features; constructing a temporal differential consistency constraint based on the time change rate of the attention weights and the temporal differential error between the radar features and the AIS features, wherein the temporal differential consistency constraint is used to force the time change rate to be inversely proportional to the temporal differential error; generating fused features based on the constructed temporal differential consistency constraint and the attention weights; and generating a ship trajectory prediction of the target ship based on the fused features. This application, by introducing a temporal differential consistency constraint and a delay-aware attention module, performs precise time alignment and anti-oscillation fusion of radar and AIS features to generate stable fused features, thereby achieving accurate ship trajectory prediction and effectively improving situational awareness accuracy and navigation safety. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the ship traffic situation awareness process in this application;

[0044] Figure 2 This is a schematic diagram of the data acquisition and preprocessing process in this application;

[0045] Figure 3 This is a schematic diagram of the temporal differential consistency constraint for feature fusion in this application;

[0046] Figure 4 This is the first schematic diagram of dynamic threshold adjustment in this application;

[0047] Figure 5 This is the second schematic diagram of dynamic threshold adjustment in this application;

[0048] Figure 6 This is the third schematic diagram of dynamic threshold adjustment in this application. Detailed Implementation

[0049] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is to be understood that various forms of implementation of the present disclosure are intended and should not be limited to the embodiments set forth herein. Rather, the embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0050] Please refer to Figure 1As shown, this application provides a method for fault location and detection in communication equipment, including:

[0051] S101. Obtain the radar and AIS characteristics of the target vessel;

[0052] The distributed data acquisition nodes deployed on the port's shore base, anchorage buoys, and shipboard platforms enable synchronous data access from radar sensors, AIS receivers, and hydrological and meteorological sensors.

[0053] like Figure 2 As shown, the radar sensor outputs a two-dimensional or three-dimensional point cloud sequence, which includes sea clutter interference from non-target reflection sources such as sea waves and seabirds. To suppress interference and extract the true ship target trajectory, an adaptive Kalman filter is constructed, and the noise covariance matrix is ​​dynamically adjusted in the state transition model. The environmental dynamics are estimated in real time based on the changes in the inter-frame position residuals of the point cloud, thereby improving the steady-state extraction capability of the target trajectory. The filtered radar trajectory data is timestamped with UTC standard time for each target position and geocoded uniformly using the WGS-84 coordinate system. The AIS receiver parses the AIS messages received on the VHF channel, extracts the ship's unique identifier MMSI, current speed, heading, and message transmission time, aligns the timestamps to the second-level accuracy according to the message structure, and cleans up duplicate messages or incomplete content to ensure data consistency.

[0054] In one specific implementation, the process of cleaning duplicate or incomplete messages to ensure data consistency further includes: labeling the physical acquisition location and precise acquisition time of each data set received, and recording this information synchronously. The hydrological and meteorological sensors collect environmental factors such as visibility, wind speed, wind direction, air pressure, and wave height, which serve as auxiliary input for subsequent situation assessment.

[0055] like Figure 3 As shown, in order to address the inter-modal time delay issues between radar and AIS in terms of data update frequency, transmission link, and equipment synchronization accuracy, a delay estimation sub-network is constructed to automatically infer and compensate for the relative delay between modes.

[0056] In one specific implementation, the above-described construction of a delay estimation subnetwork for automatically inferring and compensating for relative delays between modalities further includes: the network input is the time derivative of the feature sequences of each modality, expressed as:

[0057]

[0058] in, The time differential form of each modal feature sequence, It is the feature vector of mode m at time t, and Δt is the sampling interval. The variation trend of the modal features in the time dimension is enhanced by the differentiation operation.

[0059] The delay estimation network structure consists of three fully connected neural networks. The hidden layers use the LeakyReLU activation function to enhance nonlinear expressive power and avoid neuron death problems. The output layer uses the Sigmoid activation function to map the delay prediction value δm to the standardized time range, which facilitates subsequent interpolation operations.

[0060] Linear time-shift compensation is performed on the AIS data based on the estimated latency value. ).

[0061]

[0062]

[0063] in, The compensated AIS features are shown, where t is the UTC timestamp of the radar sensor data, and δ is the value of the AIS feature. AIS This is the relative delay time for AIS. and They are earlier and later than t-δ, respectively. AIS The interpolation coefficient controls the weighting ratio of the compensated data to the two most recent AIS message timestamps. These are the interpolation weighting coefficients. For the original eigenvalues, δ AIS It is the estimated AIS latency output by the network.

[0064] This compensation method maintains the continuity of AIS data while aligning its time axis to the reference time of the radar trajectory.

[0065] The final output structured dataset is represented as follows:

[0066]

[0067] in, express The radar trajectory data at any given time includes the ship's position coordinates; express The AIS data at any given time includes the ship's MMSI identifier and motion status; express Real-time hydrological and meteorological data, including environmental parameters such as visibility and wave height.

[0068] This dataset serves as the input basis for subsequent steps, ensuring initial alignment of the multi-source data.

[0069] The "radar feature" is defined as the filtered ship trajectory position vector, which includes coordinates (x, y) and velocity components; the "AIS feature" is defined as the cleaned ship motion state vector, which includes speed v and heading θ.

[0070] The feature extraction process is achieved through an adaptive Kalman filter. For example, in a port environment, the filter dynamically adjusts the covariance matrix according to wave interference, reducing noise caused by sea clutter by 30% and ensuring stable output features.

[0071] S102. Determine the attention weights based on the probability distributions of the radar features and the AIS features;

[0072] The radar features and the AIS features are concatenated to form a concatenated feature; the concatenated feature is input into a linear transformation matrix to generate a linearly transformed result; the linearly transformed result is normalized using softmax to determine the attention weights.

[0073] Perform feature stitching operation to combine radar feature vectors With AIS feature vectors The connection forms a joint feature vector. ∥ Concatenation refers to merging the elements of two feature vectors in dimensional order into a single vector with a higher dimension. For example, if the radar feature dimension is 10 and the AIS feature dimension is 5, the concatenated vector will have a dimension of 15. This operation ensures the integration of different modal information in the feature space.

[0074] A linear transformation is then performed, and the concatenated features are input into the pre-trained weight matrix. Generate linear transformation results:

[0075]

[0076] in, It is a dimension of The transformation matrix is ​​denoted by d, where d is the output dimension. The linear transformation maps the feature space through matrix multiplication, for example, mapping a 15-dimensional input to an 8-dimensional output, thus enhancing the separability of features.

[0077] Finally, the attention weights are calculated using the softmax normalization function:

[0078]

[0079] Softmax normalization transforms the linear transformation result into a probability distribution, ensuring... ∈[0,1] and the weight sum is 1. This represents the fusion weight ratio of the radar and AIS modes at time t. For example, when the radar data quality is high, It may be close to 0.7 (radar) and 0.3 (AIS).

[0080] Dynamic weight allocation solves the priority problem of multimodal feature fusion and avoids the sensitivity problem caused by fixed weights.

[0081] S103. Based on the time change rate of the attention weight and the temporal differential error between the radar feature and the AIS feature, construct a temporal differential consistency constraint;

[0082] Constructing loss terms It is defined as the product of the L2 norm of the temporal derivative of the attention weight and the square norm of the temporal differential error between modes:

[0083]

[0084] in, The L2 norm represents the rate of change of weights, quantifying the amplitude of oscillations over time. The square norm represents the differential error between modes, measuring the difference in the trend of characteristic changes.

[0085] The forced time change rate of the loss term is inversely proportional to the differential error. For example, when the differential error increases, the penalty weight change rate of the loss term is reduced, thereby suppressing fusion oscillation.

[0086] During training, the Temporal Differential Consistency Loss (TDCL Loss) and the main task loss are jointly optimized to improve the robustness and temporal smoothness of the model under multi-source fusion.

[0087] In one specific implementation, the above-mentioned construction loss term also includes: the total loss function is defined as:

[0088]

[0089] Where T represents the time window, This represents the model's predicted output at time t. For real labels, (t) is the temporal differential consistency loss term at time t, and λ is the weighting coefficient between the two losses, the value of which is obtained through the gradient of the main task loss. gradient with consistency loss The ratio is adaptively adjusted to dynamically balance the optimal weights between accuracy and consistency.

[0090] The time-series differential error is calculated through a first-order low-pass filter, including: obtaining the average feature value of the radar feature and the AIS feature within a past time window; and calculating the difference between the average feature value and the current value of the radar feature or the AIS feature to determine the time-series differential error.

[0091] Calculate the time-series differential error First-order low-pass filtering method is used:

[0092]

[0093] Where m∈{r, a} represents the mode type, ΔFm(t) represents the low-pass differential characteristic of mode m at time t, and τ is the width of the integration window, set as the maximum possible level delay time constant of the sensor. The value of the modal feature at time point s.

[0094] This filter differential structure can effectively suppress the effects of transient noise, making the changes in attention weights smoother over time.

[0095] The integration window width is the maximum delay time constant of the sensor; for example, when the maximum radar delay is 100ms, τ=0.1s. The feature mean is calculated by integration of the feature mean within the past τ window, and the difference is calculated to represent the current feature mean. The difference from the mean.

[0096] Perform a Fast Fourier Transform on the attention weight sequence to obtain a spectrum; perform a power suppression operation on the high-frequency components of the spectrum to generate a corrected spectrum; perform an Inverse Fast Fourier Transform on the corrected spectrum to obtain a corrected attention weight sequence; and construct a temporal differential consistency constraint based on the corrected attention weight sequence.

[0097] When high-frequency oscillations are detected, perform spectral correction:

[0098] First, the attention weight sequence Perform a Fast Fourier Transform (FFT) to obtain the spectrum S(f). The FFT converts the time-domain sequence into a frequency-domain representation and identifies the frequency components.

[0099] Subsequently, power suppression is applied to the high-frequency components with f > 0.5 Hz, generating a corrected spectrum S′(f) = S(f)•β, where β = 0.1 corresponds to a 10 dB attenuation. The power suppression operation reduces high-frequency energy through multiplication.

[0100] Finally, perform an inverse fast Fourier transform on S′(f) to obtain the corrected sequence. This sequence is used to replace the original weights to construct constraints. For example, the amplitude of the 0.8Hz component in the original sequence is reduced from 0.5 to 0.05 to eliminate weight abrupt changes.

[0101] The temporal differential consistency constraint is constructed based on the loss term and the correction sequence, forcing smooth weight changes. This constraint serves as a regularization term during training; for example, when ship acceleration causes asynchronous radar and AIS data, the constraint mechanism limits the weight change rate to within ±10%, preventing abrupt changes in fused features.

[0102] S104. Generate fusion features based on the constructed temporal differential consistency constraint and the attention weight;

[0103] The radar features and the AIS features are concatenated to form a concatenated feature; the attention weights are used to weight the concatenated feature to generate a fused feature.

[0104] Perform feature stitching operation to combine radar features With AIS features splicing together to form joint features ⊕ ].

[0105] Feature concatenation is achieved through vector concatenation, for example, merging position and velocity features into a single vector. Then, attention weights αr and a(t) are used to weight the concatenated features:

[0106]

[0107] The weighting operation applies weights to each element of the concatenated feature, under the constraint of temporal differential consistency. For example, when When the value is [0.6, 0.4], radar features contribute 60% and AIS features contribute 40%, and the constraint ensures that the weight changes gradually.

[0108] Final output fused features As an input for trajectory prediction, it maintains feature continuity during ship maneuvering and avoids high-frequency jumps.

[0109] S105. Based on the fusion features, generate a predicted ship trajectory for the target ship;

[0110] Fuse feature sequences Input a Long Short-Term Memory (LSTM) neural network to predict the ship's position (x, y), speed v, and heading θ sequence within the next 30 seconds.

[0111] Calculate the nearest encounter distance (CPA) and nearest encounter time (TCPA) between ships based on the predicted trajectory:

[0112]

[0113] Where p(t) is the position vector, and i and j represent time points.

[0114] like Figure 6 , Figure 4 As shown, a basic CPA threshold is set according to the combination of ship types (e.g., 0.8 nautical miles between cargo ships, 1.2 nautical miles between cargo ships and fishing boats, and 1.5 nautical miles between cruise ships and other ships), and dynamically adjusted: when visibility is <1 nautical mile and wave height is >3 meters, the threshold is increased by 20%; when the uncertainty of the fusion feature position is >0.1, the threshold is decreased by 15%.

[0115] The ship type combinations include: cargo ship-cargo ship, cargo ship-fishing vessel, cruise ship-any combination, etc.

[0116] like Figure 5 As shown, in a specific implementation, the above-mentioned threshold is increased by 20% when visibility is <1 nautical mile and wave height is >3 meters, and the threshold is decreased by 15% when the uncertainty of the fused feature position is >0.1. This further includes: when visibility is less than 1 nautical mile and wave height is more than 3 meters, the current threshold is multiplied by 1.2 to increase the threshold, enhancing collision sensitivity; when the position uncertainty index in the fused feature exceeds 0.1, the current threshold is multiplied by 0.85 to decrease the threshold, improving recognition robustness.

[0117] Trajectory prediction uses LSTM to achieve temporal modeling, such as outputting the position sequence for the next 30 steps; collision risk assessment combines environmental factors to improve the accuracy of early warning.

[0118] This application also provides a ship traffic situation awareness device, including:

[0119] The acquisition module acquires the radar and AIS characteristics of the target vessel.

[0120] The weighting module determines the attention weights based on the probability distributions of the radar features and the AIS features;

[0121] The construction module constructs a temporal differential consistency constraint based on the temporal change rate of the attention weight and the temporal differential error between the radar feature and the AIS feature. The temporal differential consistency constraint is used to force the temporal change rate to be inversely proportional to the temporal differential error.

[0122] The fusion module generates fusion features based on the constructed temporal differential consistency constraint and the attention weight;

[0123] The generation module generates a predicted ship trajectory for the target ship based on the fused features.

[0124] This application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0125] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the above-described method.

[0126] The above description of the embodiments is provided to enable those skilled in the art to understand and apply this application. Those skilled in the art will readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without inventive effort. Therefore, this application is not limited to the above embodiments, and any improvements and modifications made to this application based on the disclosure thereof should be within the scope of protection of this application.

Claims

1. A method for ship traffic situational awareness, characterized in that, include: Acquire the radar and AIS signatures of the target vessel; Based on the probability distributions of the radar features and the AIS features, the attention weights are determined; Based on the temporal rate of change of the attention weight and the temporal differential error between the radar feature and the AIS feature, a temporal differential consistency constraint is constructed, which is used to force the temporal rate of change to be inversely proportional to the temporal differential error; Based on the constructed temporal differential consistency constraint and the attention weight, a fusion feature is generated; Based on the fusion features, a predicted ship trajectory for the target vessel is generated.

2. The method for ship traffic situational awareness according to claim 1, characterized in that, Based on the probability distributions of the radar features and the AIS features, attention weights are determined, including: The radar features and the AIS features are spliced ​​together to form a spliced ​​feature; The splicing features are input into a linear transformation matrix to generate the linearly transformed result; The result after the linear transformation is normalized using softmax to determine the attention weights.

3. The method for ship traffic situational awareness according to claim 1, characterized in that, Constructing temporal differential consistency constraints includes: Construct a loss term, which is the product of the norm of the temporal derivative of the attention weight and the square norm of the temporal differential error; Construct temporal differential consistency constraints based on the loss term.

4. The method for ship traffic situational awareness according to claim 1, characterized in that, The timing differential error includes calculation through a first-order low-pass filter, specifically including the following steps: Obtain the average feature value of the radar feature and the AIS feature within a past time window; The timing differential error is determined by calculating the difference between the average value of the feature and the current value of the radar feature or the AIS feature.

5. The method for ship traffic situational awareness according to claim 1, characterized in that, Constructing temporal differential consistency constraints includes: Perform a Fast Fourier Transform on the attention weight sequence to obtain the spectrum; A power suppression operation is performed on the high-frequency components of the spectrum to generate a corrected spectrum; Perform an inverse fast Fourier transform on the modified spectrum to obtain the modified attention weight sequence; The temporal differential consistency constraint is constructed based on the modified attention weight sequence.

6. The method for ship traffic situational awareness according to claim 1, characterized in that, Based on the constructed temporal differential consistency constraint and the attention weights, fused features are generated, including: The radar features and the AIS features are spliced ​​together to form a spliced ​​feature; The attention weights are used to weight the spliced ​​features to generate fused features.

7. The method for ship traffic situational awareness according to claim 1, characterized in that, The calculation steps for the timing differential error include: The width of the integration window is set to the maximum delay time constant of the sensor, and the time constant is determined based on the sensor characteristics. A first-order low-pass filter is performed based on the integral window width to calculate the timing differential error.

8. A ship traffic situational awareness device, characterized in that, include: The acquisition module acquires the radar and AIS characteristics of the target vessel. The weighting module determines the attention weights based on the probability distributions of the radar features and the AIS features; The construction module constructs a temporal differential consistency constraint based on the temporal change rate of the attention weight and the temporal differential error between the radar feature and the AIS feature. The temporal differential consistency constraint is used to force the temporal change rate to be inversely proportional to the temporal differential error. The fusion module generates fusion features based on the constructed temporal differential consistency constraint and the attention weight; The generation module generates a predicted ship trajectory for the target ship based on the fused features.

9. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed in a computer, causes the computer to perform the method described in any one of claims 1-7.

Images