A shore-ship communication early warning system based on multi-mode communication technology

Abstract

This invention discloses a shore-to-ship communication early warning system based on multimodal communication technology, comprising: a speech recognition and command generation module for receiving voice content from shore stations and VHF radio stations; a communication strategy learning module for strategy learning using Q-learning; a multimodal graph construction module for extracting verification command text; a spatiotemporal graph neural network processing module for generating spatiotemporal embedding features; an improved TranAD prediction module for inputting the spatiotemporal embedding features into an improved TranAD model; a cooperative adversarial training module for constructing temporal and spatial branches; a cross-feedback constraint module for generator backpropagation; an anomaly scoring and early warning output module for generating anomaly scores; and a model update and deployment module for updating and deploying the improved TranAD model. This invention can achieve communication assurance and safety early warning in maritime scenarios.

Description

Technical Field

[0001] This invention relates to the field of intelligent communication and maritime safety technology, and in particular to a shore-to-ship communication early warning system based on multi-mode communication technology. Background Technology

[0002] Traditional shore-to-ship communication relies primarily on manual operation and a single communication mode, such as VHF voice radio for command interaction. This approach suffers from problems like unclear voice quality, frequent interference, and reliance on human experience for content recognition, making efficient and accurate communication management difficult. Especially in the complex maritime environment, voice channels are susceptible to interference from weather, electromagnetic fields, and other factors, resulting in low command reception accuracy. Furthermore, existing systems are mostly driven by static rules, lacking the ability to automatically identify and warn of communication anomalies, making them unsuitable for increasingly complex multi-ship collaborative communication scenarios and posing a potential threat to maritime traffic safety.

[0003] In recent years, with the development of artificial intelligence and edge computing, spatiotemporal modeling methods based on graph neural networks and unsupervised anomaly detection models such as TranAD have been gradually introduced into the field of communication security, enabling the modeling and monitoring of ship communication behavior. However, these methods are mostly limited to single-modal feature modeling and lack the ability to jointly model voice commands, ship trajectories, and channel states, making it difficult for the models to fully understand and integrate multi-source data in practical applications. In addition, existing model training methods are mostly single-channel optimization, lacking reinforcement learning and adversarial training mechanisms to ensure consistency between temporal and spatial information, thus limiting the model's discriminative ability. At the same time, model deployment still relies on high-performance servers, which are not suitable for embedded scenarios, limiting the widespread application of real-time early warning systems in edge nodes.

[0004] To address the aforementioned issues, current technology has not yet developed a systematic solution capable of achieving a fully intelligent closed-loop process between shore stations and target vessels, encompassing voice recognition, communication strategy learning, spatiotemporal anomaly detection, and early warning dissemination. The lack of dynamic model updates and pruning / quantization mechanisms also leads to performance degradation and high resource consumption after long-term deployment, making it difficult to meet the dual requirements of real-time performance and low power consumption at sea.

[0005] Therefore, how to provide a shore-to-ship communication early warning system based on multi-mode communication technology is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a shore-to-ship communication early warning system based on multi-mode communication technology. This invention integrates speech recognition, graph neural network modeling, Q-learning policy learning, and TranAD anomaly detection technology. It constructs a multi-modal input tensor that includes speech features, trajectory features, and channel state. It uses a spatiotemporal graph neural network and an improved TranAD model for collaborative dual adversarial training, and improves the spatiotemporal consistency modeling capability through cross-feedback constraints. The complete process of communication behavior anomaly detection, policy output, and lightweight model deployment is described in detail. It has the advantages of high recognition accuracy, timely early warning response, low deployment resources, and high degree of intelligence in communication decision-making.

[0007] A shore-to-ship communication early warning system based on multimode communication technology according to an embodiment of the present invention includes: The voice recognition and command generation module is used to receive voice content from shore stations and VHF radio stations and send inspection command text to the target vessel. The communication strategy learning module is used to collect channel status data of target ships, execute association rule algorithms to generate anomaly tags, and combine Q-learning algorithms to generate communication transmission timing and channel selection instructions. The multimodal graph construction module is used to extract semantic features from the verification instruction text, construct a multimodal graph, and generate a multimodal input sheet. The spatiotemporal graph neural network processing module is used to input multimodal input tensors into the spatiotemporal graph neural network layer with attention mechanism to generate spatiotemporal embedding features; The improved TranAD prediction module is used to input spatiotemporal embedding features into the encoder and decoder of the improved TranAD model to generate a reconstruction error sequence; The collaborative adversarial training module is used to construct temporal adversarial branches and spatial adversarial branches. The improved TranAD model and the spatiotemporal graph neural network layer are used as generators, respectively, and adversarial training is carried out in combination with the corresponding discriminator. The cross-feedback constraint module is used to apply temporal correlation features and spatial structure representation as feedback information to the backpropagation of the generator. The anomaly scoring and early warning output module is used to generate an anomaly score by weighted fusion of the reconstruction error sequence, temporal discrimination probability score and spatial discrimination probability score, and to output communication transmission timing and channel selection instructions in conjunction with the score. The model update and deployment module is used to build incremental datasets and distribute the pruned and INT4 quantized improved TranAD model to the ship's edge nodes to output early warning information in real time.

[0008] Optionally, modules can be integrated using the following methods: Step 1: Receive voice content from shore station and VHF radio, perform voice recognition and keyword extraction, trigger the inspection process based on the recognition results, and transmit the inspection command text to the target vessel via VHF radio. Step 2: Collect channel state data, solve the strategy using the Q-learning algorithm, and output communication transmission timing instructions and channel selection instructions; combine AIS positioning data and verification instruction text to construct a multimodal graph and construct a spatiotemporal input tensor; Step 3: Input the multimodal input tensor into the spatiotemporal graph neural network layer based on the attention mechanism to model spatial and temporal dependencies and output spatiotemporal embedding features; insert the spatiotemporal graph neural network layer based on the attention mechanism between the encoder and decoder of the improved TranAD model; Step 4: Input the spatiotemporal embedding features into the improved TranAD encoder and decoder to perform self-supervised reconstruction, generate a reconstruction sequence, and record the reconstruction error sequence; Step 5: Establish a collaborative dual adversarial mechanism, in which the temporal adversarial branch is composed of TranAD as the generator and a temporal discriminator, and the spatial adversarial branch is composed of a spatiotemporal graph neural network layer containing an attention mechanism as the generator and a spatial discriminator. Step 6: Generate an anomaly score based on the reconstruction error sequence, temporal discrimination probability score, and spatial discrimination probability score; generate early warning information by combining it with a preset threshold; and output the early warning information, communication transmission timing instruction, and channel selection instruction. Step 7: Send the pruned and quantized improved TranAD parameters to the ship's edge node, and output real-time early warning information and corresponding communication transmission timing and channel selection commands.

[0009] Optionally, step one specifically includes: The shore-to-ship communication early warning system receives voice content input from shore stations and VHF radio stations, and uses a voice recognition algorithm to convert the voice content into text content with timestamps. Based on a preset keyword rule base, the converted text content is identified for keywords, and the keywords are extracted and compared with the key phrases for triggering verification defined in the preset keyword rule base; When the recognition result contains a key phrase that triggers the verification, the triggering condition is met, and the verification process and listening function are automatically started. The verification process specifically includes: The target vessel is selected through the shore-to-ship communication early warning system and its AIS positioning data is integrated for target identification. The system receives verification content, including verification text input and recorded voice input. It performs a text preprocessing algorithm on the verification text input to generate a structured text draft, and performs segmentation and buffering processing on the voice input to form a voice segment sequence. Variational mode decomposition and wavelet packet entropy filtering are performed sequentially on the speech slice sequence to obtain the purified speech sequence; Extract the Mel spectrogram and Mel cepstral coefficient features from the cleaned speech sequence to generate a speech feature sequence; The speech feature sequence is aligned with phoneme-level Chinese-English mixed speech using a dynamic time warping algorithm to generate a draft of the instruction text corresponding to the time axis. Based on the keywords of the initial draft of the instruction text and the initial draft of the structured text, the template matching algorithm calls the preset shouting template to generate the verification instruction text; A text-to-speech algorithm is used to convert the inspection command text into natural speech, generate a voice packet for transmission, and drive a VHF radio to send the voice packet as an inspection command to the target ship.

[0010] Optionally, step two specifically includes: The shore-to-ship communication early warning system uses keyword recognition to trigger the listening function, drives the VHF radio to receive voice packets, parses the voice packets in real time to obtain the real-time listening content, and aggregates it with the historical inspection command text. An association rule mining algorithm is used to analyze historical verification command texts and real-time monitoring content to generate anomaly tags; the association rule mining algorithm is the Apriori algorithm. Collect and update the channel state data of the target vessel to form a channel state data sequence; Construct a channel state-action mapping based on channel state data; The channel state-action mapping and anomaly label are input into the Q-learning-based policy learning module to update the state-action values ​​and output the communication transmission timing command and channel selection command. Obtain the AIS positioning data of the target vessel and the corresponding inspection command text, and extract trajectory features and semantic features; The trajectory features are assigned to the ship nodes and the semantic features are assigned to the speech nodes, and the ship nodes and speech nodes are respectively constructed as the basic node set of the multimodal graph; Based on the geographical proximity between ships, the communication event correlation in the inspection instruction text, and the temporal sequence between nodes, a graph structure connection composed of spatial edges, semantic edges, and temporal edges is established between ship nodes and voice nodes. The trajectory features, semantic features, edge relationships, and channel state data sequences associated with nodes in the constructed multimodal graph are uniformly encoded to generate a multimodal input tensor.

[0011] Optionally, step three specifically includes: The multimodal input tensor is fed into a spatiotemporal graph neural network layer based on an attention mechanism; The spatiotemporal graph neural network layer aggregates features of the spatial connection relationships between each node and its neighboring nodes in the multimodal graph through graph convolution operations. Based on the adjacency matrix constructed by spatial edges, semantic edges and temporal edges, it calculates the weighted feature sum of the node and its neighboring nodes according to the edge weight coefficients, and generates spatial feature representations through mapping by nonlinear activation functions. The spatiotemporal graph neural network layer further performs temporal encoding operations on the feature sequences of each node at multiple time steps. By introducing positional encoding into the node feature sequences, and employing a multi-head self-attention mechanism, the temporal feature representation between any time steps is calculated. Spatial and temporal feature representations are fused and encoded to generate unified spatiotemporal embedding features.

[0012] Optionally, step four specifically includes: The spatiotemporal embedding features are input into the encoder of the improved TranAD model; The spatiotemporal embedding features are compressed using an encoder to generate a latent spatial representation; The latent space representation is input into the decoder of the TranAD model, and a self-supervised reconstruction operation is performed to recover the predicted sequence. The difference between the predicted sequence and the spatiotemporal embedded features is compared to generate the corresponding reconstruction error sequence.

[0013] Optionally, the steps of the coordinated dual-countermeasure mechanism specifically include: Construct a temporal adversarial branch and use the improved TranAD model as the generator. , generator The output predicted sequence is input into the time series discriminator; The time-series discriminator comprises, in sequence, an input layer, a bidirectional long short-term memory network layer, an attention mechanism layer, and a fully connected classification layer; The generator is determined by the timing discriminator. The output predicted sequence is temporally discriminated against with the corresponding spatiotemporal embedding features to generate temporal discrimination results; Constructing a spatial adversarial branch, using an attention-based spatiotemporal graph neural network layer as the generator. , generator The spatiotemporal embedding features output are input into the spatial discriminator; The spatial discriminator includes a graph structured input layer, a graph convolutional layer, a graph-level pooling layer, and a fully connected classification layer; The generator is determined by the spatial discriminator. The output spatiotemporal embedding features are compared with the corresponding multimodal input tensor to generate spatial discrimination results; The temporal and spatial discrimination results are trained in parallel to improve the generator. Generator The parameters are updated separately for the discriminator and the temporal adversarial branch, and cross-feedback constraints are applied to the temporal adversarial branch and the spatial adversarial branch during the parameter update process; Synchronous Adjustment Generator Based on Cross-Feedback Constraints and generator This ensures consistency between temporal and spatial features, completing the collaborative dual-adversarial training process.

[0014] Optionally, the generator Generator The parameters are updated separately for both the discriminator and the temporal adversarial branch, and cross-feedback constraints are applied to the temporal adversarial branch and the spatial adversarial branch during the parameter update process, specifically including: During the parameter update process, the temporal correlation features generated by the bidirectional long short-term memory network layer and the attention mechanism layer in the temporal adversarial branch are transmitted as temporal feedback information to the spatial adversarial branch. The temporal correlation feature is the forward and backward hidden state sequences of the bidirectional long short-term memory network layer and the temporal representation vector generated by the attention mechanism.

[0015] Furthermore, the spatial structure representation generated by graph convolutional layers and graph-level pooling layers in the spatial adversarial branch is transmitted as spatial feedback information to the temporal adversarial branch; The spatial structure representation refers to the graph-level spatial embedding vector generated by the graph-level pooling layer and the node spatial feature map output by the graph convolutional layer.

[0016] Under cross-feedback constraints, temporal feedback information and spatial feedback information are introduced into the generator as additional adjustment terms. With generator In the backpropagation update of the parameters, the two generators adjust the generation direction of the predicted sequence and spatiotemporal embedded features based on the temporal feedback information and spatial feedback information during the parallel training phase.

[0017] Optionally, step six specifically includes: The reconstructed error sequence, the temporal discrimination probability score output by the temporal discriminator, and the spatial discrimination probability score output by the spatial discriminator are weighted and fused to generate a unified anomaly scoring result. The abnormal score is compared with a preset threshold. If the abnormal score exceeds the preset threshold, it is determined that the current ship communication behavior is abnormal, and the abnormal category is determined according to the discrimination label corresponding to the abnormal score. Based on the anomaly level classification standard, the anomaly score is mapped to the shore-to-ship communication early warning level; The generated shore-to-ship communication early warning level and anomaly category label are combined as early warning information output, and the communication transmission timing instruction and channel selection instruction generated by Q-learning are output in conjunction.

[0018] Optionally, step seven specifically includes: Write the abnormal samples corresponding to the abnormal scoring results and the operation log of the shore-to-ship communication early warning system to the background data storage log. The improved TranAD model is retrained using an incremental dataset constructed from anomalous samples and runtime logs. After retraining, dynamic pruning is performed on the spatiotemporal graph neural network layer based on the attention mechanism embedded in the updated improved TranAD model. The low-importance neural network parameter nodes in the spatiotemporal graph neural network structure are pruned according to the node importance score index to generate a simplified improved TranAD model structure. The weight parameters of the pruned improved TranAD model were subjected to INT4 quantization. After pruning and quantization, the parameters of the improved TranAD model are sent to the edge computing nodes deployed at the ship's end. An improved TranAD model is loaded and deployed on an embedded computing platform. Combining the latest multimodal input tensor and channel state data, it outputs real-time warning information and corresponding communication transmission timing instructions and channel selection instructions.

[0019] The beneficial effects of this invention are: This invention constructs a multimodal input tensor integrating speech recognition, trajectory analysis, channel state awareness, and semantic information mining. This effectively fuses the spatial, temporal, and semantic features inherent in ship communication behavior, thereby enhancing the ability to model the patterns of communication behavior. Compared to traditional anomaly identification methods based on a single data source, this multimodal fusion mechanism significantly improves the model's adaptability and recognition accuracy in complex communication scenarios.

[0020] By combining the improved TranAD model with a spatiotemporal graph neural network based on an attention mechanism, a collaborative dual adversarial training mechanism is used to deeply model communication behavior. While retaining the self-supervised reconstruction advantage of TranAD, temporal and spatial discriminators are introduced to evaluate the temporal and spatial consistency of the generated data, respectively. The generator is guided to continuously optimize its prediction ability through cross-feedback constraints, which effectively solves the problems of overfitting to local patterns and poor generalization ability in anomaly detection in existing methods.

[0021] By employing a Q-learning-based communication strategy learning module that combines channel state data and abnormal pattern labels, the system can dynamically output optimal communication transmission timing and channel selection commands. This enables intelligent scheduling of communication resources and interference avoidance, further enhancing the reliability and stability of shore-to-ship communication. Simultaneously, the system supports incremental retraining and dynamic pruning optimization of the model. After compressing model parameters using INT4 quantization, it is deployed to shipboard edge computing nodes, achieving lightweight, low-power, and high-real-time communication early warning inference on an embedded platform. This overcomes the bottleneck of traditional models struggling to run on resource-constrained terminals.

[0022] In summary, this invention improves the accuracy of shore-to-ship communication anomaly detection while realizing intelligent output of communication strategies and efficient deployment of models, demonstrating good engineering feasibility and application promotion value. Attached Figure Description

[0023] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the overall structure of a shore-to-ship communication early warning system based on multi-mode communication technology proposed in this invention. Figure 2 This is an overall flowchart of a shore-to-ship communication early warning method based on multi-mode communication technology proposed in this invention; Figure 3 This is a schematic diagram of the collaborative dual-countermeasure mechanism of a shore-to-ship communication early warning system based on multi-mode communication technology proposed in this invention. Detailed Implementation

[0024] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0025] refer to Figure 1-3 A shore-to-ship communication early warning system based on multi-mode communication technology includes: The voice recognition and command generation module is used to receive voice content from shore stations and VHF radio stations and send inspection command text to the target vessel. The communication strategy learning module is used to collect channel status data of target ships, execute association rule algorithms to generate anomaly tags, and combine Q-learning algorithms to generate communication transmission timing and channel selection instructions. The multimodal graph construction module is used to extract semantic features from the verification instruction text, construct a multimodal graph, and generate a multimodal input tensor. The spatiotemporal graph neural network processing module is used to input multimodal input tensors into the spatiotemporal graph neural network layer with attention mechanism to generate spatiotemporal embedding features; The improved TranAD prediction module is used to input spatiotemporal embedding features into the encoder and decoder of the improved TranAD model to generate a reconstruction error sequence; The collaborative adversarial training module is used to construct temporal adversarial branches and spatial adversarial branches. The improved TranAD model and the spatiotemporal graph neural network layer are used as generators, respectively, and adversarial training is carried out in combination with the corresponding discriminator. The cross-feedback constraint module is used to apply temporal correlation features and spatial structure representation as feedback information to the backpropagation of the generator. The anomaly scoring and early warning output module is used to generate an anomaly score by weighted fusion of the reconstruction error sequence, temporal discrimination probability score and spatial discrimination probability score, and to output communication transmission timing and channel selection instructions in conjunction with the score. The model update and deployment module is used to build incremental datasets and distribute the pruned and INT4 quantized improved TranAD model to the ship's edge nodes to output early warning information in real time.

[0026] In this embodiment, the modules are interconnected using the following method: Step 1: Receive voice content from shore station and VHF radio, perform voice recognition and keyword extraction, trigger the inspection process based on the recognition results, and transmit the inspection command text to the target vessel via VHF radio. Step 2: Collect channel state data, solve the strategy using the Q-learning algorithm, and output communication transmission timing instructions and channel selection instructions; combine AIS positioning data and verification instruction text to construct a multimodal graph and construct a spatiotemporal input tensor; Step 3: Input the multimodal input tensor into the spatiotemporal graph neural network layer based on the attention mechanism to model spatial and temporal dependencies and output spatiotemporal embedding features; insert the spatiotemporal graph neural network layer based on the attention mechanism between the encoder and decoder of the improved TranAD model; Step 4: Input the spatiotemporal embedding features into the improved TranAD encoder and decoder to perform self-supervised reconstruction, generate a reconstruction sequence, and record the reconstruction error sequence; Step 5: Establish a collaborative dual adversarial mechanism, in which the temporal adversarial branch is composed of TranAD as the generator and a temporal discriminator, and the spatial adversarial branch is composed of a spatiotemporal graph neural network layer containing an attention mechanism as the generator and a spatial discriminator. Step 6: Generate an anomaly score based on the reconstruction error sequence, temporal discrimination probability score, and spatial discrimination probability score; generate early warning information by combining it with a preset threshold; and output the early warning information, communication transmission timing instruction, and channel selection instruction. Step 7: Send the pruned and quantized improved TranAD parameters to the ship's edge node, and output real-time early warning information and corresponding communication transmission timing and channel selection commands.

[0027] In this embodiment, step one specifically includes: The shore-to-ship communication early warning system receives voice content input from shore stations and ships, and uses a voice recognition algorithm to convert the voice content into text content with timestamps. The speech recognition algorithm is implemented through a speech-to-text interface, which connects to an external speech recognition service using a REST API-based remote call method and implements streaming transmission of speech data based on a WebSocket long connection mechanism. The speech recognition algorithm is based on the Whisper model, which is an end-to-end multilingual speech recognition model. It adopts an encoder-decoder structure and has the capabilities of real-time recognition, speech segmentation, timestamp annotation, and mixed Chinese and English speech recognition. Based on a preset keyword rule base, the converted text content is identified for keywords, and the keywords are extracted and compared with the key phrases for triggering verification defined in the preset keyword rule base; When the recognition result contains a key phrase that triggers the verification, the triggering condition is met, and the verification process and listening function are automatically started. Triggering conditions for the inspection process: Identification of inspection-related keywords and phrases, including: "perform a security check" and "please dock for inspection"; The listening function can be triggered either by manually clicking the "Start Listening" button or automatically by recognizing response keywords, including: "Request to enter port", "Reply to berthing", and "Received". The verification process specifically includes: The target vessel is selected through the shore-to-ship communication early warning system and its AIS positioning data is integrated for target identification. The target identification information and communication parameters, including channel number, location coordinates and identification mark, are obtained. The system receives verification content, including verification text input and recorded voice input. It performs text preprocessing algorithms to standardize the format of text input and generate a structured text draft. It performs segmentation and buffering processing on voice input to form a voice segment sequence. Long voice messages are segmented into 500ms segments and correctly reassembled when received out of order by marking them with sequence numbers. The buffering process includes dividing the recorded audio into frames according to a set time window and storing them sequentially into a buffer queue to support the segmentation processing of real-time audio streams and the stable input of subsequent feature extraction modules. The text preprocessing algorithm includes word segmentation and part-of-speech tagging, regularization rule matching, spell correction and command template matching, which are used to clean, standardize and semantically structure the input text content.

[0028] Variational mode decomposition and wavelet packet entropy filtering are performed sequentially on the speech slice sequence to obtain the purified speech sequence; The variational mode decomposition process performs frequency band decomposition on the speech segment sequence based on the sliding window method, decomposes the speech signal into several modal components, and dynamically determines the modes to be retained based on the energy distribution, while filtering out high-frequency interference components. The wavelet packet entropy filtering process uses a wavelet packet decomposition algorithm to decompose the signal into multi-scale sub-bands and calculates the entropy value of each sub-band to measure the frequency domain disturbance intensity. The dominant disturbance component is filtered out based on a preset entropy threshold. The entropy threshold is adaptively updated based on statistical information of historical speech features within the window, and is used to achieve dynamic adjustment of filter coefficients; By combining the aforementioned variational mode decomposition and adaptive wavelet packet entropy filtering, real-time noise reduction processing can be performed on mixed background noise including ocean wave noise and engine interference, and the purified speech sequence can be output.

[0029] Extract the Mel spectrogram and Mel cepstral coefficient features from the cleaned speech sequence to generate a speech feature sequence; A Mel spectrum calculation algorithm combining short-time Fourier transform and Mel filter bank is used to extract the Mel spectrum of the cleaned speech sequence; the Mel frequency cepstral coefficients are calculated by discrete cosine transform to generate the speech feature sequence.

[0030] The speech feature sequence is aligned at the phoneme level with mixed Chinese and English speech using a dynamic time warping algorithm, generating an alignment result and a draft of the instruction text corresponding to the time axis. Based on the keywords of the initial draft of the instruction text and the initial draft of the structured text, the template matching algorithm calls the preset shouting template to generate the verification instruction text; The template matching algorithm is based on a keyword rule base. It extracts keywords and compares semantic fragments on the initial draft of the instruction text, and calls the matched preset shout template to generate the verification instruction text.

[0031] A text-to-speech algorithm is used to convert the inspection command text into natural speech, generating a voice packet for transmission. This voice packet will be used by the shore station to send the inspection command via VHF radio.

[0032] The text-to-speech algorithm includes the FastSpeech speech synthesis model based on the non-autoregressive Transformer architecture and the HiFi-GAN vocoder. The FastSpeech model converts the verification instruction text into a speech spectrogram, and the HiFi-GAN converts the spectrogram into a playable speech waveform, ultimately generating natural speech.

[0033] The voice packets are acquired and transmitted in real time using a C language program, with a sampling rate of 44.1 kHz and a PCM encoding format. During the transmission of the voice packets, a Hanning window function is used to window the voice waveform to reduce spectral leakage and optimize subsequent filtering and recognition effects, thereby improving the robustness of the voice packets in the wireless communication link.

[0034] Speech recognition algorithms and text-to-speech algorithms are applied to the receiving and transmitting sides of the communication, respectively, undertaking the tasks of speech understanding and speech generation, and working together to build a complete voice interaction link. In this embodiment, step two specifically includes: The shore-to-ship communication early warning system uses keyword recognition to trigger the listening function. The receiving program of the integrated satellite communication box controls the receiving channel, drives the VHF radio to receive voice packets, and parses the voice packets in real time to obtain the real-time listening content, which is then aggregated with the historical inspection command text. During voice packet reception, beamforming algorithms are used to dynamically adjust the phase of multiple sub-antennas to form a directional receiving beam to improve the signal reception gain of the target ship. Combined with spatial diversity reception technology, multipath signal combining algorithms are used to reduce signal fading interference and improve the quality of voice signal reception.

[0035] The association rule mining algorithm is used to analyze the historical inspection command text and real-time monitoring content to generate ship communication behavior patterns and abnormal pattern labels. The association rule mining algorithm employs the Apriori algorithm to extract semantic information from historical verification command texts and real-time monitoring content, constructing structured data records. Each structured data record contains at least the following fields: ship identifier, communication object, communication time, channel number, keywords, semantic category, and communication action, etc. Each structured data record is treated as a transaction sample and combined to form a transaction database; the feature items in each transaction (such as "call instruction", "channel CH16", "ship A", "request to dock", etc.) are used as the candidate set for the algorithm; The Apriori algorithm is used to calculate the support of feature item combinations in the transaction database, and high-frequency itemsets are selected to obtain the patterns of ship communication behaviors that occur frequently in specific scenarios. Several communication behavior association rules are generated based on the association strength (confidence) and conditional probability relationship between frequent itemsets. For example: "If the keyword is 'dock' and the channel is CH16, then the target vessel type is 'cargo ship'". The transaction items corresponding to the real-time monitored content are matched with the generated set of association rules; if a certain communication behavior does not meet any high confidence rule, or if a low support item appears in the combination of frequent items, it is considered to deviate from normal behavior; Anomaly pattern labels are generated for the above-mentioned deviation behaviors, including: anomaly type (such as keyword anomaly, channel mismatch, semantic conflict), corresponding anomaly field, deviation rule ID, confidence value and event timestamp, etc., for subsequent modules to detect anomalies and correct strategies.

[0036] Collect and update the channel status data of the target ship, including channel signal-to-noise ratio, bandwidth occupancy and interference indicators, to form a channel status data sequence; A channel state-action mapping is constructed based on channel state data, and the action corresponding to each channel state is defined as sending, waiting or switching channels; The channel state-action mapping and abnormal mode labels are input into the Q-learning-based policy learning module to update the state-action values ​​and output the communication transmission timing command and channel selection command. The step of inputting the channel state-action mapping and abnormal mode labels into the Q-learning-based policy learning module for state-action value update is as follows: The state space and action space are initialized to construct the state space, which includes the channel state characteristics of the target ship at the current moment. The channel state characteristics include: signal-to-noise ratio, bandwidth utilization, interference intensity, etc. Construct an action space containing combinations of actions with different communication transmission timing options and channel number options. Input the channel state-action mapping and abnormal mode labels: The channel state-action mapping table output by the preceding module and the corresponding abnormal mode labels are used as the current training samples and input into the Q-learning policy learning module. The channel state-action mapping provides the action selection options under the current state, while the anomaly mode label indicates whether the current communication behavior is abnormal. In the policy learning module, for each state-action pair, the current Q-value is calculated; according to the Bellman equation, the Q-value is iteratively updated, with the current immediate reward in the update formula set according to the anomaly mode label. The maximum Q-value for subsequent states is obtained through a greedy policy or an ε-greedy policy; the iteration is repeated until the Q-value converges or the preset number of training rounds is reached.

[0037] Based on the updated Q-value table, the action pair with the largest Q-value in the current state is selected as the optimal action combination; the two components in the optimal action combination are analyzed: one is the communication transmission timing instruction, and the other is the channel selection instruction.

[0038] The communication transmission timing instruction and channel selection instruction are output to the embedded computing platform, completing the action output process of the policy learning module.

[0039] Obtain the AIS positioning data of the target vessel and the corresponding inspection command text, and extract trajectory features and semantic features; The trajectory features are assigned to the ship nodes and the semantic features are assigned to the speech nodes, and the ship nodes and speech nodes are respectively constructed as the basic node set of the multimodal graph; The AIS positioning data is used to construct a trajectory sequence in chronological order. Trajectory features are extracted using the following methods: a sliding window is used to statistically analyze the rate of change of velocity, rate of change of heading, and acceleration per unit time; Fourier transform is introduced to extract periodic motion pattern features from the trajectory sequence; and spatial behavior indicators such as the density distribution of trajectory points, displacement direction vector, and trajectory curvature are calculated. This ultimately forms the trajectory feature sequence of the target vessel, which serves as the attribute input for the vessel nodes in the graph.

[0040] Based on the verification instruction text, a BERT pre-trained language model is used for context encoding to extract semantic features at the instruction level. The verification instruction text is segmented, embedded, and encoded. Part-of-speech tagging and entity recognition are combined to extract key information such as action verbs, ship identification words, action targets, time and space indicators. A semantic feature sequence with consistent dimensions is output as the attribute information input of the speech node.

[0041] Based on the geographical proximity between ships, the communication event correlation in the inspection instruction text, and the temporal sequence between nodes, a graph structure connection composed of spatial edges, semantic edges, and temporal edges is established between ship nodes and voice nodes. The trajectory features and speech features of each node are fused and encoded with the corresponding channel state data sequence to generate a multimodal input tensor with spatiotemporal correlation.

[0042] In this embodiment, step three specifically includes: The multimodal input tensor is fed into a spatiotemporal graph neural network layer based on an attention mechanism; The spatiotemporal graph neural network layer first aggregates the spatial connection relationships between each node and its neighboring nodes in the multimodal graph through graph convolution operations. Based on the adjacency matrix constructed by spatial edges, semantic edges and temporal edges, it calculates the weighted feature sum between nodes and their neighboring nodes according to the edge weight coefficients, and generates spatial feature representations through mapping using a nonlinear activation function. The spatiotemporal graph neural network layer further performs temporal encoding operations on the feature sequences of each node at multiple time steps. By introducing positional encoding into the node feature sequences, and employing a multi-head self-attention mechanism, the temporal feature representation between any time steps is calculated. Spatial and temporal feature representations are fused and encoded, and attention weight coefficients for corresponding spatial, semantic, and temporal edges are calculated separately. Weighted processing is applied to each type of edge, and a unified spatiotemporal embedding representation is generated.

[0043] In this embodiment, step four specifically includes: The spatiotemporal embedding features are input into the encoder of the improved TranAD model; The spatiotemporal embedding features are compressed using an encoder to generate a latent spatial representation; The latent space representation is input into the decoder of the TranAD model, and a self-supervised reconstruction operation is performed to recover the predicted sequence. The self-supervised reconstruction operation is based on the encoder-decoder structure of the improved TranAD model, and specifically includes the following steps: The spatiotemporal embedding feature tensor obtained after modeling by the spatiotemporal graph neural network layer is used as input and fed into the encoder module of the TranAD model. The encoder consists of a multi-layer feedforward neural network, which sequentially performs dimensionality reduction and compression operations on the input features, and extracts compressed features in the latent representation space through linear transformation and nonlinear activation function processing, outputting the latent space representation tensor.

[0044] The latent space representation tensor is input into the decoder module of the TranAD model. The decoder structure performs stepwise reconstruction of the latent representation, reconstructing a prediction sequence with the same dimension as the original input tensor. During the reconstruction process, the decoder incorporates residual connections and normalization mechanisms to improve the accuracy and stability of feature recovery.

[0045] The reconstructed predicted sequence is compared with the original spatiotemporal embedded features at each time step, and the numerical difference is calculated to form the corresponding reconstruction error sequence. The difference calculation uses the mean absolute error index, which is averaged over all feature dimensions and the time series dimension to obtain the overall reconstruction error value corresponding to the sample.

[0046] The difference between the predicted sequence and the spatiotemporal embedded features is compared to generate the corresponding reconstruction error sequence.

[0047] In this embodiment, the steps of the coordinated dual-countermeasure mechanism specifically include: Construct a temporal adversarial branch and use the improved TranAD model as the generator. , generator The output predicted sequence is input into the time series discriminator; The time-series discriminator comprises, in sequence, an input layer, a bidirectional long short-term memory network layer, an attention mechanism layer, and a fully connected classification layer; The generator is determined by the timing discriminator. The output predicted sequence is temporally discriminated against with the corresponding spatiotemporal embedding features to generate temporal discrimination results; The steps for timing determination are as follows: Generator The output predicted sequence and the corresponding spatiotemporal embedding feature are aligned one-to-one according to the time dimension to form an input pair containing the predicted value and the true embedding value at the same time step; Each input pair undergoes a feature concatenation operation to form a unified adversarial input representation, which is then input to the input layer of the temporal discriminator to complete the initial feature mapping process and generate the adversarial input representation.

[0048] The adversarial input representation is mapped and fed into a bidirectional long short-term memory network layer to extract the forward and backward hidden state sequences of the predicted sequence in the time dimension, and to construct a global dynamic behavior pattern representation of the sequence.

[0049] The hidden state sequence is input into the attention mechanism layer, the attention weight at each time step is calculated, and a weighted fused temporal representation vector is generated.

[0050] The time representation vector is fed into a fully connected classification layer to complete the nonlinear mapping and final binary classification operation, outputting a discriminant label and a temporal discriminant probability score, forming a temporal discriminant result. This temporal discriminant result serves as the basis for calculating the adversarial loss and is used to optimize the generator. Backward gradient propagation and parameter updates are performed with the time series discriminator itself.

[0051] Constructing a spatial adversarial branch, using an attention-based spatiotemporal graph neural network layer as the generator. , generator The spatiotemporal embedding features output are input into the spatial discriminator; The spatial discriminator includes a graph structured input layer, a graph convolutional layer, a graph-level pooling layer, and a fully connected classification layer; The generator is determined by the spatial discriminator. The output spatiotemporal embedding features are spatially discriminated against the corresponding multimodal input tensor to generate spatial discrimination results; The steps for spatial discrimination are as follows: Generator The output spatiotemporal embedding features are aligned at the node level with the corresponding multimodal input tensor to construct an input graph with spatial structure information; the input graph is fed into the graph structure input layer to complete the initial graph feature transformation and map out the node representation vector; The node representation is input into the graph convolutional layer. Based on the adjacency relationships of spatial edges, semantic edges and temporal edges defined in the graph, local feature aggregation operation is performed to capture the spatial association patterns between nodes and their neighboring nodes. The node features output by graph convolution are fed into the graph-level pooling layer, and a global pooling mechanism is used to generate a spatial representation vector of the graph as a whole, thereby enhancing the global discriminative ability of the graph structure. The spatial representation vector is fed into a fully connected classification layer to perform nonlinear mapping and output spatial discriminant labels and spatial discriminant probability scores, forming spatial discrimination results. These spatial discrimination results serve as the adversarial loss source for the spatial adversarial branch and are used for backpropagation to update the generator. Parameters of the spatial discriminator itself The temporal discrimination results and spatial discrimination results are trained in parallel. The parallel training of temporal discrimination results and spatial discrimination results specifically involves: Temporal and spatial discrimination results are obtained separately. Within the same training cycle, corresponding discrimination loss functions are constructed using the outputs of the two discriminators, and backpropagation and parameter updates are performed on the temporal and spatial discriminators respectively. After the discriminator parameters are updated, the generator's update target is constructed based on the adversarial losses of the two discriminators, and the gradient feedback information of the temporal and spatial discriminators is simultaneously applied to the generator. With generator Perform a joint update of the generator parameters; For generator Generator The parameters are updated separately for the discriminator and the temporal adversarial branch, and cross-feedback constraints are applied to the temporal adversarial branch and the spatial adversarial branch during the parameter update process; Synchronous Adjustment Generator Based on Cross-Feedback Constraints and generator This ensures consistency between temporal and spatial features, completing the collaborative dual-adversarial training process.

[0052] The parallel training of temporal discrimination results and spatial discrimination results specifically includes: During adversarial training, temporal adversarial losses and spatial adversarial losses are constructed using the temporal discrimination results output by the temporal discriminator and the spatial discrimination results output by the spatial discriminator, respectively. These temporal and spatial adversarial losses are then used as generators. Generator The optimization objectives for both the temporal discriminator and the spatial discriminator; During the parallel training phase, the generator... Perform backpropagation updates of the parameters based on temporal adversarial loss, making the generator... The output predicted sequence approximates the corresponding spatiotemporal embedding features in the temporal dimension, and at the same time performs parameter updates on the temporal discriminator based on its discrimination results; For generator Perform backpropagation updates of parameters based on spatial adversarial loss, making the generator... The output spatiotemporal embedding features approximate the corresponding multimodal input tensor in terms of spatial structure, and at the same time perform parameter updates on the spatial discriminator based on its discrimination results.

[0053] In this embodiment, the generator Generator The parameters are updated separately for both the discriminator and the temporal adversarial branch, and cross-feedback constraints are applied to the temporal adversarial branch and the spatial adversarial branch during the parameter update process, specifically including: During the parameter update process, the temporal correlation features generated by the bidirectional long short-term memory network layer and the attention mechanism layer in the temporal adversarial branch are transmitted as temporal feedback information to the spatial adversarial branch. The temporal correlation feature is the forward and backward hidden state sequences of the bidirectional long short-term memory network layer and the temporal representation vector generated by the attention mechanism.

[0054] Furthermore, the spatial structure representation generated by graph convolutional layers and graph-level pooling layers in the spatial adversarial branch is transmitted as spatial feedback information to the temporal adversarial branch; The spatial structure representation refers to the graph-level spatial embedding vector generated by the graph-level pooling layer and the node spatial feature map output by the graph convolutional layer.

[0055] Under cross-feedback constraints, temporal feedback information and spatial feedback information are introduced into the generator as additional adjustment terms. With generator In the backpropagation update of the parameters, the two generators adjust the generation direction of the predicted sequence and spatiotemporal embedded features based on the temporal feedback information and spatial feedback information during the parallel training phase.

[0056] Spatial feedback information is introduced into the generator as an additional adjustment term. In the backpropagation update of parameters, in the update generator During the process, the spatial structure representation output by the graph convolutional layer and graph-level pooling layer in the spatial adversarial branch is used as a reference benchmark for the generator. The output predicted sequence is subjected to structural embedding mapping; the feature differences between the structural embedding and the spatial structural representation are compared to construct a spatial consistency loss term; the spatial consistency loss term is then superimposed on the generator. In the gradient calculation results, constraints are applied through the backpropagation process to guide the generator. The parameter update direction converges towards the distribution characteristics consistent with the spatial structure representation, enhancing the consistency of the predicted sequence at the spatial structure level.

[0057] Timing feedback information is introduced into the generator as an additional adjustment term. In the backpropagation update of parameters, in the update generator During the process, the temporal correlation features output by the bidirectional long short-term memory network layer and the attention mechanism layer in the temporal adversarial branch are used as a reference benchmark to optimize the generator. The output spatiotemporal embedding features are subjected to time series embedding mapping; the sequence differences between this time series embedding and the time correlation features are compared to construct a time consistency loss term; the time consistency loss term is then superimposed on the generator. In the backpropagation gradient path, the parameter updates are optimized in a direction consistent with the time-related features, thereby improving the expressive power of spatiotemporal embedding features in time series modeling.

[0058] Complete the generator Generator The synchronous training and parameter convergence of the discriminator form a collaborative adversarial optimization process with cross-feedback constraints.

[0059] In this embodiment, step six specifically includes: The reconstructed error sequence, the temporal discrimination probability score output by the temporal discriminator, and the spatial discrimination probability score output by the spatial discriminator are weighted and fused to generate a unified anomaly scoring result. The abnormal score is compared with a preset threshold. If the abnormal score exceeds the preset threshold, it is determined that the current ship communication behavior is abnormal, and the abnormal category is determined according to the discrimination label corresponding to the abnormal score. Based on the anomaly level classification standard, the anomaly score is mapped to the shore-to-ship communication early warning level; The generated shore-to-ship communication early warning level and anomaly category label are combined as early warning information output, and the communication transmission timing instruction and channel selection instruction generated by Q-learning are output in conjunction.

[0060] In this embodiment, step seven specifically includes: Write the abnormal samples corresponding to the abnormal scoring results and the operation log of the shore-to-ship communication early warning system to the background data storage log. The improved TranAD model is retrained using an incremental dataset constructed from anomalous samples and runtime logs. After retraining, dynamic pruning is performed on the spatiotemporal graph neural network layer based on the attention mechanism embedded in the updated improved TranAD model. The low-importance neural network parameter nodes in the spatiotemporal graph neural network structure are pruned according to the node importance score index to generate a simplified improved TranAD model structure. The dynamic pruning process performed on the attention-based spatiotemporal graph neural network layer embedded in the updated improved TranAD model specifically includes: For each neural network parameter node in the spatiotemporal graph neural network layer, a node importance score index is calculated based on the model inference data recorded during its training. The node importance score index includes: weight magnitude, activation intensity, gradient contribution value, and attention score, which are used to evaluate the degree of influence of each node on the final output result during the model inference process. Based on the comprehensive score of each node, a preset importance threshold is set, and neural network parameter nodes with importance scores lower than the threshold are marked as low importance nodes; Delete all neural network parameter nodes marked as low importance and their corresponding connection weights, retain high importance nodes and connection structures, and generate a simplified spatiotemporal graph neural network structure; The simplified spatiotemporal graph neural network structure, together with other retained structures, forms a pruned improved TranAD model, which is used for subsequent model quantization and edge deployment.

[0061] The weight parameters of the pruned improved TranAD model were subjected to INT4 quantization. After pruning and quantization, the parameters of the improved TranAD model are sent to the edge computing nodes deployed at the ship's end. An improved TranAD model is loaded and deployed on an embedded computing platform. Combining the latest multimodal input tensor and channel state data, it outputs real-time warning information and corresponding communication transmission timing instructions and channel selection instructions.

[0062] The embedded computing platform refers to hardware devices or embedded system modules deployed on the ship (i.e., the target ship's local area) to execute a lightweight inference model and realize localized, low-latency processing of shore-to-ship communication early warning functions.

[0063] Example 1: To verify the feasibility and effectiveness of this invention in practical communication early warning scenarios, it was applied to a task involving monitoring the communication behavior of multiple target vessels deployed on a shipping management platform. The system was deployed on a shore-based server equipped with a VHF communication module and a ship-based platform with embedded edge computing nodes, undergoing a 20-day experimental monitoring period. During this period, shore-based voice dispatch information was communicated with the vessels via VHF radio for verification. The system automatically parsed dispatch instructions, identified keywords to trigger the verification process, and, combined with target vessel trajectories, historical interaction records, and channel status information, automatically generated communication instructions and determined whether any abnormalities existed in the communication behavior.

[0064] During the experiment, the shore station issued 582 verification voice commands. The speech recognition module achieved an average recognition accuracy of 97.3%, with Chinese keyword recognition accuracy reaching 98.5% and mixed language sentence recognition accuracy remaining above 95%. The automatic generation of verification commands took an average of 2.36 seconds, and the TTS module built using FastSpeech and HiFi-GAN had an average speech packet generation latency of 320ms, meeting the requirements for real-time communication.

[0065] The communication strategy learning module continuously collects channel state data for each ship, including signal-to-noise ratio, interference rate, and bandwidth utilization. During the experiment, the system processed over 32,000 channel state data entries. The channel selection accuracy based on the Q-learning strategy model reached 93.1%, reducing the interference rate by approximately 27.5% compared to manual selection, and shortening the average communication latency by 3.8 seconds, effectively improving communication throughput.

[0066] A multimodal graph neural network was used to construct a graph structure by fusing ship trajectory and verification semantic features. The lightweight INT4 model deployed on the embedded platform successfully completed dynamic pruning, achieving a node number compression rate of 41.3%. The anomaly score generated by the model inference identified 78 potential communication anomalies in various ship behavior sequences, including 32 high-risk labeled events. The system's early warning response latency was controlled within 1.42 seconds, with no missed or false alarms.

[0067] During the overall system operation, a total of 112 early warning outputs were triggered, jointly generating communication transmission timing instructions and channel selection instructions, and synchronizing them in real time at the ship's edge nodes for decision support. According to manual verification and comparison, 108 of the early warnings highly matched the actual anomalies, achieving an accuracy rate of 96.4%, demonstrating the excellent early warning performance and practical deployment value of this invention in complex maritime communication environments. Specific experimental data are shown in Table 1: Table 1. Statistical Table of Measured Data for Shore-to-Ship Communication and Early Warning System

[0068] As shown in Table 1, the shore-to-ship communication early warning system proposed in this invention exhibits high stability and anomaly detection capabilities in the actual operation of multiple vessels. Among them, the device on vessel number V-K654 performed best in terms of inspection frequency, channel sample size, and anomaly score, with an average of 15 inspections per day, 1855 channel status samples, an anomaly score as high as 0.97, and inference latency controlled within 1.25 seconds, indicating that the model can maintain high detection and response efficiency even in high-frequency communication scenarios.

[0069] Most vessels that triggered warnings had anomaly scores above 0.88. In contrast, vessels that did not trigger warnings (such as V-B312, V-D843, V-G777, and V-J998) generally had anomaly scores below 0.80, indicating that the system's scoring mechanism has a clear ability to distinguish between normal and abnormal communication behavior. Furthermore, the communication command latency for all vessels remained within the range of 1100–1500 ms, and the inference latency was stable between 1.2 and 1.5 seconds, fully demonstrating the real-time processing capabilities of the lightweight model and meeting the timeliness requirements of warnings in maritime communication scenarios.

[0070] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A shore-to-ship communication early warning system based on multi-mode communication technology, characterized in that, include: The voice recognition and command generation module is used to receive voice content from shore stations and VHF radio stations and send inspection command text to the target vessel. The communication strategy learning module is used to collect channel status data of target ships, execute association rule algorithms to generate anomaly tags, and combine Q-learning algorithms to generate communication transmission timing and channel selection instructions. The multimodal graph construction module is used to extract semantic features from the verification instruction text, construct a multimodal graph, and generate a multimodal input tensor. The spatiotemporal graph neural network processing module is used to input multimodal input tensors into the spatiotemporal graph neural network layer with attention mechanism to generate spatiotemporal embedding features; The improved TranAD prediction module is used to input spatiotemporal embedding features into the encoder and decoder of the improved TranAD model to generate a reconstruction error sequence; The collaborative adversarial training module is used to construct temporal adversarial branches and spatial adversarial branches. The improved TranAD model and the spatiotemporal graph neural network layer are used as generators, respectively, and adversarial training is carried out in combination with the corresponding discriminator. The cross-feedback constraint module is used to apply temporal correlation features and spatial structure representation as feedback information to the backpropagation of the generator. The anomaly scoring and early warning output module is used to generate an anomaly score by weighted fusion of the reconstruction error sequence, temporal discrimination probability score and spatial discrimination probability score, and to output communication transmission timing and channel selection instructions in conjunction with the score. The model update and deployment module is used to build incremental datasets and distribute the pruned and INT4 quantized improved TranAD model to the ship's edge nodes to output early warning information in real time.

2. The shore-to-ship communication early warning system based on multi-mode communication technology according to claim 1, characterized in that, The modules are connected in the following way: Step 1: Receive voice content from shore station and VHF radio, perform voice recognition and keyword extraction, trigger the inspection process based on the recognition results, and transmit the inspection command text to the target vessel via VHF radio. Step 2: Collect channel state data, solve the strategy using the Q-learning algorithm, and output communication transmission timing instructions and channel selection instructions; combine AIS positioning data and verification instruction text to construct a multimodal graph and construct a spatiotemporal input tensor; Step 3: Input the multimodal input tensor into the spatiotemporal graph neural network layer based on the attention mechanism to model spatial and temporal dependencies and output spatiotemporal embedding features; A spatiotemporal graph neural network layer based on an attention mechanism is inserted between the encoder and decoder of the improved TranAD model; Step 4: Input the spatiotemporal embedding features into the improved TranAD encoder and decoder to perform self-supervised reconstruction, generate a reconstruction sequence, and record the reconstruction error sequence; Step 5: Establish a collaborative dual adversarial mechanism, in which the temporal adversarial branch is composed of TranAD as the generator and a temporal discriminator, and the spatial adversarial branch is composed of a spatiotemporal graph neural network layer containing an attention mechanism as the generator and a spatial discriminator. Step 6: Generate an anomaly score based on the reconstruction error sequence, temporal discrimination probability score, and spatial discrimination probability score; generate early warning information by combining it with a preset threshold; and output the early warning information, communication transmission timing instruction, and channel selection instruction. Step 7: Send the pruned and quantized improved TranAD parameters to the ship's edge node, and output real-time early warning information and corresponding communication transmission timing and channel selection commands.

3. A shore-to-ship communication early warning system based on multi-mode communication technology according to claim 2, characterized in that, Step one specifically includes: The shore-to-ship communication early warning system receives voice content input from shore stations and VHF radio stations, and uses a voice recognition algorithm to convert the voice content into text content with timestamps. Based on a preset keyword rule base, the converted text content is identified for keywords, and the keywords are extracted and compared with the key phrases for triggering verification defined in the preset keyword rule base; When the recognition result contains a key phrase that triggers the verification, the triggering condition is met, and the verification process and listening function are automatically started. The verification process specifically includes: The target vessel is selected through the shore-to-ship communication early warning system and its AIS positioning data is integrated for target identification. The system receives verification content, including verification text input and recorded voice input. It performs a text preprocessing algorithm on the verification text input to generate a structured text draft, and performs segmentation and buffering processing on the voice input to form a voice segment sequence. Variational mode decomposition and wavelet packet entropy filtering are performed sequentially on the speech slice sequence to obtain the purified speech sequence; Extract the Mel spectrogram and Mel cepstral coefficient features from the cleaned speech sequence to generate a speech feature sequence; The speech feature sequence is aligned with phoneme-level Chinese-English mixed speech using a dynamic time warping algorithm to generate a draft of the instruction text corresponding to the time axis. Based on the keywords of the initial draft of the instruction text and the initial draft of the structured text, the template matching algorithm calls the preset shouting template to generate the verification instruction text; A text-to-speech algorithm is used to convert the inspection command text into natural speech, generate a voice packet for transmission, and drive a VHF radio to send the voice packet as an inspection command to the target ship.

4. A shore-to-ship communication early warning system based on multi-mode communication technology according to claim 2, characterized in that, Step two specifically includes: The shore-to-ship communication early warning system uses keyword recognition to trigger the listening function, drives the VHF radio to receive voice packets, parses the voice packets in real time to obtain the real-time listening content, and aggregates it with the historical inspection command text. An association rule mining algorithm is used to analyze historical verification command texts and real-time monitoring content to generate anomaly tags; the association rule mining algorithm is the Apriori algorithm. Collect and update the channel state data of the target vessel to form a channel state data sequence; Construct a channel state-action mapping based on channel state data; The channel state-action mapping and anomaly label are input into the Q-learning-based policy learning module to update the state-action values ​​and output the communication transmission timing command and channel selection command. Obtain the AIS positioning data of the target vessel and the corresponding inspection command text, and extract trajectory features and semantic features; The trajectory features are assigned to the ship nodes and the semantic features are assigned to the speech nodes, and the ship nodes and speech nodes are respectively constructed as the basic node set of the multimodal graph; Based on the geographical proximity between ships, the communication event correlation in the inspection instruction text, and the temporal sequence between nodes, a graph structure connection composed of spatial edges, semantic edges, and temporal edges is established between ship nodes and voice nodes. The trajectory features, semantic features, edge relationships, and channel state data sequences associated with nodes in the constructed multimodal graph are uniformly encoded to generate a multimodal input tensor.

5. A shore-to-ship communication early warning system based on multi-mode communication technology according to claim 2, characterized in that, Step three specifically includes: The multimodal input tensor is fed into a spatiotemporal graph neural network layer based on an attention mechanism; The spatiotemporal graph neural network layer aggregates features of the spatial connection relationships between each node and its neighboring nodes in the multimodal graph through graph convolution operations. Based on the adjacency matrix constructed by spatial edges, semantic edges and temporal edges, it calculates the weighted feature sum of the node and its neighboring nodes according to the edge weight coefficients, and generates spatial feature representations through mapping by nonlinear activation functions. The spatiotemporal graph neural network layer further performs temporal encoding operations on the feature sequences of each node at multiple time steps. By introducing positional encoding into the node feature sequences, and employing a multi-head self-attention mechanism, the temporal feature representation between any time steps is calculated. Spatial and temporal feature representations are fused and encoded to generate unified spatiotemporal embedding features.

6. A shore-to-ship communication early warning system based on multi-mode communication technology according to claim 2, characterized in that, Step four specifically includes: The spatiotemporal embedding features are input into the encoder of the improved TranAD model; The spatiotemporal embedding features are compressed using an encoder to generate a latent spatial representation; The latent space representation is input into the decoder of the TranAD model, and a self-supervised reconstruction operation is performed to recover the predicted sequence. The difference between the predicted sequence and the spatiotemporal embedded features is compared to generate the corresponding reconstruction error sequence.

7. A shore-to-ship communication early warning system based on multi-mode communication technology according to claim 2, characterized in that, The steps of the coordinated dual-counter mechanism specifically include: Construct a temporal adversarial branch and use the improved TranAD model as the generator. , generator The output predicted sequence is input into the time series discriminator; The time-series discriminator comprises, in sequence, an input layer, a bidirectional long short-term memory network layer, an attention mechanism layer, and a fully connected classification layer; The generator is determined by the timing discriminator. The output predicted sequence is temporally discriminated against with the corresponding spatiotemporal embedding features to generate temporal discrimination results; Constructing a spatial adversarial branch, using an attention-based spatiotemporal graph neural network layer as the generator. , generator The spatiotemporal embedding features output are input into the spatial discriminator; The spatial discriminator includes a graph structured input layer, a graph convolutional layer, a graph-level pooling layer, and a fully connected classification layer; The generator is determined by the spatial discriminator. The output spatiotemporal embedding features are compared with the corresponding multimodal input tensor to generate spatial discrimination results; The temporal and spatial discrimination results are trained in parallel to improve the generator. Generator The parameters are updated separately for the discriminator and the temporal adversarial branch, and cross-feedback constraints are applied to the temporal adversarial branch and the spatial adversarial branch during the parameter update process; Synchronous Adjustment Generator Based on Cross-Feedback Constraints and generator This ensures consistency between temporal and spatial features, completing the collaborative dual-adversarial training process.

8. A shore-to-ship communication early warning system based on multi-mode communication technology according to claim 7, characterized in that, The generator Generator The parameters are updated separately for both the discriminator and the temporal adversarial branch, and cross-feedback constraints are applied to the temporal adversarial branch and the spatial adversarial branch during the parameter update process, specifically including: During the parameter update process, the temporal correlation features generated by the bidirectional long short-term memory network layer and the attention mechanism layer in the temporal adversarial branch are transmitted as temporal feedback information to the spatial adversarial branch. The temporal correlation feature is the forward and backward hidden state sequence of the bidirectional long short-term memory network layer and the temporal representation vector generated by the attention mechanism; Furthermore, the spatial structure representation generated by graph convolutional layers and graph-level pooling layers in the spatial adversarial branch is transmitted as spatial feedback information to the temporal adversarial branch; The spatial structure representation refers to the graph-level spatial embedding vector generated by the graph-level pooling layer and the node spatial feature map output by the graph convolutional layer. Under cross-feedback constraints, temporal feedback information and spatial feedback information are introduced into the generator as additional adjustment terms. With generator In the backpropagation update of the parameters, the two generators adjust the generation direction of the predicted sequence and spatiotemporal embedded features based on the temporal feedback information and spatial feedback information during the parallel training phase.

9. A shore-to-ship communication early warning system based on multi-mode communication technology according to claim 2, characterized in that, Step six specifically includes: The reconstructed error sequence, the temporal discrimination probability score output by the temporal discriminator, and the spatial discrimination probability score output by the spatial discriminator are weighted and fused to generate a unified anomaly scoring result. The abnormal score is compared with a preset threshold. If the abnormal score exceeds the preset threshold, it is determined that the current ship communication behavior is abnormal, and the abnormal category is determined according to the discrimination label corresponding to the abnormal score. Based on the anomaly level classification standard, the anomaly score is mapped to the shore-to-ship communication early warning level; The generated shore-to-ship communication early warning level and anomaly category label are combined as early warning information output, and the communication transmission timing instruction and channel selection instruction generated by Q-learning are output in conjunction.

10. A shore-to-ship communication early warning system based on multi-mode communication technology according to claim 2, characterized in that, Step seven specifically includes: Write the abnormal samples corresponding to the abnormal scoring results and the operation log of the shore-to-ship communication early warning system to the background data storage log. The improved TranAD model is retrained using an incremental dataset constructed from anomalous samples and runtime logs. After retraining, dynamic pruning is performed on the spatiotemporal graph neural network layer based on the attention mechanism embedded in the updated improved TranAD model. The low-importance neural network parameter nodes in the spatiotemporal graph neural network structure are pruned according to the node importance score index to generate a simplified improved TranAD model structure. The weight parameters of the pruned improved TranAD model were subjected to INT4 quantization. After pruning and quantization, the parameters of the improved TranAD model are sent to the edge computing nodes deployed at the ship's end. An improved TranAD model is loaded and deployed on an embedded computing platform. Combining the latest multimodal input tensor and channel state data, it outputs real-time warning information and corresponding communication transmission timing instructions and channel selection instructions.

Images