Ship response system based on AI large model data fusion

Abstract

The invention discloses a ship response system based on AI large model data fusion, and belongs to the technical field of ships, and the system comprises a dual-mode positioning module which is used for carrying out the centimeter-level positioning of a ship in a target sea area according to a GNSS system and an INS system, and obtaining the centimeter-level positioning ship position data; the data synchronization module is used for acquiring and synchronizing ship multi-source data according to the centimeter-level positioned ship position data to obtain ship situation data; and the edge-cloud collaboration module is used for monitoring the ship in real time according to the ship situation data and an edge-shore-based collaboration mechanism and dynamically adjusting the navigational speed and the fuel supply plan to stop the ship according to the sea condition. The problems that in the prior art, comprehensive and accurate comprehensive information is difficult to form, hidden dangers cannot be found in time, and accidents cannot be prevented in advance are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of ships, and particularly to a ship response system based on AI large model data fusion. Background Art

[0002] In recent years, in the shipping field, the existing ship monitoring technologies have made remarkable progress and a relatively complete monitoring system has been established. This system comprehensively applies a variety of advanced technologies, among which satellite communication technology plays a key role. Through the globally covered satellite network, the position information of ships can be obtained in real time. Whether the ship is sailing in the vast ocean or complex inland river channels, its dynamics can be accurately captured. The Geographic Information System (GIS) combines the ship position information with electronic nautical charts to visually present the ship's navigation track, location and surrounding geographical environment in a visual way, providing a clear ship operation situation map for monitoring personnel. The Automatic Identification System (AIS) is also an important part of the existing monitoring technologies. The AIS equipment installed on ships can automatically and frequently send relevant information of the ship itself, such as ship type, size, heading, speed, etc. Surrounding ships and shore-based monitoring stations can receive this information, realizing information interaction between ships and real-time tracking of ships by the shore-based. In addition, sensor technology has also been widely used in ship monitoring. Various sensors installed in key parts of the ship, such as temperature sensors, pressure sensors, liquid level sensors, etc., can monitor parameters such as the operating state of ship equipment and cargo storage conditions in real time. Once an abnormal situation is found, an alarm can be issued in time to ensure the safe navigation of the ship.

[0003] Although the existing ship monitoring technologies are relatively mature, there are still some problems to be solved urgently. On the one hand, the data fusion and processing capabilities are insufficient. At present, the data formats and standards from different monitoring devices and systems are different, resulting in difficulties in data fusion and it is difficult to form comprehensive and accurate ship comprehensive information. At the same time, in the face of a large amount of monitoring data, the existing data processing algorithms and platforms are inefficient and cannot analyze and mine the potential information behind the data in time, so that some safety hazards and abnormal situations cannot be discovered in time. On the other hand, there are blind spots in monitoring coverage. Although technologies such as satellite communication and AIS have a wide coverage range, in some special areas, such as polar regions, near remote islands and complex urban canyon waters, the signals may be interfered or blocked, resulting in the loss or inaccuracy of ship monitoring data. In addition, for some small ships and non-cooperative ships, because they may not install or turn on the specified monitoring equipment, these ships are outside the monitoring blind spots, increasing the difficulty of maritime safety management. Moreover, the existing monitoring systems have limited intelligence, mostly only capable of realizing simple data monitoring and alarm functions, lacking intelligent prediction and decision support for the ship navigation situation, and it is difficult to take effective preventive measures in advance to avoid accidents. Summary of the Invention

[0004] The technical problem solved by the present invention is that in the prior art, it is difficult to form comprehensive, accurate and integrated information, and it is impossible to detect potential hazards in time and prevent accidents in advance.

[0005] To solve the above technical problem, the present invention provides the following technical solutions:

[0006] The ship response system based on AI large model data fusion provided by the present invention includes:

[0007] A dual-mode positioning module, which is used to perform centimeter-level positioning on ships in the target sea area according to the GNSS system and the INS system, and obtain the ship position data at centimeter level;

[0008] A data synchronization module, which is used to obtain and synchronize multi-source ship data according to the ship position data at centimeter level, and obtain ship situation data;

[0009] An edge-cloud collaboration module, which is used to monitor ships in real time according to the ship situation data and the edge-shore collaboration mechanism, and dynamically adjust the ship speed and fuel supply plan to stop sailing according to the sea conditions.

[0010] Further, performing centimeter-level positioning on ships in the target sea area according to the GNSS system and the INS system, and obtaining the ship position data at centimeter level, includes:

[0011] In the target sea area, divide the target sea area into several grid regions according to the ship traffic, terrain features and signal occlusion conditions;

[0012] Set up reference stations at the center of each grid region, add redundant reference stations in areas where signals are vulnerable to interference, and set the coverage radius of the reference stations and redundant reference stations;

[0013] Establish a dynamic adjustment mechanism to adjust the positions of the reference stations and redundant reference stations according to ship traffic and signal changes;

[0014] Install a GNSS receiver and an INS system on the ship as a mobile station to receive satellite signals and measure the motion state of the ship;

[0015] Use satellite communication or wireless communication to establish a real-time communication link between the reference station and the mobile station, perform time synchronization processing on the received data, and align the data at different times to the same time reference;

[0016] Fuse the data of the GNSS system and the INS system through an extended Kalman filter algorithm to calculate the position, heading and speed of the ship;

[0017] Based on the evaluation indicators of position error, heading error, and speed error, the position, heading, and speed of the ship are accurately evaluated to obtain the accuracy evaluation result;

[0018] According to the accuracy evaluation result, the parameters of the extended Kalman filter algorithm are adjusted and optimized to make the calculated ship position reach centimeter-level accuracy, and the ship position data with centimeter-level positioning is obtained.

[0019] Furthermore, the multi-source data of the ship includes radar data, sonar data, video data, and AIS data.

[0020] Furthermore, the multi-source data of the ship is synchronized according to the ship position data with centimeter-level positioning to obtain the ship situation data, including:

[0021] The time of radar data, sonar data, video data, and AIS data is uniformly calibrated by using the time synchronization protocol PTP;

[0022] The ship position data with centimeter-level positioning is spatially associated with radar data, sonar data, video data, and AIS data, and the ship situation data is fused and generated.

[0023] Furthermore, the spatial association of radar data, sonar data, video data, and AIS data with the ship position data with centimeter-level positioning includes:

[0024] The affine transformation algorithm is used to convert radar data and sonar data into a relative coordinate system centered on the ship position with centimeter-level positioning;

[0025] The multi-target tracking algorithm based on Kalman filter is used to map the ship position in the video data to the ship position with centimeter-level positioning;

[0026] The linear interpolation algorithm is used to align the ship position information in the AIS data with the centimeter-level positioning data in time, and the nearest neighbor matching algorithm is used to match the longitude and latitude coordinates in the AIS data with the centimeter-level positioning data.

[0027] Furthermore, the edge-cloud collaboration module includes an edge computing layer, a shore-based decision-making layer, and a collaborative training layer;

[0028] The edge computing layer includes edge computing units deployed on each ship, which are used to dynamically adjust the compression rate of the ship situation data according to the sea conditions by using a lightweight large model, and transmit the adjusted ship situation data to the shore-based decision-making layer by using the blind area storage technology;

[0029] The shore-based decision-making layer includes shore-based nodes, which are used to predict the obstacle trajectories within a preset circumferential distance of the ship in a preset future time period according to the received adjusted ship situation data and a pre-trained dynamic obstacle trajectory prediction model, and dynamically adjust the ship's speed and fuel supply to decide whether to stop sailing;

[0030] The collaborative training layer is used to perform edge training on each lightweight large model using the edge computing units on each ship, aggregate each lightweight large model using the shore-based nodes to generate a shore-based large model, optimize the shore-based large model by updating the ship situation data, and transfer the knowledge of the shore-based large model to the lightweight large models.

[0031] Further, using the lightweight large model to dynamically adjust the compression rate of the ship situation data according to the sea conditions, and transmitting the adjusted ship situation data to the shore-based decision-making layer using the blind area storage technology, including:

[0032] Install a wave height meter with an accuracy of ±0.1m, an ultrasonic anemometer with a measurement range of 0-60m / s, and a six-axis inertial measurement unit on the ship;

[0033] Collect sea condition data including wave height and wind speed in real time through the edge computing units on each ship and convert it into time series sea condition data;

[0034] Use an LSTM network to extract features from the time series sea condition data to generate a feature vector including wave height and wind speed;

[0035] Integrate the ship type and operation status labels into the feature vector through a fully connected layer to generate a feature representation;

[0036] Based on the feature representation, make a prediction using a pre-constructed lightweight large model and output a compression rate parameter;

[0037] Adjust the compression rate of the ship situation data according to the compression rate parameter, and transmit the adjusted ship situation data to the shore-based decision-making layer using the blind area storage technology;

[0038] Among them, the lightweight large model is constructed based on the LightGBM gradient boosting decision tree model.

[0039] Further, the obstacle trajectory prediction model includes:

[0040] A double-layer LSTM structure, including a forward LSTM network and a backward LSTM network, with 128 units in each layer of the LSTM network, which is used to process the trajectory sequence of the ship situation data and generate LSTM time series features;

[0041] The graph neural network, including nodes and edges, is used to generate an obstacle relationship feature matrix by aggregating 3-hop neighbor information using the GraphSAGE algorithm based on node features, edge features, and a message passing mechanism, and generate GNN relationship features. Among them, the node features include the motion state and type of the obstacle; the edge features include the relative distance, speed difference, and heading angle between obstacles.

[0042] The attention mechanism module, including 4 multi-head self-attention layers, is used to fuse LSTM temporal features and GNN relationship features using a Transformer decoder, generate attention weights through learnable parameters, and generate a fused feature representation.

[0043] The prediction output layer includes a trajectory prediction branch and a risk assessment branch. The trajectory prediction branch is used to output the position probability distribution of the ship at a preset circumferential distance within a preset future time period using a fully connected layer. The risk assessment branch is used to output the collision probability through a sigmoid activation function based on the position probability distribution of the ship at a preset circumferential distance within a preset future time period.

[0044] Furthermore, adopting a curriculum learning paradigm, first pre-training a two-layer LSTM structure, and then gradually adding a graph neural network and an attention mechanism module, the training method for training the obstacle trajectory prediction model includes:

[0045] Obtain the historical trajectories of obstacles, ocean current influence coefficients, obstacle types, interactions between obstacles, and heading change rates within the target sea area.

[0046] Only activate the two-layer LSTM network, use the training set containing the historical trajectories of obstacles and ocean current influence coefficients to process the trajectory sequence of obstacle situation data, and generate LSTM temporal features.

[0047] Then activate the graph neural network, use the training set containing obstacle types and interactions between obstacles, and adopt the GraphSAGE algorithm to aggregate 3-hop neighbor information to generate an obstacle relationship feature matrix and generate GNN relationship features.

[0048] Then activate the attention mechanism module, use the training set containing the heading change rate, use a Transformer decoder to fuse LSTM temporal features and GNN relationship features, generate attention weights through learnable parameters, and generate a fused feature representation.

[0049] Then activate the prediction output layer to output the position probability distribution and collision probability of the ship at a preset circumferential distance within a preset future time period.

[0050] Among them, the learning rate is decayed in stages, the order of message passing in the graph neural network is gradually increased from 1 to 3, the number of attention heads in the attention mechanism module is gradually increased from 4 to 8, and backpropagation training is performed according to a loss function that includes trajectory prediction error, collision probability prediction error, and model complexity penalty, to obtain a trained obstacle trajectory prediction model.

[0051] Furthermore, the loss function is expressed as:

[0052] ;

[0053] In the formula, represents the loss function, , and respectively represent the trajectory prediction error , the collision probability prediction error and the model complexity penalty weight coefficients, represents the total number of training samples, represents the true value of the trajectory prediction error of the th training sample, represents the predicted value of the trajectory prediction error of the th training sample, represents the true value of the collision probability prediction error of the th training sample, represents the predicted value of the collision probability prediction error of the th training sample, represents the logarithmic function, represents the regularization coefficient, represents the number of model parameters, represents the th model parameter.

[0054] Compared with the prior art, the beneficial effects of the present invention:

[0055] 1. The present invention realizes centimeter-level positioning through a dual-mode positioning module to obtain high-precision ship position data. Based on this, the data synchronization module synchronizes multi-source ship data to obtain ship situation data. Furthermore, the edge-cloud collaboration module monitors the ship in real time according to the ship situation data and the edge-shore-based collaboration mechanism, and dynamically adjusts the ship speed and fuel supply plan to stop sailing. This method of high-precision positioning and multi-source data fusion enables ship monitoring to obtain more comprehensive and accurate information. The adjustment of the ship speed and fuel supply plan based on this information is more scientific and reasonable, which can effectively improve the economy and safety of ship navigation, reduce unnecessary fuel consumption and safety risks caused by unreasonable navigation plans. It solves the problems in the prior art that it is difficult to form comprehensive, accurate and comprehensive information, and it is impossible to detect hidden dangers in time and prevent accidents in advance.

[0056] 2. The present invention performs centimeter-level positioning on ships in the target sea area according to the GNSS system and the INS system. Through a series of fine operations such as dividing grid areas, setting up reference stations and redundant reference stations, and establishing a dynamic adjustment mechanism, combined with the extended Kalman filter algorithm to fuse data and optimize parameters, finally, ship position data with centimeter-level accuracy is obtained. This high-precision positioning can accurately grasp the real-time position of the ship, providing a solid foundation for subsequent multi-source data fusion and situation analysis. At the same time, the dynamic adjustment mechanism can adjust the positions of the reference station and the redundant reference station according to the ship traffic and signal changes, ensuring high-precision positioning in different situations, greatly improving the accuracy and reliability of ship monitoring, and reducing monitoring errors and safety hazards caused by positioning errors.

[0057] 3. The present invention integrates various ship multi-source data such as radar data, sonar data, video data and AIS data, and uses the time synchronization protocol PTP for time unified calibration. Then, affine transformation algorithm, multi-target tracking algorithm based on Kalman filter, linear interpolation algorithm and nearest neighbor matching algorithm are used to spatially associate the ship position data with centimeter-level positioning with these multi-source data, and ship situation data is fused and generated. This fusion and spatial association of multi-source data can make full use of the advantages of different data sources, comprehensively and accurately present the surrounding environment and its own state of the ship, provide richer and more intuitive information for monitoring personnel, help to detect potential safety problems and abnormal situations in time, and improve the comprehensiveness and effectiveness of ship monitoring.

[0058] 4. The edge-cloud collaboration module of the present invention includes an edge computing layer, a shore-based decision-making layer, and a collaborative training layer, and each layer works collaboratively. The edge computing layer uses a lightweight large model to dynamically adjust the compression ratio of ship situation data according to sea conditions and adopts blind area storage technology to transmit data, which not only ensures the timely transmission of data but also reduces the occupancy of network bandwidth. The shore-based decision-making layer can accurately predict the obstacle trajectory within a preset circumferential distance of the ship in a preset future time period based on the received data and a pre-trained dynamic obstacle trajectory prediction model, and dynamically adjust the ship's speed and fuel supply, and decide whether to stop sailing, achieving autonomous response and efficient decision-making for the ship. The collaborative training layer continuously optimizes the model through edge training and shore-based aggregation to improve the accuracy of prediction and the scientific nature of decision-making. This mechanism of edge-cloud collaboration and intelligent prediction greatly improves the intelligent level of ship monitoring, can take effective preventive measures in advance, avoid accidents, and ensure the safe navigation of ships. Description of the Drawings

[0059] Figure 1 It is a schematic structural diagram of a ship response system based on AI large model data fusion provided by an embodiment of the present invention;

[0060] Figure 2 It is a schematic structural diagram of an edge-cloud collaboration module provided by an embodiment of the present invention;

[0061] Figure 3 It is a schematic structural diagram of an obstacle trajectory prediction model provided by an embodiment of the present invention;

[0062] Figure 4 It is a schematic basic flow diagram of a training method for an obstacle trajectory prediction model provided by an embodiment of the present invention. Detailed Embodiments

[0063] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following detailed description of the specific embodiments of the present invention will be given in conjunction with the accompanying drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them.

[0064] Embodiment 1

[0065] As Figure 1 shown, an embodiment of the present invention provides a ship response system based on AI large model data fusion, including:

[0066] A dual-mode positioning module for centimeter-level positioning of ships in the target sea area according to the GNSS system and the INS system to obtain ship position data with centimeter-level positioning.

[0067] The present invention performs positioning by combining the Global Navigation Satellite System (GNSS) and the Inertial Navigation System (INS). GNSS determines the position by receiving satellite signals, but in some complex environments such as being blocked by high-rise buildings or having weak satellite signals in the deep ocean, the positioning accuracy and reliability will be affected. INS uses sensors such as accelerometers and gyroscopes to measure the acceleration and angular velocity of the ship, and obtains position, velocity, and attitude information through integral operations, but there is a problem of error accumulation. The present invention complements the advantages of the two through a dual-mode positioning module. GNSS provides high-precision absolute position information, and INS provides short-term high-precision positioning and attitude information when the GNSS signal is lost, thereby achieving centimeter-level positioning of ships in the target sea area.

[0068] The present invention performs fusion processing on the data collected by GNSS and INS, and uses algorithms such as Kalman filtering to fuse the high-precision position information of GNSS with the continuous measurement data of INS, suppress the error accumulation of INS, and improve the accuracy and reliability of positioning, and finally obtain the ship position data with centimeter-level positioning.

[0069] A data synchronization module is used to acquire and synchronize the multi-source data of the ship according to the ship position data with centimeter-level positioning to obtain the ship situation data.

[0070] Based on the ship position data with centimeter-level positioning output by the dual-mode positioning module, the present invention acquires the multi-source data of the ship, and through a data synchronization mechanism, aligns the time and unifies the format of data from different sources and in different formats to ensure the consistency and integrity of the data. The present invention integrates and analyzes the synchronized multi-source data of the ship, extracts key information, and forms the ship situation data.

[0071] An edge-cloud collaboration module is used to monitor the ship in real time according to the ship situation data and the edge-shore collaboration mechanism, and dynamically adjust the ship speed and fuel supply plan to stop sailing according to the sea conditions.

[0072] Based on the ship situation data and the edge-shore collaboration mechanism, the present invention realizes real-time monitoring of the ship. Edge computing devices are deployed on the ship to perform real-time processing and analysis of the ship situation data and quickly respond to the local needs of the ship. At the same time, key data is uploaded to the cloud server on the shore, and using big data analysis and artificial intelligence algorithms, the ship situation data is analyzed and mined more deeply to provide global decision-making support for the ship, such as adjusting the ship speed and fuel supply plan in real time according to sea condition information such as wind and wave size, water flow speed, etc. and the ship situation data.

[0073] For example, when the sea conditions are severe, the ship speed is reduced to ensure the safety of the ship, and at the same time, the fuel supply plan is adjusted to reduce fuel consumption; when the sea conditions are good, the ship speed is appropriately increased to improve the transportation efficiency, and through the edge-cloud collaboration mechanism, the autonomous response and optimized operation of the ship are realized.

[0074] Embodiment 2

[0075] As Figure 2 shown, this is another embodiment of the present invention. With the same inventive concept as Embodiment 1, it provides the implementation steps of a ship response system based on AI large model data fusion, including:

[0076] Step 1: Construct a dual-mode positioning module.

[0077] The dual-mode positioning module provided by the present invention is used to perform centimeter-level positioning on ships in the target sea area according to the GNSS system and the INS system, and obtain the ship position data with centimeter-level positioning.

[0078] In this embodiment, the ship multi-source data includes radar data, sonar data, video data, and AIS data.

[0079] In this embodiment, performing centimeter-level positioning on ships in the target sea area according to the GNSS system and the INS system, and obtaining the ship position data with centimeter-level positioning includes:

[0080] Step 1.1: In the target sea area, divide the target sea area into several grid regions according to the ship traffic, terrain features, and signal occlusion conditions.

[0081] The present invention comprehensively considers multiple factors such as ship traffic, terrain features, and signal occlusion conditions, and uses geographic information system (GIS) technology to perform spatial analysis on the target sea area. By analyzing the ship density in different regions and the influence of terrain undulations such as islands and reefs on signal propagation, the sea area is divided into several grid regions. The size and shape of each grid region are dynamically adjusted according to the actual situation to ensure relative consistency of the signal characteristics and ship activities within the region.

[0082] The present invention improves the scientificity and pertinence of positioning planning. Through grid division, corresponding positioning strategies can be formulated according to the characteristics of different regions. For example, increasing the investment in positioning resources in areas with high ship traffic, and taking special positioning measures in areas with complex terrain and vulnerable signal interference, thereby improving the overall positioning efficiency and accuracy.

[0083] Step 1.2: Set up reference stations at the center of each grid region, add redundant reference stations in areas where signals are vulnerable to interference, and set the coverage radius of the reference stations and redundant reference stations.

[0084] In the present invention, a reference station is set at the center of each grid area. The reference station receives precise signals from satellites and serves as a reference point for positioning. Redundant reference stations are added in areas where signals are vulnerable to interference, such as near the coast with high-rise buildings or areas with large sea waves and complex signal reflections on the sea surface, to improve the reliability and fault tolerance of positioning. According to the signal propagation model and actual tests, the coverage radii of the reference stations and redundant reference stations are determined to ensure that ships within the coverage range can receive stable signals.

[0085] The present invention enhances the stability and reliability of the positioning system. The reference station provides precise reference signals for ships, while the redundant reference stations play a role when the main reference station fails or the signal is interfered, ensuring that ships can always obtain reliable positioning information and avoiding positioning interruption or accuracy degradation caused by the failure of a single reference station.

[0086] Step 1.3: Establish a dynamic adjustment mechanism to adjust the positions of the reference stations and redundant reference stations according to ship traffic and signal changes.

[0087] The present invention monitors ship traffic and signal changes in real time. Through a sensor network and data analysis algorithms, information such as the position, speed, and signal strength of ships is obtained. Based on this information, optimization algorithms such as genetic algorithms and particle swarm algorithms are used to dynamically adjust the positions of the reference stations and redundant reference stations to adapt to changes in ship traffic and the signal environment.

[0088] The present invention improves the adaptability and flexibility of the positioning system. With changes in ship traffic and the signal environment, dynamically adjusting the positions of the reference stations and redundant reference stations can ensure that the positioning system is always in the best working state, improving the accuracy and efficiency of positioning.

[0089] Step 1.4: Install a GNSS receiver and an INS system on the ship to receive satellite signals and measure the motion state of the ship as a mobile station.

[0090] In the present invention, a GNSS receiver and an INS system are installed on the ship. The GNSS receiver receives satellite signals to obtain the preliminary position information of the ship; the INS system uses sensors such as accelerometers and gyroscopes to measure the motion state of the ship, such as acceleration and angular velocity. By fusing the data of the two systems, the accuracy and reliability of positioning can be improved.

[0091] The present invention provides real-time position and motion state information for ships. The combined use of the GNSS receiver and the INS system can give full play to the advantages of the two systems. When the GNSS signal is interfered or lost, the INS system can continue to provide short-term high-precision positioning and motion state information to ensure the navigation safety of ships.

[0092] Step 1.5: Establish a real-time communication link between the reference station and the mobile station using satellite communication or wireless communication, perform time synchronization processing on the received data, and align the data at different times to the same time reference.

[0093] The present invention uses satellite communication or wireless communication technology to establish a real-time communication link between the reference station and the mobile station, ensuring that the reference signal of the reference station can be transmitted to the mobile station on the ship in a timely manner. Perform time synchronization processing on the received data, and use technologies such as the Network Time Protocol (NTP) or the Precision Time Protocol (PTP) to align the data at different times to the same time reference to ensure the consistency and accuracy of the data.

[0094] The present invention realizes real-time data interaction between the reference station and the mobile station. The establishment of the real-time communication link provides guarantee for data transmission, and the time synchronization processing ensures the accuracy and reliability of the data, providing a basis for subsequent data fusion processing.

[0095] Step 1.6: Fuse the data of the GNSS system and the INS system through the extended Kalman filter algorithm, and calculate the position, heading and speed of the ship.

[0096] The extended Kalman filter algorithm is a state estimation method for nonlinear systems. By fusing the data of the GNSS system and the INS system, comprehensively considering the measurement errors and noise characteristics of the two systems, the position, heading and speed of the ship are calculated. At each moment, according to the state equation and observation equation of the system, the state of the ship is predicted and updated, continuously improving the positioning accuracy.

[0097] The present invention improves the positioning accuracy and stability. The extended Kalman filter algorithm can effectively suppress the measurement errors and noise of the GNSS and INS systems, make full use of the information of the two systems, and improve the positioning accuracy and reliability.

[0098] Step 1.7: Based on the evaluation indexes of position error, heading error and speed error, evaluate the position, heading and speed of the ship to obtain the accuracy evaluation result.

[0099] The present invention evaluates the position, heading and speed of the ship based on evaluation indexes such as position error, heading error and speed error. By comparing with high-precision reference data, the numerical values of each error index are calculated to evaluate the accuracy of the positioning result. The accuracy evaluation result can intuitively reflect the accuracy level of the positioning system and provide a basis for subsequent parameter adjustment and optimization.

[0100] Step 1.8: Adjust and optimize the parameters of the extended Kalman filter algorithm according to the accuracy evaluation results, so that the calculated ship position reaches centimeter-level accuracy, and obtain the ship position data with centimeter-level positioning.

[0101] According to the accuracy evaluation results, the present invention uses optimization algorithms such as the gradient descent method and the genetic algorithm to adjust and optimize the parameters of the extended Kalman filter algorithm. By continuously adjusting the parameters, the calculated ship position reaches centimeter-level accuracy.

[0102] The present invention improves the accuracy and performance of the positioning system. By adjusting and optimizing the algorithm parameters, the positioning accuracy can be further improved to meet the requirements of the ship intelligent monitoring and autonomous response system for high-precision positioning.

[0103] Step 2: Construct a data synchronization module.

[0104] The data synchronization module provided by the present invention is used to obtain and synchronize the ship multi-source data according to the ship position data with centimeter-level positioning, and obtain the ship situation data.

[0105] In this embodiment, the obtaining and synchronizing the ship multi-source data according to the ship position data with centimeter-level positioning to obtain the ship situation data includes:

[0106] Step 2.1: Use the Precision Time Protocol (PTP) to uniformly calibrate the time of radar data, sonar data, video data, and AIS data.

[0107] Based on the Precision Time Protocol (PTP), through the information exchange between the master and slave clocks, the present invention uses the Best Master Clock Algorithm (BMC) to determine the best master clock in the network. Then, the slave clock calculates the clock deviation and link delay according to the timestamp information sent by the master clock, and adjusts its own clock to achieve time synchronization with the master clock. In the ship monitoring system, devices such as radar, sonar, video, and AIS are distributed at different positions and transmit data through the network. PTP can ensure the consistency of the data collected by these devices in time.

[0108] Step 2.2: Perform spatial association on the ship position data with centimeter-level positioning, radar data, sonar data, video data, and AIS data, and fuse them to generate ship situation data.

[0109] The present invention comprehensively processes the data from different sensors and systems to generate comprehensive and accurate ship operation state information. By performing spatial association on the ship position data with centimeter-level positioning, radar, sonar, video, and AIS data, the advantages of various data sources can be fully utilized to make up for the deficiencies of a single data source.

[0110] Spatially associate radar data, sonar data, video data, and AIS data with the ship position data with centimeter-level positioning, including:

[0111] Step 2.2.1: Use the affine transformation algorithm to convert the radar data and sonar data into a relative coordinate system centered on the ship position with centimeter-level positioning.

[0112] The present invention utilizes affine transformation to map points in one coordinate system to another through linear transformation. In ship monitoring, the present invention converts the radar data and sonar data into a relative coordinate system centered on the ship position with centimeter-level positioning, which can facilitate the unified processing and analysis of data from different sensors. By determining the correspondence between the radar and sonar data and the centimeter-level positioning data, the affine transformation matrix is calculated, and the coordinate points in the radar and sonar data are transformed according to the transformation matrix so that they are in the same coordinate system as the centimeter-level positioning data.

[0113] Step 2.2.2: Use the multi-target tracking algorithm based on Kalman filter to map the ship positions in the video data to the ship positions with centimeter-level positioning.

[0114] The present invention utilizes Kalman filter to effectively track the motion state of the target by predicting and updating the state of the target. In ship monitoring, the present invention maps the ship positions in the video data to the ship positions with centimeter-level positioning. The Kalman filter algorithm can track multiple ship targets, and estimate and predict the motion state of the ships according to the observation information in the video data such as the positions and speeds of the ships.

[0115] Step 2.2.3: Use the linear interpolation algorithm to align the ship position information in the AIS data with the centimeter-level positioning data in terms of time, and use the nearest neighbor matching algorithm to match the longitude and latitude coordinates in the AIS data with the centimeter-level positioning data.

[0116] The present invention utilizes the linear interpolation algorithm to estimate the data values at unknown times through the linear relationship between known data points, realizing the time alignment of the AIS data and the centimeter-level positioning data. The nearest neighbor matching algorithm is to find the closest matching item to the target data in the known dataset, which is used for the coordinate matching of the AIS data and the centimeter-level positioning data, ensuring the temporal consistency of the AIS data and the centimeter-level positioning data, avoiding data mismatch problems caused by time differences. The nearest neighbor matching algorithm can quickly and accurately match the longitude and latitude coordinates in the AIS data with the centimeter-level positioning data, providing an accurate coordinate basis for subsequent data fusion.

[0117] Step 3: Construct an edge-cloud collaboration module.

[0118] The edge-cloud collaboration module provided by the present invention is used to monitor ships in real time according to ship situation data and the edge-shore collaboration mechanism, and dynamically adjust the ship speed and fuel supply plan to stop sailing according to sea conditions.

[0119] As Figure 2 shown, the edge-cloud collaboration module includes an edge computing layer, a shore-based decision-making layer, and a collaborative training layer.

[0120] Step 3.1: Construct the edge computing layer.

[0121] The edge computing layer includes edge computing units deployed on each ship, which are used to dynamically adjust the compression rate of ship situation data according to sea conditions by using a lightweight large model, and transmit the adjusted ship situation data to the shore-based decision-making layer by using the blind area storage technology.

[0122] In this embodiment, dynamically adjusting the compression rate of ship situation data according to sea conditions by using a lightweight large model and transmitting the adjusted ship situation data to the shore-based decision-making layer by using the blind area storage technology includes:

[0123] Step 3.1.1: Install a wave height meter with an accuracy of ±0.1 m, an ultrasonic anemometer with a measurement range of 0-60 m / s, and a six-axis inertial measurement unit on the ship.

[0124] Step 3.1.2: Real-time collect sea condition data including wave height and wind speed through the edge computing units on each ship and convert it into time series sea condition data.

[0125] Step 3.1.3: Use the LSTM network to extract features from the time series sea condition data and generate a feature vector including wave height and wind speed.

[0126] Step 3.1.4: Integrate the ship type and operation status labels into the feature vector through a fully connected layer to generate a feature representation.

[0127] Step 3.1.5: Based on the feature representation, make a prediction based on a pre-constructed lightweight large model and output a compression rate parameter.

[0128] Step 3.1.6: Adjust the compression rate of ship situation data according to the compression rate parameter, and transmit the adjusted ship situation data to the shore-based decision-making layer by using the blind area storage technology.

[0129] Among them, the lightweight large model is constructed based on the LightGBM gradient boosting decision tree model.

[0130] Step 3.2: Construct the shore-based decision-making layer.

[0131] The shore-based decision-making layer includes shore-based nodes, which are used to predict the obstacle trajectories within a preset circumferential distance of the ship in a future preset time period according to the received adjusted ship situation data and a pre-trained dynamic obstacle trajectory prediction model, and dynamically adjust the ship's speed and fuel supply to decide whether to suspend navigation.

[0132] As Figure 3 shown, the obstacle trajectory prediction model includes a double-layer LSTM structure, a graph neural network, an attention mechanism module, and a prediction output layer.

[0133] The double-layer LSTM structure includes a forward LSTM network and a backward LSTM network. Each layer of the LSTM network includes 128 units, which are used to process the trajectory sequence of the ship situation data and generate LSTM time series features.

[0134] The graph neural network includes nodes and edges, which are used to generate an obstacle relationship feature matrix by aggregating 3-hop neighbor information using the GraphSAGE algorithm according to node features, edge features, and a message passing mechanism, and generate GNN relationship features. Among them, the node features include the obstacle motion state and the obstacle type; the edge features include the relative distance, speed difference, and heading angle between obstacles.

[0135] The attention mechanism module includes 4 multi-head self-attention layers, which are used to fuse the LSTM time series features and the GNN relationship features using a Transformer decoder, generate attention weights through learnable parameters, and generate a fused feature representation.

[0136] The prediction output layer includes a trajectory prediction branch and a risk assessment branch. The trajectory prediction branch is used to output the position probability distribution within a preset circumferential distance of the ship in a future preset time period using a fully connected layer. The risk assessment branch is used to output the collision probability through a sigmoid activation function according to the position probability distribution within a preset circumferential distance of the ship in a future preset time period.

[0137] In this embodiment, a curriculum learning paradigm is adopted. First, the double-layer LSTM structure is pre-trained, and then the graph neural network and the attention mechanism module are gradually added. The training method for training the obstacle trajectory prediction model includes:

[0138] Obtain the historical trajectories of obstacles, the ocean current influence coefficient, the obstacle type, the interaction between obstacles, and the heading change rate in the target sea area.

[0139] Only activate the double-layer LSTM network, and use the training set containing the historical trajectories of obstacles and the ocean current influence coefficient to process the trajectory sequence of the obstacle situation data and generate LSTM time series features.

[0140] Reactivate the graph neural network, and use a training set containing obstacle types and interactions between obstacles. Adopt the GraphSAGE algorithm to perform 3-order neighbor information aggregation to generate an obstacle relationship feature matrix, and generate GNN relationship features.

[0141] Reactivate the attention mechanism module, and use a training set containing the rate of change of heading. Use the Transformer decoder to fuse the LSTM time series features and GNN relationship features, and generate attention weights through learnable parameters to generate a fused feature representation.

[0142] Reactivate the prediction output layer, and output the position probability distribution and collision probability of the ship at a preset circumferential distance within a preset future time period.

[0143] Among them, the learning rate is decayed in stages, the message passing order of the graph neural network is gradually increased from 1 to 3, the number of attention heads in the attention mechanism module is gradually increased from 4 to 8, and backpropagation training is performed according to a loss function containing trajectory prediction error, collision probability prediction error, and model complexity penalty to obtain a trained obstacle trajectory prediction model.

[0144] In this embodiment, the loss function is expressed as:

[0145] ;

[0146] In the formula, represents the loss function, , and respectively represent the weight coefficients of the trajectory prediction error , the collision probability prediction error and the model complexity penalty , represents the total number of training samples, represents the true value of the trajectory prediction error of the th training sample, represents the predicted value of the trajectory prediction error of the th training sample, represents the true value of the collision probability prediction error of the th training sample, represents the predicted value of the collision probability prediction error of the th training sample, represents the logarithmic function, represents the regularization coefficient, represents the number of model parameters, represents the th model parameter.

[0147] Step 3.3: Cooperative training layer.

[0148] The collaborative training layer is used to perform edge training on each lightweight large model using the edge computing units on each ship, aggregate each lightweight large model using the shore-based nodes to generate a shore-based large model, optimize the shore-based large model by updating the ship situation data, and transfer the knowledge of the shore-based large model to the lightweight large model.

[0149] Through the edge training of the edge computing units, the model aggregation of the shore-based nodes, the model optimization based on the ship situation data, and the knowledge transfer to the lightweight large model, the collaborative training layer realizes a distributed and collaborative optimization machine learning system, which can make full use of the computing resources of the ships and the shore-based, protect data privacy, improve the adaptability and performance of the model, and provide strong support for the intelligent monitoring and autonomous response of the ships. At the same time, the collaborative training mechanism enables the system to continuously adapt to the changes in the actual environment and achieve continuous performance improvement and optimization.

[0150] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. Among them, the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, abbreviated as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, abbreviated as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, abbreviated as EPROM), programmable read-only memory (Programmable Red-Only Memory, abbreviated as PROM), read-only memory (Read-Only Memory, abbreviated as ROM), magnetic memory, flash memory, magnetic disk or optical disk. These computer program instructions can also be stored in a computer-readable memory capable of guiding a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device realizes the functions specified in one process Figure 1 one process or multiple processes and / or boxes Figure 1 or multiple boxes specified in the boxes.

[0151] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. The ship response system based on AI large model data fusion is characterized by: include: The dual-mode positioning module is used to perform centimeter-level positioning of ships in the target sea area based on the GNSS system and the INS system to obtain centimeter-level positioning ship position data; The data synchronization module is used to obtain and synchronize the multi-source data of the ship according to the centimeter-level positioning ship position data to obtain the ship situation data; The edge-cloud collaboration module is used to monitor ships in real time and dynamically adjust the speed and fuel supply plan according to sea conditions based on ship situation data and edge-shore collaboration mechanisms.

2. The ship response system based on AI large model data fusion according to claim 1, characterized in that: According to the GNSS system and INS system, centimeter-level positioning is performed on the ship in the target sea area to obtain centimeter-level positioning ship position data, including: In the target sea area, the target sea area is divided into several grid areas according to the ship flow, terrain characteristics and signal obstruction; Set up a reference station at the center of each grid area, add redundant reference stations in areas where signals are susceptible to interference, and set the coverage radius of the reference station and redundant reference stations; Establish a dynamic adjustment mechanism to adjust the positions of reference stations and redundant reference stations according to ship traffic and signal changes; Installing GNSS receivers and INS systems on ships as mobile stations to receive satellite signals and measure the motion status of ships; Use satellite communication or wireless communication to establish a real-time communication link between the base station and the mobile station, perform time synchronization on the received data, and align the data at different times to the same time reference; The data from the GNSS system and the INS system are fused and processed through the extended Kalman filter algorithm to calculate the position, heading and speed of the ship; Based on the position error, heading error and speed error evaluation indicators, the accuracy of the ship's position, heading and speed is evaluated to obtain the accuracy evaluation results; According to the accuracy evaluation results, the parameters of the extended Kalman filter algorithm are adjusted and optimized to make the calculated ship position reach centimeter-level accuracy and obtain centimeter-level positioning ship position data.

3. The ship response system based on AI large model data fusion according to claim 1, characterized in that: The multi-source ship data includes radar data, sonar data, video data and AIS data.

4. The ship response system based on AI large model data fusion as claimed in claim 3, characterized in that: The method of acquiring and synchronizing multi-source ship data according to the centimeter-level positioned ship position data to obtain ship situation data includes: The time synchronization protocol PTP is used to uniformly calibrate the time of radar data, sonar data, video data and AIS data; The centimeter-level ship position data is spatially correlated with radar data, sonar data, video data and AIS data, and fused to generate ship situation data.

5. The ship response system based on AI large model data fusion as claimed in claim 4, characterized in that: Spatial correlation of radar, sonar, video and AIS data with centimeter-level vessel position data, including: An affine transformation algorithm is used to convert radar data and sonar data into a relative coordinate system centered on the centimeter-level positioning of the ship; The ship position in the video data is mapped to the centimeter-level positioning of the ship using a multi-target tracking algorithm based on Kalman filtering; The linear interpolation algorithm is used to time-align the ship position information in the AIS data with the centimeter-level positioning data, and the nearest neighbor matching algorithm is used to coordinate-match the longitude and latitude coordinates in the AIS data with the centimeter-level positioning data.

6. The ship response system based on AI large model data fusion according to claim 1, characterized in that: The edge-cloud collaboration module includes an edge computing layer, a shore-based decision layer, and a collaborative training layer; The edge computing layer includes edge computing units deployed on each ship, which are used to dynamically adjust the compression rate of ship situation data according to sea conditions using a lightweight large model, and transmit the adjusted ship situation data to the shore-based decision-making layer using blind area storage technology; The shore-based decision layer includes a shore-based node, which is used to predict the obstacle trajectory of the ship at a preset distance around the ship in a preset time period in the future according to the received adjusted ship situation data and the pre-trained dynamic obstacle trajectory prediction model, and dynamically adjust the ship's speed and fuel supply to decide whether to stop sailing; The collaborative training layer is used to use the edge computing units on each ship to perform edge training on each lightweight large model, use the shore-based nodes to aggregate each lightweight large model to generate a shore-based large model, optimize the shore-based large model by updating the ship situation data, and migrate the knowledge of the shore-based large model to the lightweight large model.

7. The ship response system based on AI large model data fusion according to claim 6, characterized in that: The lightweight large model is used to dynamically adjust the compression rate of the ship situation data according to the sea conditions, and the blind area storage technology is used to transmit the adjusted ship situation data to the shore-based decision-making layer, including: Install a wave height meter with an accuracy of ±0.1m, an ultrasonic anemometer with a measurement range of 0-60m / s, and a six-axis inertial measurement unit on the ship; The edge computing units on each ship collect sea condition data including wave height and wind speed in real time and convert them into time series sea condition data; The LSTM network is used to extract features from time series sea condition data to generate feature vectors containing wave height and wind speed; The ship type and operating status labels are integrated into the feature vector through the fully connected layer to generate feature representation; According to the feature representation, prediction is performed based on a pre-built lightweight large model, and compression rate parameters are output; The compression rate of the ship situation data is adjusted according to the compression rate parameter, and the adjusted ship situation data is transmitted to the shore-based decision-making layer using the blind area storage technology; Among them, the lightweight large model is built based on the LightGBM gradient boosting decision tree model.

8. The ship response system based on AI large model data fusion according to claim 6, characterized in that: The obstacle trajectory prediction model includes: The two-layer LSTM structure includes a forward LSTM network and a backward LSTM network. Each layer of the LSTM network includes 128 units, which are used to process the trajectory sequence of the ship situation data and generate LSTM time series features. A graph neural network, including nodes and edges, is used to generate an obstacle relationship feature matrix and a GNN relationship feature by performing a 3rd-order neighbor information aggregation based on node features, edge features, and a message passing mechanism using a GraphSAGE algorithm; wherein the node features include obstacle motion states and obstacle types; and the edge features include relative distances, speed differences, and heading angles between obstacles; The attention mechanism module includes 4 multi-head self-attention layers, which are used to fuse LSTM temporal features with GNN relational features using the Transformer decoder, generate attention weights through learnable parameters, and generate fused feature representations; The prediction output layer includes a trajectory prediction branch and a risk assessment branch. The trajectory prediction branch is used to output the position probability distribution of the ship at a preset distance in a future preset time period according to the fusion feature representation using the fully connected layer. The risk assessment branch is used to output the collision probability through a sigmoid activation function based on the position probability distribution of the ship at a preset distance in a future preset time period.

9. The ship response system based on AI large model data fusion according to claim 8, characterized in that: The curriculum learning paradigm is adopted to pre-train the two-layer LSTM structure first, and then gradually add the graph neural network and attention mechanism modules to train the obstacle trajectory prediction model, including: Obtain the historical trajectory of obstacles, current influence coefficients, obstacle types, interactions between obstacles, and heading change rates in the target sea area; Only the two-layer LSTM network is activated, and the trajectory sequence of obstacle situation data is processed using the training set containing the historical trajectory of obstacles and the influence coefficient of ocean currents to generate LSTM time series features; Then, the graph neural network is activated, and the training set containing obstacle types and interactions between obstacles is used. The GraphSAGE algorithm is used to aggregate the third-order neighbor information to generate the obstacle relationship feature matrix and generate the GNN relationship features. Reactivate the attention mechanism module, use the training set containing the heading change rate, use the Transformer decoder to fuse the LSTM temporal features and the GNN relational features, generate the attention weights through the learnable parameters, and generate the fused feature representation; Then activate the prediction output layer to output the position probability distribution and collision probability of the ship at a preset distance around the preset time period in the future; Among them, the learning rate is decayed in stages, the order of graph neural network message passing is gradually increased from 1st to 3rd, the number of attention heads in the attention mechanism module is gradually increased from 4 to 8, and back-propagation training is performed according to the loss function including trajectory prediction error, collision probability prediction error and model complexity penalty to obtain a trained obstacle trajectory prediction model.

10. The ship response system based on AI large model data fusion according to claim 9, characterized in that: The loss function is expressed as: ； In the formula, represents the loss function, , and They represent the trajectory prediction errors , collision probability prediction error and model complexity penalty The weight coefficient of represents the total number of training samples, Indicates The true value of the trajectory prediction error of training samples, Indicates The trajectory prediction error prediction value of training samples, Indicates The true value of the collision probability prediction error of training samples, Indicates The prediction value of the collision probability prediction error of training samples, represents the logarithmic function, represents the regularization coefficient, represents the number of model parameters, Indicates model parameters.

Images