Ship Response System Based on AI Large Model Data Fusion

Abstract

The present invention discloses a ship response system based on AI large model data fusion, belonging to the field of ship technology, including: a dual-mode positioning module for centimeter-level positioning of ships in the target sea area according to the GNSS system and the INS system to obtain ship position data with centimeter-level positioning; a data synchronization module for acquiring and synchronizing multi-source ship data according to the ship position data with centimeter-level positioning to obtain ship situation data; an edge-cloud collaboration module for monitoring the ship in real time according to the ship situation data and the edge-shore base collaboration mechanism and dynamically adjusting the ship speed and fuel supply plan to stop sailing according to the sea conditions. The present invention solves the problems in the prior art that it is difficult to form comprehensive, accurate and comprehensive information, and it is impossible to detect potential hazards in time and prevent accidents in advance.

Description

Technical Field

[0001] The present invention relates to the field of ship technology, and in particular to a ship response system based on AI large model data fusion. Background Art

[0002] In recent years, in the shipping field, the existing ship monitoring technology has made significant progress and built a relatively complete monitoring system. This system uses a variety of advanced technologies, among which satellite communication technology plays a key role. Through the global satellite network, the location information of the ship can be obtained in real time. Whether the ship is sailing in the vast ocean or the complex inland waterway, its dynamics can be accurately captured. The geographic information system (GIS) combines the ship's location information with the electronic chart, presenting the ship's navigation track, location and surrounding geographical environment in an intuitive and visual way, providing monitoring personnel with a clear picture of the ship's operation status. The automatic identification system (AIS) is also an important part of the existing monitoring technology. The AIS equipment installed on the ship can automatically and frequently send out relevant information about the ship, such as the ship type, size, heading, speed, etc. The surrounding ships and shore-based monitoring stations can receive this information, realizing information exchange between ships and real-time tracking of ships by shore-based stations. In addition, sensor technology has also been widely used in ship monitoring. Various sensors installed in key parts of the ship, such as temperature sensors, pressure sensors, liquid level sensors, etc., can monitor the operating status of ship equipment, cargo storage conditions and other parameters in real time. Once an abnormal situation is detected, an alarm can be issued in time to ensure the safe navigation of the ship.

[0003] Although the existing ship monitoring technology is relatively mature, there are still some problems that need to be solved. On the one hand, the data fusion and processing capabilities are insufficient. At present, there are differences in the data formats and standards from different monitoring equipment and systems, which makes it difficult to merge data and form comprehensive and accurate ship information. At the same time, in the face of massive monitoring data, the existing data processing algorithms and platforms are inefficient and cannot timely and effectively analyze and mine the potential information behind the data, so that some safety hazards and abnormal situations cannot be discovered in time. On the other hand, there are blind spots in monitoring coverage. Although technologies such as satellite communications and AIS have a wide coverage range, in some special areas, such as polar regions, near remote islands, and complex urban canyon waters, signals may be interfered or blocked, resulting in loss or inaccuracy of ship monitoring data. In addition, for some small ships and non-cooperative ships, since they may not have installed or turned on the prescribed monitoring equipment, these ships are outside the monitoring blind area, which increases the difficulty of maritime safety management. Moreover, the intelligence level of existing monitoring systems is limited. Most of them can only realize simple data monitoring and alarm functions. They lack intelligent prediction and decision support for ship navigation status, and it is difficult to take effective preventive measures in advance to avoid accidents. Summary of the Invention

[0004] The technical problem solved by the present invention is that in the prior art, it is difficult to form comprehensive, accurate and integrated information, and it is impossible to detect potential hazards in time and prevent accidents in advance.

[0005] To solve the above technical problems, the present invention provides the following technical solutions:

[0006] The ship response system based on AI large model data fusion provided by the present invention includes:

[0007] A dual-mode positioning module, which is used to perform centimeter-level positioning on ships in the target sea area according to the GNSS system and the INS system, and obtain the ship position data at centimeter level;

[0008] A data synchronization module, which is used to obtain and synchronize the multi-source data of the ship according to the ship position data at centimeter level, and obtain the ship situation data;

[0009] An edge-cloud collaboration module, which is used to monitor the ship in real time according to the ship situation data and the edge-shore collaboration mechanism, and dynamically adjust the ship speed and fuel supply plan to stop sailing according to the sea conditions.

[0010] Further, performing centimeter-level positioning on ships in the target sea area according to the GNSS system and the INS system, and obtaining the ship position data at centimeter level, includes:

[0011] In the target sea area, divide the target sea area into several grid areas according to the ship traffic, terrain features and signal occlusion conditions;

[0012] Set up reference stations at the center of each grid area, add redundant reference stations in areas where signals are vulnerable to interference, and set the coverage radius of the reference stations and redundant reference stations;

[0013] Establish a dynamic adjustment mechanism to adjust the positions of the reference stations and redundant reference stations according to the ship traffic and signal changes;

[0014] Install a GNSS receiver and an INS system on the ship as a mobile station to receive satellite signals and measure the motion state of the ship;

[0015] Use satellite communication or wireless communication to establish a real-time communication link between the reference station and the mobile station, perform time synchronization processing on the received data, and align the data at different times to the same time reference;

[0016] Fuse the data of the GNSS system and the INS system through an extended Kalman filter algorithm, and calculate the position, heading and speed of the ship;

[0017] Based on the evaluation metrics of position error, heading error, and speed error, the position, heading, and speed of the ship are accurately evaluated to obtain the accuracy evaluation result;

[0018] According to the accuracy evaluation result, the parameters of the extended Kalman filter algorithm are adjusted and optimized to make the calculated ship position reach centimeter-level accuracy, and the ship position data with centimeter-level positioning is obtained.

[0019] Furthermore, the multi-source data of the ship includes radar data, sonar data, video data, and AIS data.

[0020] Furthermore, the multi-source data of the ship is synchronized according to the ship position data with centimeter-level positioning to obtain the ship situation data, including:

[0021] The time synchronization protocol PTP is used to uniformly calibrate the time of radar data, sonar data, video data, and AIS data;

[0022] The ship position data with centimeter-level positioning is spatially associated with radar data, sonar data, video data, and AIS data, and the ship situation data is fused and generated.

[0023] Furthermore, the spatial association of radar data, sonar data, video data, and AIS data with the ship position data with centimeter-level positioning includes:

[0024] The affine transformation algorithm is used to convert radar data and sonar data into a relative coordinate system centered on the ship position with centimeter-level positioning;

[0025] The multi-target tracking algorithm based on Kalman filter is used to map the ship position in the video data to the ship position with centimeter-level positioning;

[0026] The linear interpolation algorithm is used to align the ship position information in the AIS data with the centimeter-level positioning data in time, and the nearest neighbor matching algorithm is used to match the longitude and latitude coordinates in the AIS data with the centimeter-level positioning data.

[0027] Furthermore, the edge-cloud collaboration module includes an edge computing layer, a shore-based decision-making layer, and a collaborative training layer;

[0028] The edge computing layer includes edge computing units deployed on each ship, which are used to dynamically adjust the compression rate of the ship situation data according to the sea conditions by using a lightweight large model, and transmit the adjusted ship situation data to the shore-based decision-making layer by using the blind area storage technology;

[0029] The shore-based decision-making layer includes shore-based nodes, which are used to predict the obstacle trajectories within a preset circumferential distance of the ship in a future preset time period according to the received adjusted ship situation data and a pre-trained dynamic obstacle trajectory prediction model, and dynamically adjust the ship's speed and fuel supply, and decide whether to stop sailing;

[0030] The collaborative training layer is used to perform edge training on each lightweight large model using the edge computing units on each ship, aggregate each lightweight large model using the shore-based nodes to generate a shore-based large model, optimize the shore-based large model by updating the ship situation data, and transfer the knowledge of the shore-based large model to the lightweight large model.

[0031] Further, using the lightweight large model to dynamically adjust the compression rate of the ship situation data according to the sea conditions, and using the blind area storage technology to transmit the adjusted ship situation data to the shore-based decision-making layer, including:

[0032] Install a wave height meter with an accuracy of ±0.1 m, an ultrasonic anemometer with a measurement range of 0 - 60 m / s, and a six-axis inertial measurement unit on the ship;

[0033] Collect sea condition data including wave height and wind speed in real time through the edge computing units on each ship and convert it into time series sea condition data;

[0034] Use the LSTM network to extract features from the time series sea condition data to generate a feature vector including wave height and wind speed;

[0035] Integrate the ship type and operation status labels into the feature vector through a fully connected layer to generate a feature representation;

[0036] Based on the feature representation, perform prediction based on a pre-constructed lightweight large model and output a compression rate parameter;

[0037] Adjust the compression rate of the ship situation data according to the compression rate parameter, and use the blind area storage technology to transmit the adjusted ship situation data to the shore-based decision-making layer;

[0038] Among them, the lightweight large model is constructed based on the LightGBM gradient boosting decision tree model.

[0039] Further, the obstacle trajectory prediction model includes:

[0040] A double-layer LSTM structure, including a forward LSTM network and a backward LSTM network, with 128 units in each layer of the LSTM network, which is used to process the trajectory sequence of the ship situation data and generate LSTM time series features;

[0041] The graph neural network, including nodes and edges, is used to generate an obstacle relationship feature matrix by aggregating 3-hop neighbor information using the GraphSAGE algorithm based on node features, edge features, and a message passing mechanism, and generate GNN relationship features; wherein, the node features include the motion state and type of the obstacle; the edge features include the relative distance, speed difference, and heading angle between obstacles;

[0042] The attention mechanism module, including 4 multi-head self-attention layers, is used to fuse LSTM temporal features and GNN relationship features using a Transformer decoder, generate attention weights through learnable parameters, and generate a fused feature representation;

[0043] The prediction output layer includes a trajectory prediction branch and a risk assessment branch. The trajectory prediction branch is used to output the position probability distribution of the ship at a preset circumferential distance within a preset future time period using a fully connected layer. The risk assessment branch is used to output the collision probability through a sigmoid activation function based on the position probability distribution of the ship at a preset circumferential distance within a preset future time period.

[0044] Furthermore, adopting a curriculum learning paradigm, first pre-training a two-layer LSTM structure, and then gradually adding a graph neural network and an attention mechanism module, the training method for training the obstacle trajectory prediction model includes:

[0045] Obtain the historical trajectories of obstacles, ocean current influence coefficients, obstacle types, interactions between obstacles, and heading change rates within the target sea area;

[0046] Only activate the two-layer LSTM network, use the training set containing the historical trajectories of obstacles and ocean current influence coefficients to process the trajectory sequence of obstacle situation data, and generate LSTM temporal features;

[0047] Then activate the graph neural network, use the training set containing obstacle types and interactions between obstacles, and adopt the GraphSAGE algorithm to aggregate 3-hop neighbor information to generate an obstacle relationship feature matrix and generate GNN relationship features;

[0048] Then activate the attention mechanism module, use the training set containing the heading change rate, use a Transformer decoder to fuse LSTM temporal features and GNN relationship features, generate attention weights through learnable parameters, and generate a fused feature representation;

[0049] Then activate the prediction output layer to output the position probability distribution and collision probability of the ship at a preset circumferential distance within a preset future time period;

[0050] Among them, the learning rate is decayed in stages, the number of message passing orders of the graph neural network is gradually increased from 1 to 3, the number of attention heads in the attention mechanism module is gradually increased from 4 to 8, and backpropagation training is performed according to a loss function that includes trajectory prediction error, collision probability prediction error, and model complexity penalty, to obtain a trained obstacle trajectory prediction model.

[0051] Further, the loss function is expressed as:

[0052] ;

[0053] In the formula, represents the loss function, , and respectively represent the trajectory prediction error , the collision probability prediction error and the model complexity penalty weight coefficients, represents the total number of training samples, represents the true value of the trajectory prediction error of the th training sample, represents the predicted value of the trajectory prediction error of the th training sample, represents the true value of the collision probability prediction error of the th training sample, represents the predicted value of the collision probability prediction error of the th training sample, represents the logarithmic function, represents the regularization coefficient, represents the number of model parameters, represents the th model parameter.

[0054] Compared with the prior art, the beneficial effects of the present invention are:

[0055] 1. The present invention achieves centimeter-level positioning through a dual-mode positioning module to obtain high-precision ship position data. Based on this, the data synchronization module synchronizes multi-source ship data to obtain ship situation data. Furthermore, the edge-cloud collaboration module monitors the ship in real time according to the ship situation data and the edge-shore base collaboration mechanism, and dynamically adjusts the ship speed and fuel supply plan to stop the ship. This method of high-precision positioning and multi-source data fusion enables ship monitoring to obtain more comprehensive and accurate information. The adjustment of the ship speed and fuel supply plan based on this information is more scientific and reasonable, which can effectively improve the economy and safety of ship navigation, reduce unnecessary fuel consumption, and safety risks caused by unreasonable navigation plans. It solves the problems in the prior art that it is difficult to form comprehensive, accurate and comprehensive information, and it is impossible to detect potential hazards in time and prevent accidents in advance.

[0056] 2. The present invention performs centimeter-level positioning on ships in the target sea area according to the GNSS system and the INS system. Through a series of fine operations such as dividing grid areas, setting up reference stations and redundant reference stations, and establishing a dynamic adjustment mechanism, combined with the extended Kalman filter algorithm to fuse data and optimize parameters, finally, ship position data with centimeter-level accuracy is obtained. This high-precision positioning can accurately grasp the real-time position of the ship, providing a solid foundation for subsequent multi-source data fusion and situation analysis. At the same time, the dynamic adjustment mechanism can adjust the positions of the reference stations and redundant reference stations according to ship traffic and signal changes to ensure high-precision positioning in different situations, greatly improving the accuracy and reliability of ship monitoring, and reducing monitoring errors and safety hazards caused by positioning errors.

[0057] 3. The present invention integrates various ship multi-source data such as radar data, sonar data, video data, and AIS data, and uses the time synchronization protocol PTP for time unified calibration. Then, affine transformation algorithms, multi-target tracking algorithms based on Kalman filtering, linear interpolation algorithms, and nearest neighbor matching algorithms are used to spatially associate the ship position data with centimeter-level positioning with these multi-source data, and ship situation data is fused and generated. The fusion and spatial association of this multi-source data can make full use of the advantages of different data sources to comprehensively and accurately present the surrounding environment and its own state of the ship, providing richer and more intuitive information for monitoring personnel, helping to detect potential safety problems and abnormal situations in time, and improving the comprehensiveness and effectiveness of ship monitoring.

[0058] 4. The edge-cloud collaboration module of the present invention includes an edge computing layer, a shore-based decision-making layer, and a collaborative training layer, and each layer works collaboratively. The edge computing layer uses a lightweight large model to dynamically adjust the compression rate of ship situation data according to sea conditions and adopts a blind area storage technology to transmit data, which not only ensures the timely transmission of data but also reduces the occupancy of network bandwidth. The shore-based decision-making layer can accurately predict the obstacle trajectory within a preset circumferential distance of the ship within a preset future time period based on the received data and a pre-trained dynamic obstacle trajectory prediction model, and dynamically adjust the ship's speed and fuel supply, and decide whether to stop sailing, realizing autonomous response and efficient decision-making for the ship. The collaborative training layer continuously optimizes the model through edge training and shore-based aggregation to improve the accuracy of prediction and the scientific nature of decision-making. This mechanism of edge-cloud collaboration and intelligent prediction greatly improves the intelligent level of ship monitoring, can take effective preventive measures in advance, avoid accidents, and ensure the safe navigation of the ship. Description of the Drawings

[0059] Figure 1 It is a schematic structural diagram of a ship response system based on AI large model data fusion provided by an embodiment of the present invention;

[0060] Figure 2 It is a schematic structural diagram of an edge-cloud collaboration module provided by an embodiment of the present invention;

[0061] Figure 3 It is a schematic structural diagram of an obstacle trajectory prediction model provided by an embodiment of the present invention;

[0062] Figure 4 It is a schematic basic process diagram of a training method for an obstacle trajectory prediction model provided by an embodiment of the present invention. Detailed Embodiments

[0063] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following detailed description of the specific embodiments of the present invention will be given in conjunction with the drawings of the specification. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention.

[0064] Embodiment 1

[0065] As Figure 1 shown, an embodiment of the present invention provides a ship response system based on AI large model data fusion, including:

[0066] A dual-mode positioning module, configured to perform centimeter-level positioning on ships in the target sea area according to the GNSS system and the INS system to obtain ship position data with centimeter-level positioning.

[0067] The present invention performs positioning by combining the Global Navigation Satellite System (GNSS) and the Inertial Navigation System (INS). GNSS determines the position by receiving satellite signals, but in some complex environments such as being blocked by high-rise buildings or having weak satellite signals in the deep ocean, the positioning accuracy and reliability will be affected. INS uses sensors such as accelerometers and gyroscopes to measure the acceleration and angular velocity of the ship, and obtains position, velocity, and attitude information through integral operations, but there is a problem of error accumulation. The present invention complements the advantages of the two through a dual-mode positioning module. GNSS provides high-precision absolute position information, and INS provides short-term high-precision positioning and attitude information when the GNSS signal is lost, thereby achieving centimeter-level positioning of ships in the target sea area.

[0068] The present invention performs fusion processing on the data collected by GNSS and INS, and uses algorithms such as Kalman filtering to fuse the high-precision position information of GNSS with the continuous measurement data of INS, suppress the error accumulation of INS, and improve the accuracy and reliability of positioning, and finally obtain the ship position data with centimeter-level positioning.

[0069] A data synchronization module is used to obtain and synchronize the multi-source data of the ship according to the ship position data with centimeter-level positioning to obtain the ship situation data.

[0070] Based on the ship position data with centimeter-level positioning output by the dual-mode positioning module, the present invention obtains the multi-source data of the ship. Through the data synchronization mechanism, the data from different sources and in different formats are time-aligned and format-unified to ensure the consistency and integrity of the data. The present invention integrates and analyzes the synchronized multi-source data of the ship, extracts key information, and forms the ship situation data.

[0071] An edge-cloud collaboration module is used to monitor the ship in real time according to the ship situation data and the edge-shore collaboration mechanism, and dynamically adjust the ship speed and fuel supply plan to stop sailing according to the sea conditions.

[0072] Based on the ship situation data and the edge-shore collaboration mechanism, the present invention realizes real-time monitoring of the ship. Edge computing devices are deployed on the ship to perform real-time processing and analysis of the ship situation data, and quickly respond to the local needs of the ship. At the same time, key data is uploaded to the cloud server on the shore, and using big data analysis and artificial intelligence algorithms, the ship situation data is analyzed and mined more deeply to provide global decision-making support for the ship, such as adjusting the ship speed and fuel supply plan in real time according to sea condition information such as wind and wave size, water flow speed, etc. and the ship situation data.

[0073] For example, when the sea conditions are severe, the ship speed is reduced to ensure the safety of the ship, and at the same time, the fuel supply plan is adjusted to reduce fuel consumption; when the sea conditions are good, the ship speed is appropriately increased to improve the transportation efficiency, and through the edge-cloud collaboration mechanism, the autonomous response and optimized operation of the ship are achieved.

[0074] Embodiment 2

[0075] As Figure 2 shown, this is another embodiment of the present invention. With the same inventive concept as Embodiment 1, it provides the implementation steps of a ship response system based on AI large model data fusion, including:

[0076] Step 1: Construct a dual-mode positioning module.

[0077] The dual-mode positioning module provided by the present invention is used to perform centimeter-level positioning on ships in the target sea area according to the GNSS system and the INS system, and obtain the ship position data with centimeter-level positioning.

[0078] In this embodiment, the ship multi-source data includes radar data, sonar data, video data, and AIS data.

[0079] In this embodiment, performing centimeter-level positioning on ships in the target sea area according to the GNSS system and the INS system, and obtaining the ship position data with centimeter-level positioning, includes:

[0080] Step 1.1: In the target sea area, divide the target sea area into several grid regions according to ship traffic, terrain features, and signal occlusion conditions.

[0081] The present invention comprehensively considers multiple factors such as ship traffic, terrain features, and signal occlusion conditions, and uses geographic information system (GIS) technology to perform spatial analysis on the target sea area. By analyzing the ship density in different regions and the influence of terrain undulations such as islands and reefs on signal propagation, the sea area is divided into several grid regions. The size and shape of each grid region are dynamically adjusted according to the actual situation to ensure relative consistency of signal characteristics and ship activities within the region.

[0082] The present invention improves the scientificity and pertinence of positioning planning. Through grid division, corresponding positioning strategies can be formulated according to the characteristics of different regions. For example, increase the investment in positioning resources in areas with high ship traffic, and take special positioning measures in areas with complex terrain and vulnerable signal interference, thereby improving the overall positioning efficiency and accuracy.

[0083] Step 1.2: Set a reference station at the center of each grid region, add redundant reference stations in areas where signals are vulnerable to interference, and set the coverage radius of the reference station and the redundant reference stations.

[0084] In the present invention, a reference station is set at the center of each grid area. The reference station receives precise signals from satellites and serves as a reference point for positioning. Redundant reference stations are added in areas where signals are vulnerable to interference, such as near the coast with high-rise buildings or areas with large sea waves and complex signal reflections on the sea surface, to improve the reliability and fault tolerance of positioning. According to the signal propagation model and actual tests, the coverage radii of the reference stations and redundant reference stations are determined to ensure that ships within the coverage can receive stable signals.

[0085] The present invention enhances the stability and reliability of the positioning system. The reference station provides precise reference signals for ships, while the redundant reference stations come into play when the main reference station fails or the signal is interfered, ensuring that ships can always obtain reliable positioning information and avoiding positioning interruption or accuracy degradation caused by the failure of a single reference station.

[0086] Step 1.3: Establish a dynamic adjustment mechanism to adjust the positions of the reference stations and redundant reference stations according to ship traffic and signal changes.

[0087] The present invention monitors ship traffic and signal changes in real time. Through a sensor network and data analysis algorithms, information such as the position, speed, and signal strength of ships is obtained. Based on this information, optimization algorithms such as genetic algorithms and particle swarm algorithms are used to dynamically adjust the positions of the reference stations and redundant reference stations to adapt to changes in ship traffic and signal environments.

[0088] The present invention improves the adaptability and flexibility of the positioning system. With changes in ship traffic and signal environments, dynamically adjusting the positions of the reference stations and redundant reference stations can ensure that the positioning system is always in the best working state, improving the accuracy and efficiency of positioning.

[0089] Step 1.4: Install a GNSS receiver and an INS system on the ship to serve as a mobile station to receive satellite signals and measure the motion state of the ship.

[0090] In the present invention, a GNSS receiver and an INS system are installed on the ship. The GNSS receiver receives satellite signals to obtain the preliminary position information of the ship; the INS system uses sensors such as accelerometers and gyroscopes to measure the motion state of the ship, such as acceleration and angular velocity. By fusing the data of the two systems, the accuracy and reliability of positioning can be improved.

[0091] The present invention provides real-time position and motion state information for ships. The combined use of the GNSS receiver and the INS system can give full play to the advantages of the two systems. When the GNSS signal is interfered or lost, the INS system can continue to provide short-term high-precision positioning and motion state information to ensure the navigation safety of ships.

[0092] Step 1.5: Establish a real-time communication link between the reference station and the mobile station using satellite communication or wireless communication, perform time synchronization processing on the received data, and align the data at different times to the same time reference.

[0093] The present invention uses satellite communication or wireless communication technology to establish a real-time communication link between the reference station and the mobile station, ensuring that the reference signal of the reference station can be transmitted to the mobile station on the ship in a timely manner. Perform time synchronization processing on the received data, and use technologies such as the Network Time Protocol (NTP) or the Precision Time Protocol (PTP) to align the data at different times to the same time reference to ensure the consistency and accuracy of the data.

[0094] The present invention realizes real-time data interaction between the reference station and the mobile station. The establishment of the real-time communication link provides guarantee for data transmission, and the time synchronization processing ensures the accuracy and reliability of the data, providing a basis for subsequent data fusion processing.

[0095] Step 1.6: Fuse the data of the GNSS system and the INS system through the Extended Kalman Filter algorithm, and calculate the position, heading and speed of the ship.

[0096] The Extended Kalman Filter algorithm is a state estimation method for nonlinear systems. By fusing the data of the GNSS system and the INS system, comprehensively considering the measurement errors and noise characteristics of the two systems, the position, heading and speed of the ship are calculated. At each moment, according to the state equation and observation equation of the system, the state of the ship is predicted and updated to continuously improve the positioning accuracy.

[0097] The present invention improves the positioning accuracy and stability. The Extended Kalman Filter algorithm can effectively suppress the measurement errors and noise of the GNSS and INS systems, make full use of the information of the two systems, and improve the positioning accuracy and reliability.

[0098] Step 1.7: Based on the evaluation indicators of position error, heading error and speed error, evaluate the position, heading and speed of the ship to obtain the accuracy evaluation result.

[0099] The present invention evaluates the position, heading and speed of the ship based on evaluation indicators such as position error, heading error and speed error. By comparing with high-precision reference data, the values of each error indicator are calculated to evaluate the accuracy of the positioning result. The accuracy evaluation result can intuitively reflect the accuracy level of the positioning system and provide a basis for subsequent parameter adjustment and optimization.

[0100] Step 1.8: Adjust and optimize the parameters of the extended Kalman filter algorithm according to the accuracy evaluation results, so that the calculated ship position reaches centimeter-level accuracy, and obtain the ship position data with centimeter-level positioning.

[0101] According to the accuracy evaluation results, the present invention uses optimization algorithms such as the gradient descent method and the genetic algorithm to adjust and optimize the parameters of the extended Kalman filter algorithm. By continuously adjusting the parameters, the calculated ship position reaches centimeter-level accuracy.

[0102] The present invention improves the accuracy and performance of the positioning system. By adjusting and optimizing the algorithm parameters, the positioning accuracy can be further improved to meet the high-precision positioning requirements of the ship intelligent monitoring and autonomous response system.

[0103] Step 2: Construct a data synchronization module.

[0104] The data synchronization module provided by the present invention is used to obtain and synchronize the multi-source data of the ship according to the ship position data with centimeter-level positioning, and obtain the ship situation data.

[0105] In this embodiment, the obtaining and synchronizing the multi-source data of the ship according to the ship position data with centimeter-level positioning to obtain the ship situation data includes:

[0106] Step 2.1: Use the Precision Time Protocol (PTP) to uniformly calibrate the time of radar data, sonar data, video data, and AIS data.

[0107] Based on the Precision Time Protocol (PTP), through the information exchange between the master clock and the slave clock, the best master clock algorithm (BMC) is used to determine the best master clock in the network. Then, the slave clock calculates the clock deviation and link delay according to the timestamp information sent by the master clock, and adjusts its own clock to achieve time synchronization with the master clock. In the ship monitoring system, devices such as radar, sonar, video, and AIS are distributed at different positions and transmit data through the network. PTP can ensure the consistency of the data collected by these devices in time.

[0108] Step 2.2: Perform spatial association on the ship position data with centimeter-level positioning, radar data, sonar data, video data, and AIS data, and fuse them to generate ship situation data.

[0109] The present invention comprehensively processes the data from different sensors and systems to generate comprehensive and accurate ship operation state information. By performing spatial association on the ship position data with centimeter-level positioning, radar, sonar, video, and AIS data, the advantages of various data sources can be fully utilized to make up for the deficiencies of a single data source.

[0110] Spatially correlate radar data, sonar data, video data, and AIS data with the ship position data with centimeter-level positioning, including:

[0111] Step 2.2.1: Use the affine transformation algorithm to convert the radar data and sonar data into a relative coordinate system centered on the ship position with centimeter-level positioning.

[0112] The present invention utilizes affine transformation to map points in one coordinate system to another coordinate system through linear transformation. In ship monitoring, the present invention converts the radar data and sonar data into a relative coordinate system centered on the ship position with centimeter-level positioning, which can facilitate the unified processing and analysis of data from different sensors. By determining the correspondence between the radar and sonar data and the centimeter-level positioning data, the affine transformation matrix is calculated, and the coordinate points in the radar and sonar data are transformed according to the transformation matrix so that they are in the same coordinate system as the centimeter-level positioning data.

[0113] Step 2.2.2: Use the multi-object tracking algorithm based on Kalman filtering to map the ship positions in the video data to the ship positions with centimeter-level positioning.

[0114] The present invention utilizes Kalman filtering to effectively track the motion state of the target by predicting and updating the state of the target. In ship monitoring, the present invention maps the ship positions in the video data to the ship positions with centimeter-level positioning. The Kalman filtering algorithm can track multiple ship targets, and estimate and predict the motion state of the ships based on the observation information in the video data such as the positions and speeds of the ships.

[0115] Step 2.2.3: Use the linear interpolation algorithm to align the ship position information in the AIS data with the centimeter-level positioning data in terms of time, and use the nearest neighbor matching algorithm to match the longitude and latitude coordinates in the AIS data with the centimeter-level positioning data.

[0116] The present invention utilizes the linear interpolation algorithm to estimate the data values at unknown times through the linear relationship between known data points, achieving the time alignment of the AIS data and the centimeter-level positioning data. The nearest neighbor matching algorithm is to find the matching item closest to the target data in the known dataset, which is used for the coordinate matching of the AIS data and the centimeter-level positioning data, ensuring the temporal consistency of the AIS data and the centimeter-level positioning data, avoiding data mismatch problems caused by time differences. The nearest neighbor matching algorithm can quickly and accurately match the longitude and latitude coordinates in the AIS data with the centimeter-level positioning data, providing an accurate coordinate basis for subsequent data fusion.

[0117] Step 3: Construct an edge-cloud collaboration module.

[0118] The edge-cloud collaboration module provided by the present invention is used to monitor ships in real time according to ship situation data and the edge-shore collaboration mechanism, and dynamically adjust the ship speed and fuel supply plan to stop sailing according to sea conditions.

[0119] As Figure 2 shown, the edge-cloud collaboration module includes an edge computing layer, a shore-based decision-making layer, and a collaborative training layer.

[0120] Step 3.1: Construct the edge computing layer.

[0121] The edge computing layer includes edge computing units deployed on each ship, which are used to dynamically adjust the compression ratio of ship situation data according to sea conditions by using a lightweight large model, and transmit the adjusted ship situation data to the shore-based decision-making layer by using blind area storage technology.

[0122] In this embodiment, dynamically adjusting the compression ratio of ship situation data according to sea conditions by using a lightweight large model, and transmitting the adjusted ship situation data to the shore-based decision-making layer by using blind area storage technology includes:

[0123] Step 3.1.1: Install a wave height meter with an accuracy of ±0.1 m, an ultrasonic anemometer with a measurement range of 0 - 60 m / s, and a six-axis inertial measurement unit on the ship.

[0124] Step 3.1.2: Real-time collect sea condition data including wave height and wind speed through the edge computing units on each ship and convert it into time series sea condition data.

[0125] Step 3.1.3: Use an LSTM network to extract features from the time series sea condition data to generate a feature vector including wave height and wind speed.

[0126] Step 3.1.4: Incorporate the ship type and operation status labels into the feature vector through a fully connected layer to generate a feature representation.

[0127] Step 3.1.5: Based on the feature representation, perform prediction based on a pre-constructed lightweight large model and output a compression ratio parameter.

[0128] Step 3.1.6: Adjust the compression ratio of the ship situation data according to the compression ratio parameter, and transmit the adjusted ship situation data to the shore-based decision-making layer by using blind area storage technology.

[0129] Among them, the lightweight large model is constructed based on the LightGBM gradient boosting decision tree model.

[0130] Step 3.2: Construct the shore-based decision-making layer.

[0131] The shore-based decision-making layer includes shore-based nodes, which are used to predict the obstacle trajectories within a preset circumferential distance of the ship in a future preset time period according to the received adjusted ship situation data and a pre-trained dynamic obstacle trajectory prediction model, and dynamically adjust the ship's speed and fuel supply to decide whether to stop sailing.

[0132] As Figure 3 shown, the obstacle trajectory prediction model includes a double-layer LSTM structure, a graph neural network, an attention mechanism module, and a prediction output layer.

[0133] The double-layer LSTM structure includes a forward LSTM network and a backward LSTM network. Each layer of the LSTM network includes 128 units, which are used to process the trajectory sequence of the ship situation data and generate LSTM temporal features.

[0134] The graph neural network includes nodes and edges, which are used to generate an obstacle relationship feature matrix by aggregating 3-hop neighbor information using the GraphSAGE algorithm according to node features, edge features, and a message passing mechanism, and generate GNN relationship features. Among them, the node features include the obstacle motion state and the obstacle type; the edge features include the relative distance, speed difference, and heading angle between obstacles.

[0135] The attention mechanism module includes 4 multi-head self-attention layers, which are used to fuse the LSTM temporal features and GNN relationship features using a Transformer decoder, generate attention weights through learnable parameters, and generate a fused feature representation.

[0136] The prediction output layer includes a trajectory prediction branch and a risk assessment branch. The trajectory prediction branch is used to output the position probability distribution within a preset circumferential distance of the ship in a future preset time period using a fully connected layer. The risk assessment branch is used to output the collision probability through a sigmoid activation function according to the position probability distribution within a preset circumferential distance of the ship in a future preset time period.

[0137] In this embodiment, a curriculum learning paradigm is adopted. First, the double-layer LSTM structure is pre-trained, and then the graph neural network and the attention mechanism module are gradually added. The training method for training the obstacle trajectory prediction model includes:

[0138] Obtain the historical obstacle trajectories, ocean current influence coefficients, obstacle types, interactions between obstacles, and heading change rates in the target sea area.

[0139] Only activate the double-layer LSTM network, and use the training set containing the historical obstacle trajectories and ocean current influence coefficients to process the trajectory sequence of the obstacle situation data and generate LSTM temporal features.

[0140] Reactivate the graph neural network, and use the training set containing obstacle types and the interactions between obstacles. Adopt the GraphSAGE algorithm to perform 3-order neighbor information aggregation to generate an obstacle relationship feature matrix, and generate GNN relationship features;

[0141] Reactivate the attention mechanism module, and use the training set containing the rate of change of heading. Use the Transformer decoder to fuse the LSTM temporal features and GNN relationship features, and generate attention weights through learnable parameters to generate a fused feature representation;

[0142] Reactivate the prediction output layer, and output the position probability distribution and collision probability of the ship at a preset circumferential distance within a preset future time period;

[0143] Among them, the learning rate is decayed in stages, the message passing order of the graph neural network is gradually increased from 1 order to 3 orders, the number of attention heads in the attention mechanism module is gradually increased from 4 to 8, and according to the loss function containing the trajectory prediction error, collision probability prediction error, and model complexity penalty, perform backpropagation training to obtain a trained obstacle trajectory prediction model.

[0144] In this embodiment, the loss function is expressed as:

[0145] ;

[0146] In the formula, represents the loss function, , and respectively represent the trajectory prediction error , the collision probability prediction error and the model complexity penalty weight coefficients, represents the total number of training samples, represents the true value of the trajectory prediction error of the th training sample, represents the predicted value of the trajectory prediction error of the th training sample, represents the true value of the collision probability prediction error of the th training sample, represents the predicted value of the collision probability prediction error of the th training sample, represents the logarithmic function, represents the regularization coefficient, represents the number of model parameters, represents the th model parameter.

[0147] Step 3.3: Cooperative training layer.

[0148] The collaborative training layer is used to perform edge training on each lightweight large model using the edge computing units on each ship, aggregate each lightweight large model using shore-based nodes to generate a shore-based large model, optimize the shore-based large model by updating ship situation data, and transfer the knowledge of the shore-based large model to the lightweight large model.

[0149] Through the edge training of edge computing units, model aggregation of shore-based nodes, model optimization based on ship situation data, and knowledge transfer to lightweight large models, the collaborative training layer realizes a distributed and collaborative optimization machine learning system, which can make full use of the computing resources of ships and shore-based facilities, protect data privacy, improve the adaptability and performance of the model, and provide strong support for the intelligent monitoring and autonomous response of ships. At the same time, the collaborative training mechanism enables the system to continuously adapt to the changes in the actual environment and achieve continuous performance improvement and optimization.

[0150] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. Among them, the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, abbreviated as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, abbreviated as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, abbreviated as EPROM), programmable read-only memory (Programmable Red-Only Memory, abbreviated as PROM), read-only memory (Read-Only Memory, abbreviated as ROM), magnetic memory, flash memory, magnetic disk or optical disc. These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device, and the instruction device implements the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or multiple blocks specified in the block.

[0151] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A ship response system based on AI large model data fusion, characterized in that, Including: A dual-mode positioning module, which is used to perform centimeter-level positioning on ships in the target sea area according to the GNSS system and the INS system, and obtain the ship position data with centimeter-level positioning; A data synchronization module, which is used to obtain and synchronize multi-source ship data according to the ship position data with centimeter-level positioning, and obtain ship situation data; An edge-cloud collaboration module, which is used to monitor ships in real time according to the ship situation data and the edge-shore base collaboration mechanism, and dynamically adjust the ship speed and fuel supply plan to stop sailing according to the sea conditions; The edge-cloud collaboration module includes an edge computing layer, a shore-based decision-making layer, and a collaborative training layer; The edge computing layer includes edge computing units deployed on each ship, which are used to dynamically adjust the compression ratio of ship situation data according to the sea conditions by using a lightweight large model, and use the blind area storage technology to transmit the adjusted ship situation data to the shore-based decision-making layer; The shore-based decision-making layer includes shore-based nodes, which are used to predict the obstacle trajectory within a preset circumferential distance of the ship within a preset future time period according to the received adjusted ship situation data and a pre-trained dynamic obstacle trajectory prediction model, and dynamically adjust the ship speed and fuel supply, and decide whether to stop sailing; The collaborative training layer is used to perform edge training on each lightweight large model by using the edge computing units on each ship, use the shore-based nodes to aggregate each lightweight large model to generate a shore-based large model, optimize the shore-based large model by updating the ship situation data, and transfer the knowledge of the shore-based large model to the lightweight large model; Dynamically adjusting the compression ratio of ship situation data according to the sea conditions by using a lightweight large model, and using the blind area storage technology to transmit the adjusted ship situation data to the shore-based decision-making layer, including: Install a wave height meter with an accuracy of ±0.1 m, an ultrasonic anemometer with a measurement range of 0-60 m / s, and a six-axis inertial measurement unit on the ship; Real-time collect sea condition data including wave height and wind speed through the edge computing units on each ship and convert it into time series sea condition data; Use the LSTM network to extract features from the time series sea condition data to generate a feature vector including wave height and wind speed; Integrate the ship type and operation status label into the feature vector through a fully connected layer to generate a feature representation; Based on the feature representation, perform prediction based on a pre-constructed lightweight large model, and output a compression ratio parameter; Adjust the compression ratio of ship situation data according to the compression ratio parameter, and use the blind area storage technology to transmit the adjusted ship situation data to the shore-based decision-making layer; Among them, the lightweight large model is constructed based on the LightGBM gradient boosting decision tree model.

2. The ship response system based on AI large model data fusion according to claim 1, characterized in that, Performing centimeter-level positioning on ships in the target sea area according to the GNSS system and the INS system, and obtaining the ship position data with centimeter-level positioning, including: In the target sea area, divide the target sea area into several grid regions according to ship traffic, terrain features, and signal occlusion conditions; Set up reference stations at the center of each grid region, add redundant reference stations in areas where signals are vulnerable to interference, and set the coverage radius of the reference stations and redundant reference stations; Establish a dynamic adjustment mechanism to adjust the positions of the reference stations and redundant reference stations according to ship traffic and signal changes; Install a GNSS receiver and an INS system on the ship to receive satellite signals and measure the motion state of the ship as a mobile station; Establish a real-time communication link between the reference station and the mobile station using satellite communication or wireless communication, perform time synchronization processing on the received data, and align the data at different times to the same time reference; Fuse the data of the GNSS system and the INS system through the extended Kalman filter algorithm to calculate the position, heading, and speed of the ship; Based on the position error, heading error, and speed error evaluation indicators, evaluate the accuracy of the ship's position, heading, and speed to obtain the accuracy evaluation result; Adjust and optimize the parameters of the extended Kalman filter algorithm according to the accuracy evaluation result to make the calculated ship position reach centimeter-level accuracy and obtain the ship position data with centimeter-level positioning.

3. The ship response system based on AI large model data fusion according to claim 1, characterized in that, The multi-source data of the ship includes radar data, sonar data, video data, and AIS data.

4. The ship response system based on AI large model data fusion according to claim 3, characterized in that, Obtain and synchronize the multi-source data of the ship according to the ship position data with centimeter-level positioning to obtain the ship situation data, including: Adopt the time synchronization protocol PTP to uniformly calibrate the time of radar data, sonar data, video data, and AIS data; Perform spatial association on the ship position data with centimeter-level positioning, radar data, sonar data, video data, and AIS data, and fuse them to generate ship situation data.

5. The ship response system based on AI large model data fusion according to claim 4, characterized in that, Perform spatial association on radar data, sonar data, video data, and AIS data with the ship position data with centimeter-level positioning, including: Adopt the affine transformation algorithm to convert radar data and sonar data into a relative coordinate system centered on the ship position with centimeter-level positioning; Use the multi-object tracking algorithm based on Kalman filter to map the ship position in the video data to the ship position with centimeter-level positioning; Adopt the linear interpolation algorithm to align the ship position information in the AIS data with the centimeter-level positioning data in time, and adopt the nearest neighbor matching algorithm to match the longitude and latitude coordinates in the AIS data with the centimeter-level positioning data.

6. The ship response system based on AI large model data fusion according to claim 1, characterized in that, The obstacle trajectory prediction model includes: A two-layer LSTM structure, including a forward LSTM network and a backward LSTM network. Each layer of the LSTM network includes 128 units, which are used to process the trajectory sequence of the ship situation data and generate LSTM temporal features; A graph neural network, including nodes and edges, which is used to aggregate the 3-order neighbor information using the GraphSAGE algorithm according to the node features, edge features, and message passing mechanism to generate an obstacle relationship feature matrix and generate GNN relationship features; among them, the node features include the obstacle motion state and the obstacle type; the edge features include the relative distance, speed difference, and heading angle between obstacles; An attention mechanism module, including 4 multi-head self-attention layers, which is used to fuse the LSTM temporal features and GNN relationship features using the Transformer decoder, generate attention weights through learnable parameters, and generate a fused feature representation; The prediction output layer includes a trajectory prediction branch and a risk assessment branch. The trajectory prediction branch is used to output the position probability distribution of the ship within a preset circumferential distance in the future preset time period based on the fused feature representation using a fully connected layer. The risk assessment branch is used to output the collision probability through a sigmoid activation function according to the position probability distribution of the ship within a preset circumferential distance in the future preset time period.

7. The ship response system based on AI large model data fusion according to claim 6, characterized in that, A training method for training the obstacle trajectory prediction model by adopting a curriculum learning paradigm, first pre-training a double-layer LSTM structure, and then gradually adding a graph neural network and an attention mechanism module, includes: Obtain the historical trajectory of obstacles, the ocean current influence coefficient, the obstacle type, the interaction between obstacles, and the course change rate in the target sea area; Only activate the double-layer LSTM network, and use the training set including the historical trajectory of obstacles and the ocean current influence coefficient to process the trajectory sequence of the obstacle situation data to generate LSTM time series features; Then activate the graph neural network, and use the training set including the obstacle type and the interaction between obstacles, and adopt the GraphSAGE algorithm to perform 3-order neighbor information aggregation to generate an obstacle relationship feature matrix, and generate GNN relationship features; Then activate the attention mechanism module, use the training set including the course change rate, use the Transformer decoder to fuse the LSTM time series features and the GNN relationship features, and generate attention weights through learnable parameters to generate a fused feature representation; Then activate the prediction output layer to output the position probability distribution and the collision probability of the ship within a preset circumferential distance in the future preset time period; Among them, the learning rate is decayed in stages, the message passing order of the graph neural network is gradually increased from 1 order to 3 orders, the number of attention heads in the attention mechanism module is gradually increased from 4 to 8, and backpropagation training is performed according to the loss function including the trajectory prediction error, the collision probability prediction error, and the model complexity penalty to obtain the trained obstacle trajectory prediction model.

8. The ship response system based on AI large model data fusion according to claim 7, characterized in that, The loss function is expressed as: ； In the formula, represents the loss function, , and respectively represent the trajectory prediction error , the collision probability prediction error and the model complexity penalty weight coefficients, represents the total number of training samples, represents the th true value of the trajectory prediction error of the training sample, represents the th predicted value of the trajectory prediction error of the training sample, represents the th true value of the collision probability prediction error of the training sample, represents the th predicted value of the collision probability prediction error of the training sample, represents the logarithmic function, represents the regularization coefficient, represents the number of model parameters, represents the th model parameter.

Images