Intelligent ship navigation method, device and equipment and storage medium

Abstract

The invention provides an intelligent ship navigation method, device and equipment and a storage medium. The method is applied to an intelligent ship navigation system comprising a dynamic environment modeling module and a deep reinforcement learning model. The method comprises the following steps: the dynamic environment modeling module obtains wind speed, wave height and ocean current speed data, and calculates a threat index in combination with a ship structure and a loading state; the deep reinforcement learning model converts the threat index into an environmental risk entropy value, the environmental risk entropy value, the relative azimuth angle, the DCPA and the TCPA form a state space, a rule compliance indicator is set in a reward function, and an action conforming to the COLREGs rule is stimulated; and finally, the model outputs a control instruction sequence of the course correction delta psi and the speed correction delta V according to the information. According to the method, the adaptability, the real-time performance and the precision of navigation under complex sea conditions are remarkably improved.

Description

Technical Field

[0001] This application belongs to the field of navigation, and in particular relates to a ship intelligent navigation method, device, equipment and storage medium. Background Technology

[0002] Ship navigation primarily relies on satellite navigation systems, especially the Global Positioning System (GPS), which provides high-precision positioning services globally via satellite signals, enabling ships to achieve relatively reliable positioning in simple environments such as open seas. Meanwhile, electronic charts (ENCs), as an important navigation tool, digitize chart information and, through the Electronic Chart Display and Information System (ECDIS), allow crew members to intuitively view the ship's position, route, surrounding geographical environment, and nautical information, such as ports, shoals, and reefs, thereby assisting in the monitoring of the route and navigation status. Furthermore, the Automatic Identification System (AIS) enhances navigational safety, especially in densely populated areas, by exchanging information such as the identity, position, course, and speed of ships.

[0003] However, traditional ship navigation and path planning technologies have revealed numerous shortcomings when facing complex marine environments. In adverse weather conditions, such as typhoons, heavy rain, and dense fog, satellite signals are easily interfered with, leading to decreased positioning accuracy or even signal loss, increasing navigational risks. In areas with dense traffic flow, the simultaneous navigation of numerous vessels makes information exchange and conflict avoidance extremely complex. Traditional AIS and rule-based collision avoidance methods struggle to handle these situations quickly and effectively, easily leading to collisions. Furthermore, with the ever-increasing volume of data generated during ship navigation, including sensor data, meteorological data, and traffic flow data, traditional data processing methods are inefficient and unable to effectively analyze and utilize this data in a timely manner, making it difficult for ships to make rapid and appropriate decisions in the face of emergencies.

[0004] In path planning, traditional methods are mainly based on rules or simple optimization algorithms. Rule-based path planning relies on pre-set navigation rules and experience to plan paths. While it can quickly generate feasible paths in simple environments, it lacks flexibility and struggles to adapt to complex and ever-changing marine environments. Path planning based on simple optimization algorithms, such as Dijkstra's algorithm, searches and calculates with the shortest path or lowest cost as the objective, but it suffers from low computational efficiency when dealing with large-scale data and complex constraints, and its responsiveness to real-time changes in the marine environment is insufficient. Furthermore, these methods are not comprehensive enough in considering ship maneuvering characteristics and dynamic marine environmental factors, and the planned paths may not be optimal, failing to maximize navigation efficiency. Summary of the Invention

[0005] The purpose of this application is to overcome the deficiencies in the prior art and provide a ship intelligent navigation method, device, equipment and storage medium.

[0006] This application provides a ship intelligent navigation method applied to a ship intelligent navigation system, the ship intelligent navigation system including a dynamic environment modeling module and a deep reinforcement learning model, the method comprising:

[0007] The dynamic environment modeling module acquires wind speed, wave height, and ocean current speed data from meteorological data sources, and calculates a threat index based on the wind speed, wave height, and ocean current speed data and a pre-set weighting coefficient, wherein the weighting coefficient is determined based on the ship's structure and load status.

[0008] The deep reinforcement learning model processes the threat index into an environmental risk entropy value and inputs it into the state space, which also includes relative azimuth, DCPA, and TCPA.

[0009] The deep reinforcement learning model determines a rule compliance indicator in the reward function, and the rule compliance indicator is used to incentivize actions that conform to COLREGs rules;

[0010] Based on the environmental risk entropy value, the rule compliance indicator, the relative azimuth angle, the DCPA, and the TCPA, the deep reinforcement learning model outputs a sequence of control commands for heading correction Δψ and speed correction ΔV.

[0011] Optionally, the dynamic environment modeling module also performs:

[0012] Extracting static obstacle outlines from electronic nautical charts;

[0013] The static obstacle profile is subjected to morphological dilation to generate dilated obstacle data, wherein the dilation radius is calculated based on the total length of the ship and the minimum safe maneuver margin, which is obtained by inverse calculation through the ship's maneuver response surface.

[0014] The expanded obstacle data is integrated into the dynamic navigation situation map.

[0015] Optionally, the deep reinforcement learning model processes the threat index into an environmental risk entropy value and inputs it into the state space, which further includes relative azimuth, DCPA, and TCPA, including:

[0016] The DCPA and TCPA are obtained by predicting the trajectories of other ships using an improved Social-LSTM model. The improved Social-LSTM model inputs AIS trajectory points from the past time period into a bidirectional LSTM encoder, extracts time and interaction features, and outputs a predicted trajectory probability density map for the future time period through a decoder. The DCPA and TCPA are then calculated based on the predicted trajectory probability density map.

[0017] Optionally, calculating the DCPA and TCPA based on the predicted trajectory probability density map includes:

[0018] The uncertainty is quantified by adding a covariance ellipse, where the length of the major axis of the covariance ellipse is calculated based on the target ship speed, prediction duration, and standard deviation of historical course changes. This ellipse is used to construct a dynamic obstacle collision probability field and introduces a probability correction factor to the DCPA and TCPA.

[0019] Optionally, the deep reinforcement learning model processes the threat index into an environmental risk entropy value and inputs it into the state space, including:

[0020] The dynamic environment modeling module generates a dynamic navigation situation map, and a convolutional autoencoder is used to compress the dynamic navigation situation map into a feature vector. The convolutional autoencoder includes multiple layers of two-dimensional convolution operations and fully connected layers. The feature vector is input into the state space.

[0021] Optionally, the deep reinforcement learning model includes:

[0022] A multi-path feature extraction structure is adopted, including a spatial feature path and a temporal feature path. The spatial feature path uses a residual network to extract multi-scale spatial features, while the temporal feature path uses a ConvLSTM module to process the situation map at continuous time points to model the temporal evolution features. The output features are concatenated and then input into a fully connected network to generate a state representation vector.

[0023] Optionally, based on the environmental risk entropy value, the rule compliance indicator, the relative azimuth angle, the DCPA, and the TCPA, the deep reinforcement learning model outputs a sequence of control commands for heading correction Δψ and speed correction ΔV, including:

[0024] The system calculates the conflict risk index, environmental mutation index, and path deviation in real time, and triggers soft or hard replanning based on the calculated values ​​in two levels. The conflict risk index is calculated based on TCPA and DCPA, the environmental mutation index is calculated by comparing the environmental risk layer status over a continuous period, and the path deviation is calculated based on the deviation between the actual location and the planned path.

[0025] This application also provides a ship intelligent navigation device applied to a ship intelligent navigation system, the ship intelligent navigation system including a dynamic environment modeling module and a deep reinforcement learning model, the method including:

[0026] The acquisition module obtains wind speed, wave height, and ocean current speed data from meteorological data sources through the dynamic environment modeling module, and calculates a threat index based on the wind speed, wave height, and ocean current speed data and a pre-set weighting coefficient, wherein the weighting coefficient is determined based on the ship's structure and load status.

[0027] The input module is used by the deep reinforcement learning model to process the threat index into an environmental risk entropy value and input it into the state space, which also includes relative azimuth, DCPA and TCPA.

[0028] The rules module is defined by the deep reinforcement learning model in the reward function as a rule compliance indicator, which is used to incentivize actions that conform to COLREGs rules.

[0029] The execution module, based on the environmental risk entropy value, the rule compliance indicator, the relative azimuth angle, the DCPA, and the TCPA, outputs a sequence of control commands for heading correction Δψ and speed correction ΔV from the deep reinforcement learning model.

[0030] This application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0031] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the above-described method.

[0032] The beneficial effects of this application are:

[0033] This application provides a ship intelligent navigation method applied to a ship intelligent navigation system. The ship intelligent navigation system includes a dynamic environment modeling module and a deep reinforcement learning model. The method includes: the dynamic environment modeling module acquiring wind speed, wave height, and ocean current speed data from a meteorological data source, and calculating a threat index based on the wind speed, wave height, and ocean current speed data and a pre-set weighting coefficient, wherein the weighting coefficient is determined based on the ship's structure and load status; the deep reinforcement learning model processing the threat index into an environmental risk entropy value and inputting it into a state space, the state space also including relative bearing angle, DCPA, and TCPA; the deep reinforcement learning model determining a rule compliance indicator in a reward function, the rule compliance indicator being used to incentivize actions that conform to COLREGs rules; and the deep reinforcement learning model outputting a sequence of maneuvering commands for heading correction Δψ and speed correction ΔV based on the environmental risk entropy value, the rule compliance indicator, the relative bearing angle, the DCPA, and the TCPA. This application significantly improves the adaptability, real-time performance, and accuracy of navigation under complex sea conditions through dynamic environment modeling and deep reinforcement learning, achieving compliant, efficient, and robust path planning and collision avoidance decision-making. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the intelligent navigation process for ships in this application;

[0035] Figure 2 This is a schematic diagram of the dynamic navigation situation diagram construction process in this application;

[0036] Figure 3 This is a situation grid diagram with the ship as the reference center in this application;

[0037] Figure 4 This is a schematic diagram of the deep reinforcement learning model architecture in this application;

[0038] Figure 5 This is a schematic diagram of the ship intelligent navigation device in this application. Detailed Implementation

[0039] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is to be understood that various forms of implementation of the present disclosure are intended and should not be limited to the embodiments set forth herein. Rather, the embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0040] Please refer to Figure 1As shown, this application provides a ship intelligent navigation method, applied in the field of ship intelligent navigation, to solve problems such as insufficient ship navigation adaptability in complex marine environments, real-time lag in multi-source data processing, insufficient integration of path planning with ship maneuvering characteristics and dynamic environmental factors, and weak system robustness. The method includes:

[0041] S101. The dynamic environment modeling module obtains wind speed, wave height and ocean current speed data from meteorological data sources, and calculates the threat index based on the wind speed, wave height and ocean current speed data and a pre-set weighting coefficient, wherein the weighting coefficient is determined based on the ship structure and load status.

[0042] Please refer to Figure 2 As shown, the dynamic environment modeling module first constructs a spatiotemporal joint database through a multi-source heterogeneous data fusion step, adopts a distributed data lake architecture for centralized storage management, deploys a Kafka-based stream processing engine, and configures four types of theme channels in the data access layer: static waterway data channel, dynamic ship data channel, meteorological and hydrological data channel, and ship performance data channel.

[0043] In the static channel data channel, vector layers containing water depth, reefs, channel boundaries and restricted areas are periodically extracted from electronic charts (ENC). Topological consistency checks are performed and duplicate or overlapping segments are removed. Then, the data is uniformly converted to GeoJSON format to support subsequent geospatial indexing.

[0044] The dynamic ship data channel, through access to the AIS receiving module, parses the MMSI, latitude and longitude, heading COG, speed SOG and bow orientation in the standard AIS message, constructs the ship status matrix and writes it to the database with an update cycle of 10 seconds.

[0045] The meteorological and hydrological data channel calls the WMTS standard interface to synchronously acquire wind field, wave field, visibility and ocean current vector data from the global high-resolution grid forecast system. All meteorological elements are interpolated to a unified spatial grid coordinate system and timestamps are added to build a four-dimensional spatiotemporal query index.

[0046] The ship performance data channel interfaces with the shipboard PLC system to read the main engine speed n, rudder angle δ, and draft d in real time from the main engine control unit.

[0047] All data is organized using timestamps and geographic coordinates as primary keys, stored in a NoSQL database that supports spatial indexing and time-sliding window retrieval, and a multi-dimensional composite index structure is constructed to support real-time calls to subsequent navigation situation modeling and decision optimization modules.

[0048] To achieve efficient management and querying of multi-source spatiotemporal data, a dual indexing strategy is introduced when constructing the spatiotemporal joint database. Spatially, an R* tree index is built based on the PostGIS extension module for static and dynamic geographic data in GeoJSON format. This index uses a spatial bounding box structure to enable rapid geographic proximity retrieval of waterway obstacles, meteorological grids, and ship status. Temporally, a Log-StructuredMergeTree (LSM tree) index is built for the UTC timestamp of each data record type to support large-scale continuous time series data writing and sliding window update operations.

[0049] To improve the continuity and accuracy of ship status data during AIS message gaps, a multi-source fusion Kalman filter method is adopted to jointly process multiple positioning sources such as AIS, inertial navigation IMU, and radar target tracking, and a state transition model is constructed as follows:

[0050]

[0051] in, Let be the state prediction vector at time k, containing estimates of latitude and longitude, heading ψ, and velocity V. Let A be the state transition matrix and B be the control input matrix. Enter the command for the previous operation.

[0052] In the event of AIS signal loss, the ship's trajectory is completed through continuous interpolation to maintain the integrity and continuity of the situation map. The system also integrates a data quality assessment module to dynamically monitor the real-time performance and integrity of key data sources. When the latest timestamp of the meteorological data from the WMTS service lags behind the local clock by more than 1800 seconds, or when the AIS message reception rate is below 95% for five consecutive minutes, a data anomaly alarm mechanism is triggered. The alarm event is pushed to the monitoring console via Kafka, and the relevant data frames are marked as low confidence to prevent abnormal data from interfering with the stability of subsequent path planning and decision-making model outputs.

[0053] The dynamic environment modeling module obtains wind speed, wave height, and ocean current speed data from meteorological data sources and calculates the threat index based on this data.

[0054] Threat Index The calculation uses the following formula:

[0055]

[0056] in, For wind speed, For the high waves, For ocean current speed, , , The weighting factor is determined by the ship's structure and load condition.

[0057] These coefficients are dynamically adjusted based on ship type, tonnage, and load to accurately reflect the impact of meteorological and hydrological conditions on navigation risks. The calculated threat index is used to construct the subsequent environmental risk layer and, as part of the obstacle avoidance path cost function, participates in reinforcement learning action selection and model prediction path optimization.

[0058] In addition, the dynamic environment modeling module extracts static obstacle data from electronic nautical charts and performs morphological dilation processing on the static obstacle data. The dilation radius is calculated based on the ship's overall length and minimum safe maneuvering margin. Specifically, the dilation radius r is determined according to the formula... Calculation, wherein, the Given the current total length of the vessel, the stated The minimum safe control margin is obtained by inverse calculation based on the control response surface f(n, δ, d), where n is the main engine speed, δ is the rudder angle, and d is the draft.

[0059] Morphological dilation processing reflects the radius of influence of maneuver risk by dilating the outline of obstacles. The dilation result is written to the obstacle raster layer, and the crossing cost is set to infinity to prevent ships from getting too close to obstacles.

[0060] S102. The deep reinforcement learning model processes the threat index into an environmental risk entropy value and inputs it into the state space, which also includes relative azimuth, DCPA and TCPA.

[0061] The dynamic environment modeling module generates dynamic navigation situation diagrams.

[0062] Please refer to Figure 3 As shown, the dynamic navigation situation map is constructed by overlaying multiple layers to form a situation grid map with the ship as the reference center. The core area is a square with a side length of 2R, where R is three times the braking distance corresponding to the current speed, to ensure sufficient path assessment range and reaction time.

[0063] The area is divided into a uniform grid with a resolution of 0.1 nautical miles, forming the basic geographic layer. Each grid cell records water depth values ​​and seabed type codes to support path accessibility assessment and constraint adaptation of the maneuver response model.

[0064] In the construction of the static obstacle layer, the fixed obstacle contours extracted from ENC are subjected to morphological dilation to reflect their radius of influence of manipulation risk. The dilation result is written into the obstacle raster layer, and the crossing cost is set to infinity.

[0065] The dynamic target layer constructs a network for predicting the trajectory of other vessels based on an improved Social-LSTM model. The trajectory points of other vessels obtained by AIS parsing in the past 60 seconds are input into a bidirectional LSTM encoder to extract time-related speed and interaction features. The decoder outputs a two-dimensional probability density map of the predicted trajectory for the next 120 seconds and maps it to the corresponding grid area. Each grid contains a probability value for collision risk calculation.

[0066] The environmental risk layer calculates the threat index in each grid cell based on the current meteorological grid data.

[0067] Data at each layer is aligned with a unified timestamp and indexed by spatial grid numbers, enabling quick access to situational maps at any time step and sliding window updates; when the ship moves more than 50% of the grid size or a new obstacle enters the monitoring area, local grid reconstruction is triggered.

[0068] In the dynamic environment modeling module, data association processing is first performed, spatially matching the point cloud data collected by the shipborne radar with the target status broadcast by AIS. By constructing a cost matrix based on the difference between the target's spatial location distance and heading, the Hungarian algorithm is used to achieve optimal allocation, obtaining a one-to-one correspondence between the radar and AIS at the current moment. Radar points that are not successfully matched are marked as unknown obstacles and dynamically added to the obstacle layer, while their trajectory tracking queue is initialized for subsequent inter-frame consistency judgment.

[0069] After predicting the trajectory of the moving target, to quantify its uncertainty, a covariance ellipse is attached to each future location point on the predicted trajectory, and the length of its major axis is calculated as follows: Where v is the target ship's current speed and t is the prediction time. The standard deviation of the historical heading changes is used to construct the dynamic obstacle collision probability field, and probability correction factors are introduced into the DCPA and TCPA in the reinforcement learning input state space to improve decision robustness.

[0070] In the situation map dimensionality reduction stage, in order to compress the dynamically constructed multi-layer raster map and reduce the model call latency, a convolutional autoencoder structure is introduced to encode the original spatial raster map into a 256-dimensional dense feature vector. The encoder part includes 4 layers of two-dimensional convolution operations, each with a kernel size of 3x3 and a stride of 2, and connects 2 fully connected layers for feature integration, outputting a unified state vector representation for reinforcement learning models and MPC path optimization modules.

[0071] To ensure that the situational awareness range covers the entire ship braking zone, the system monitors whether the current ship position is approaching the boundary of the current 2R×2R grid map. When the distance to the boundary is less than 0.2R, the map expansion mechanism is automatically triggered. GeoJSON data of adjacent sea areas is preloaded through the local nautical chart tile index and the grid map is stitched together as needed to ensure the continuity and integrity of environmental modeling.

[0072] The deep reinforcement learning model processes the threat index into an environmental risk entropy value and inputs it into the state space, which also includes the relative azimuth, DCPA, and TCPA. DCPA and TCPA are obtained by predicting the trajectories of other vessels using an improved Social-LSTM model.

[0073] S103. The deep reinforcement learning model determines a rule compliance indicator in the reward function, and the rule compliance indicator is used to incentivize actions that conform to COLREGs rules.

[0074] Deep reinforcement learning models determine rule compliance indicators in the reward function. The reward function adopts a weighted multi-objective form, providing comprehensive feedback on the model's behavior at each time step t.

[0075] The total reward function is:

[0076]

[0077] in, The collision probability is calculated based on the overlap between the trajectory probability distribution output by the dynamic obstacle prediction layer and the predicted path of the ship. Δψ represents the change in the Euclidean distance difference between the current position and the target point, used to measure progress; Δψ represents the course adjustment at the current moment; and ΔV represents the change in speed. This is a rule compliance indicator. A value of 1 indicates that the action meets the COLREGs rule, while a value of 0 indicates that it does not.

[0078] This reward function incentivizes path progression and rule adherence by penalizing collision and manipulation complexity, ensuring that the policy network can both avoid risks and efficiently advance to the goal during training.

[0079] Please refer to Figure 4As shown, the deep reinforcement learning model is designed based on a multi-path feature extraction structure to enhance the joint understanding of spatial patterns and temporal evolution in dynamic situation maps. The input layer receives a four-channel tensor with dimensions of 4×H×W, corresponding to the geographic information layer, static obstacle layer, dynamic target layer, and environmental risk layer, respectively. The spatial feature path adopts a three-layer residual network structure, containing 16, 32, and 64 convolutional kernels respectively, with a kernel size of 3×3. A downsampling operation with a stride of 2 is used to compress the spatial size of the input tensor to 1 / 8 of the original size, while preserving multi-scale spatial boundary features and deep semantic relationships between grids. The parallel temporal feature path uses a ConvLSTM module to process dynamic situation maps at three consecutive time points. The ConvLSTM contains 64 hidden units, explicitly modeling the target movement trend and risk disturbance propagation path in the grid map over time, obtaining long-term dependency features in the state sequence. The two-dimensional feature map output by the spatial path is converted into a one-dimensional vector by global average pooling, and then concatenated with the output feature vector of the temporal path. This concatenation is then fed into a two-layer fully connected network, each layer containing 512 neurons and using the ReLU activation function. Finally, a state representation vector s is generated, which serves as the input for subsequent policy generation and value estimation.

[0080] The strategy output layer consists of two parallel control heads. The heading control head outputs a 21-dimensional discrete probability distribution, with the heading correction Δψ ranging from [-30°, -27°, ..., +30°] and a step size of 3 degrees. The speed control head outputs an 11-dimensional discrete distribution, with the speed correction ΔV ranging from [-5, -4, ..., +5] knots and a step size of 1 knot. Each control head generates action probabilities using a softmax function. , which serves as the policy distribution function in the PPO algorithm. The model outputs the state-value function V(st) in parallel, which is used for advantage function estimation in proximal policy optimization.

[0081] During the training of the deep reinforcement learning model, a high-risk interaction scenario library is first constructed using historical AIS data to improve the model's adaptability to complex navigation situations. The scenario generation module automatically selects three typical scenarios from global AIS records: cross-encounter scenarios refer to samples where two ships have a DCPA of less than 0.5 nautical miles in the predicted trajectory; multi-ship encirclement scenarios refer to samples where at least three ships are clustered within a 1-nautical-mile radius, used to train the model's ability to coordinate avoidance of dense traffic; extreme weather scenarios extract data from navigation segments with wind speeds greater than level 7 or visibility less than 1 nautical mile, which are then mapped to the situation map through an environmental risk layer to enhance the model's robustness under low visibility conditions.

[0082] The model training employs a course-based learning strategy, progressively enhancing the policy's generalization ability in three stages based on difficulty. The first stage involves single-ship obstacle avoidance training, with the environment containing only static obstacles and basic water depth constraints, totaling one million training steps to strengthen the model's basic path planning capabilities. The second stage introduces a two-ship interaction scenario, where the policy network outputs real-time verification of compliance with COLREGs navigation rules via a rule engine, returning strong penalty signals for non-compliant actions, totaling three million training steps. The third stage expands the state encoder structure in a multi-ship interaction environment, introducing a graph neural network to handle the mutual influence of multiple objectives, and training for five million steps to improve the coordination and stability of decision-making.

[0083] The training optimization uses the PPO algorithm, and its loss function is determined as follows:

[0084]

[0085]

[0086] in, For expectation operator, This represents the probability ratio between the current strategy and the old strategy. `clip` is the dominant function, and `clip` is the clipping function. This is a shearing factor to prevent the strategy from being updated too quickly. and For both old and new strategies.

[0087] The final loss function is a weighted combination of value loss, policy loss, and entropy regularization term, with the optimization objective balancing exploration efficiency and policy stability.

[0088] S104. Based on the environmental risk entropy value, the rule compliance indicator, the relative azimuth angle, the DCPA, and the TCPA, the deep reinforcement learning model outputs a sequence of control commands for heading correction Δψ and speed correction ΔV.

[0089] The deep reinforcement learning model is based on state-space input, including environmental risk entropy, rule compliance indicator, relative bearing angle, DCPA and TCPA, and outputs a sequence of control commands for heading correction Δψ and speed correction ΔV through a policy network.

[0090] The sequence of maneuvering commands is used in the rolling time-domain optimal path generation step. Based on the Model Predictive Control (MPC) framework, with the dual objective functions of minimizing energy consumption and ETA error, the optimal path point sequence within the future time window T is solved in a rolling manner to generate a smooth and executable ship navigation path. The ship's motion state is modeled using discrete-time state transition equations, considering the influence of current speed and heading on displacement. The state transition model is as follows:

[0091]

[0092] in, and Let be the coordinates of the ship's position at time k. For heading angle, For speed, Let θ be the rudder angle, L be the overall length of the hull, and Δt be the distance from the rudder.

[0093] Under this model, the state trajectory and control input for the next N steps are jointly optimized, and the objective function is determined as follows:

[0094]

[0095] in, For the predicted position of the ship at step k, As a reference point for smooth path, To represent the predicted speed of the ship at time step k, To recommend an energy-saving cruising speed, , , The weights for the loss function correspond to path tracking accuracy, energy efficiency deviation, and handling smoothness indices, respectively. Let k and k-1 represent the control variables at time step k. The objective function minimizes position deviation, velocity deviation, and control fluctuations through weighted minimization, ensuring a balance between energy efficiency, safety, and comfort in the generated path.

[0096] Three types of constraints are set during the rolling optimization process:

[0097] The first constraint is dynamics, which limits the magnitude of single-step rudder angle and velocity changes, satisfying... and This ensures that control commands can be reliably executed by the ship's response system within the limits of its hardware capabilities.

[0098] The second constraint is safety, which requires that the minimum distance from any predicted path point to an obstacle be greater than a safety threshold. ,Right now ,in Obstacle grid map derived from the dynamic environment modeling module.

[0099] The third constraint is time constraints. To address the scheduling needs of operations such as berthing or gate clearance, a soft ETA limit is introduced, which restricts the estimated arrival time (ETAk) for each step from exceeding the port's allowed time window. This is to improve scheduling consistency across the entire flight segment.

[0100] The model predictive control solution process re-executes the complete optimization process within each roll control cycle to adapt to changes in the current navigation state and environmental disturbances. First, in the initialization phase, the first M action pairs are extracted from the control command sequence output by the previous deep reinforcement learning module as the initial solution, including the rudder angle δ and the speed change ΔV. These are substituted into the state equation to predict the initial trajectory sequence, which serves as the starting point for roll optimization. In the numerical optimization process, a sequential quadratic programming algorithm is used for solving the problem. First, at the current operating point... The nonlinear objective function and constraint function are expanded using a first-order Taylor series and approximated as linear and quadratic forms, respectively, to construct a quadratic programming subproblem:

[0101]

[0102] in, The increment of the current control quantity, The Hessian matrix approximation for the objective function. It is the first-order gradient vector. Let Jacobian matrix be the constraint function. is the inequality constraint bound, and st means "constrained by".

[0103] The QP subproblem is solved using the interior-point method, which iteratively approximates the optimum within the feasible region, guaranteeing convergence to a local optimum within a finite number of steps. After determining the search direction Δz, an Armijo line search strategy is used to determine the step size factor, ensuring that each update is in the direction of descent of the objective function and satisfies the sufficient descent condition, thereby enhancing numerical stability.

[0104] To accelerate the iteration process and improve online control efficiency, the system implements a hot-start mechanism. In each control cycle, steps 2 to T of the previously optimized trajectory sequence are directly used as the initial guess values ​​for the current cycle, reducing initial iteration deviations and improving solution convergence speed. Regarding real-time performance, a hard execution cycle limit is set. When a solution time exceeds 0.5 seconds, the system automatically switches to a reduced-order model (ROM) for solving. This model extracts the dominant modes from the ship's complete kinematics model, retaining only key state variables and control channels, significantly compressing the state space dimension and reducing the computational load of the Jacobian and Hessian matrices. This ensures that the path planning system maintains stable control command output capabilities even under high traffic density or complex situations.

[0105] The dynamic replanning triggering mechanism is used to ensure the vessel's autonomous adaptability in the face of sudden risks, drastic environmental changes, or abnormal route execution. The system calculates three key indicators in real time during each control cycle to monitor route execution quality and navigation safety boundaries. The first indicator is the conflict risk index Rcoll, which is determined to be...

[0106]

[0107] in, For the closest point time, The distance to the nearest point. The set reference safety distance, A joint risk assessment reflecting the time urgency and spatial proximity between the vessel and a potential collision target.

[0108] The second item is the Environmental Sudden Change Index (Evar), determined by comparing the environmental risk layer status over two consecutive control periods.

[0109]

[0110] in, and These are the meteorological threat index vectors for each grid in the current and previous period's situation maps, respectively. The Euclidean norm represents the overall disturbance change in a meteorological field and is used to detect situations such as strong winds, sudden flow fields, or drastic changes in visibility.

[0111] The third item is path deviation. The maximum deviation between the actual location sequence and the optimal planned path is calculated and expressed as:

[0112]

[0113] in, These are the waypoints obtained through real-time positioning. The expected point in the MPC path sequence is the deviation. If the deviation exceeds a certain proportion, it indicates that the current path is no longer suitable for the environment or that the accumulated control error is too large.

[0114] Based on the monitoring results, the system adjusts the path according to a two-level triggering logic.

[0115] Level 1 triggering is soft replanning, when or When initiated, the existing path is not cleared; only the rolling time domain of MPC is shortened to improve response frequency, while short-term trajectories are replanned within the original path framework. Secondary triggering is a hard replanning, when... or If the system starts up at a certain time, it indicates that the current path no longer has safety or stability in the new environment. The system clears all future path points and returns to step S102 to construct the latest dynamic navigation situation map and path input features.

[0116] To ensure logical continuity between the replanning results and the original environmental state, the system saves a snapshot of the current situation map and estimated state variables before triggering replanning, serving as a baseline for subsequent path evaluation and guaranteeing scenario consistency and smooth model transition during path switching. This mechanism effectively improves the adaptability and robustness of the path planning system under highly dynamic sea conditions and high-density traffic conditions through multi-indicator fusion judgment and a hierarchical response strategy.

[0117] Please refer to Figure 5 As shown, this application also provides a ship intelligent navigation device applied to a ship intelligent navigation system, the ship intelligent navigation system including a dynamic environment modeling module and a deep reinforcement learning model, the method including:

[0118] The acquisition module 201 acquires wind speed, wave height and ocean current speed data from meteorological data sources by the dynamic environment modeling module, and calculates a threat index based on the wind speed, wave height and ocean current speed data and a pre-set weighting coefficient, wherein the weighting coefficient is determined based on the ship structure and load status.

[0119] Input module 202, the deep reinforcement learning model processes the threat index into an environmental risk entropy value and inputs it into the state space, the state space also includes relative azimuth angle, DCPA and TCPA;

[0120] Rule module 203, in which the deep reinforcement learning model determines a rule compliance indicator in the reward function, the rule compliance indicator being used to incentivize actions that conform to COLREGs rules;

[0121] The execution module 204, based on the environmental risk entropy value, the rule compliance indicator, the relative azimuth angle, the DCPA, and the TCPA, outputs a sequence of control commands for heading correction Δψ and speed correction ΔV from the deep reinforcement learning model.

[0122] This application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0123] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the above-described method.

[0124] The above description of the embodiments is provided to enable those skilled in the art to understand and apply this application. Those skilled in the art will readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without inventive effort. Therefore, this application is not limited to the above embodiments, and any improvements and modifications made to this application based on the disclosure thereof should be within the scope of protection of this application.

Claims

1. A ship intelligent navigation method, characterized in that, Applied to a ship intelligent navigation system, the ship intelligent navigation system including a dynamic environment modeling module and a deep reinforcement learning model, the method includes: The dynamic environment modeling module acquires wind speed, wave height, and ocean current speed data from meteorological data sources, and calculates a threat index based on the wind speed, wave height, and ocean current speed data and a pre-set weighting coefficient, wherein the weighting coefficient is determined based on the ship's structure and load status. The deep reinforcement learning model processes the threat index into an environmental risk entropy value and inputs it into the state space, which also includes relative azimuth, DCPA, and TCPA. The deep reinforcement learning model determines a rule compliance indicator in the reward function, and the rule compliance indicator is used to incentivize actions that conform to COLREGs rules; Based on the environmental risk entropy value, the rule compliance indicator, the relative azimuth angle, the DCPA, and the TCPA, the deep reinforcement learning model outputs a sequence of control commands for heading correction Δψ and speed correction ΔV.

2. The method according to claim 1, characterized in that, The dynamic environment modeling module also performs: Extracting static obstacle outlines from electronic nautical charts; The static obstacle profile is subjected to morphological dilation to generate dilated obstacle data, wherein the dilation radius is calculated based on the total length of the ship and the minimum safe maneuver margin, which is obtained by inverse calculation through the ship's maneuver response surface. The expanded obstacle data is integrated into the dynamic navigation situation map.

3. The method according to claim 1, characterized in that, The deep reinforcement learning model processes the threat index into an environmental risk entropy value and inputs it into the state space, which also includes relative azimuth, DCPA, and TCPA, including: The DCPA and TCPA are obtained by predicting the trajectories of other ships using an improved Social-LSTM model. The improved Social-LSTM model inputs AIS trajectory points from the past time period into a bidirectional LSTM encoder, extracts time and interaction features, and outputs a predicted trajectory probability density map for the future time period through a decoder. The DCPA and TCPA are then calculated based on the predicted trajectory probability density map.

4. The method according to claim 3, characterized in that, Calculating the DCPA and TCPA based on the predicted trajectory probability density map includes: The uncertainty is quantified by adding a covariance ellipse, where the length of the major axis of the covariance ellipse is calculated based on the target ship speed, prediction duration, and standard deviation of historical course changes. This ellipse is used to construct a dynamic obstacle collision probability field and introduces a probability correction factor to the DCPA and TCPA.

5. The method according to claim 1, characterized in that, The deep reinforcement learning model processes the threat index into an environmental risk entropy value and inputs it into the state space, including: The dynamic environment modeling module generates a dynamic navigation situation map, and a convolutional autoencoder is used to compress the dynamic navigation situation map into a feature vector. The convolutional autoencoder includes multiple layers of two-dimensional convolution operations and fully connected layers. The feature vector is input into the state space.

6. The method according to claim 1, characterized in that, The deep reinforcement learning model includes: A multi-path feature extraction structure is adopted, including a spatial feature path and a temporal feature path. The spatial feature path uses a residual network to extract multi-scale spatial features, while the temporal feature path uses a ConvLSTM module to process the situation map at continuous time points to model the temporal evolution features. The output features are concatenated and then input into a fully connected network to generate a state representation vector.

7. The method according to claim 1, characterized in that, Based on the environmental risk entropy value, the rule compliance indicator, the relative azimuth angle, the DCPA, and the TCPA, the deep reinforcement learning model outputs a sequence of control commands for heading correction Δψ and speed correction ΔV, including: The system calculates the conflict risk index, environmental mutation index, and path deviation in real time, and triggers soft or hard replanning based on the calculated values ​​in two levels. The conflict risk index is calculated based on TCPA and DCPA, the environmental mutation index is calculated by comparing the environmental risk layer status over a continuous period, and the path deviation is calculated based on the deviation between the actual location and the planned path.

8. A ship intelligent navigation device, characterized in that, Applied to a ship intelligent navigation system, the ship intelligent navigation system includes a dynamic environment modeling module and a deep reinforcement learning model, the device includes: The acquisition module obtains wind speed, wave height, and ocean current speed data from meteorological data sources through the dynamic environment modeling module, and calculates a threat index based on the wind speed, wave height, and ocean current speed data and a pre-set weighting coefficient, wherein the weighting coefficient is determined based on the ship's structure and load status. The input module is used by the deep reinforcement learning model to process the threat index into an environmental risk entropy value and input it into the state space, which also includes relative azimuth, DCPA and TCPA. The rules module is defined by the deep reinforcement learning model in the reward function as a rule compliance indicator, which is used to incentivize actions that conform to COLREGs rules. The execution module, based on the environmental risk entropy value, the rule compliance indicator, the relative azimuth angle, the DCPA, and the TCPA, outputs a sequence of control commands for heading correction Δψ and speed correction ΔV from the deep reinforcement learning model.

9. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed in a computer, causes the computer to perform the method described in any one of claims 1-7.

Images