A method and system for intelligent decision-making in ship navigation

Abstract

This invention discloses an intelligent decision-making method and system for ship navigation. The method includes: forming a navigation element dataset through spatiotemporal preprocessing and confidence weighting; constructing a continuous three-dimensional waterway environment field using a spatiotemporally joint Gaussian probability mapping model to generate a dedicated continuous navigation risk field; outputting path and speed decisions using a deep reinforcement learning multi-objective optimization model, and constructing a joint loss function based on actual navigation observation data and execution deviations, updating the parameters of the deep reinforcement learning multi-objective optimization model online, thereby achieving rolling updates of the continuous three-dimensional waterway environment field, the dedicated continuous navigation risk field, and the navigation decision scheme. The technical solution of this invention improves the accuracy of waterway environment modeling through multi-ship collaborative perception, confidence weighting, and spatiotemporal Gaussian mapping; achieves joint path and speed decision-making by combining ship-specific constraints and reinforcement learning optimization; and enhances decision-making security, real-time performance, and adaptability through online updating of the joint loss function.

Description

Technical Field

[0001] This invention relates to the technical field of navigation decision-making, and in particular to an intelligent decision-making method and system for ship navigation. Background Technology

[0002] With the increasing intelligence of inland waterway and coastal shipping, the growing size of ships, and the continuous improvement of waterway traffic density, intelligent navigation and decision-making technologies for ships are gradually becoming an important technological direction for ensuring navigation safety and improving navigation efficiency. Existing ship navigation decision-making methods typically rely on shipboard sensors, electronic charts, AIS information, and preset navigation rules to identify the waterway environment and ship motion status, and generate navigation paths or collision avoidance suggestions accordingly.

[0003] However, complex hydrological and meteorological conditions, obstacle distribution, channel boundaries, and multi-vehicle interactions exhibit significant dynamism and uncertainty. Single-vehicle perception is prone to blind spots, and multi-vehicle collaborative data has not been fully integrated and utilized, making it difficult to accurately infer the spatiotemporal evolution of the channel environment. Furthermore, existing methods often employ unified decision-making models, which are ill-suited to differences in vessel type, size, draft, maneuverability, navigation mission, and safety level. They also lack multi-objective joint optimization capabilities, online error correction, and parameter iteration mechanisms, failing to adaptively correct for actual navigation deviations. Therefore, effectively addressing these issues has become a pressing problem for the industry. Summary of the Invention

[0004] This invention provides a method and system for intelligent decision-making in ship navigation, which integrates multi-ship collaborative perception with dedicated risk modeling to achieve path and speed optimization and online error correction, thereby improving navigation safety, real-time performance, and adaptive capabilities.

[0005] According to a first aspect of the present invention, a ship navigation intelligent decision-making method is provided, the ship navigation intelligent decision-making method comprising: Multi-source navigation element data of the target waterway area is obtained based on a multi-ship cooperative network. The multi-source navigation element data is preprocessed in time and space. Confidence weights are assigned according to the physical state indicators of the preprocessed multi-source navigation element data to obtain a navigation element dataset with confidence weights. Using the navigation element dataset with confidence weights as known samples, a spatiotemporal joint Gaussian probability mapping model is used to calculate the mean and uncertainty variance of the waterway environment at spatial grids and time nodes, thus obtaining a continuous three-dimensional waterway environment field. Based on the continuous three-dimensional waterway environment field and the individual attributes and real-time motion state of the target decision-making vessel, a dedicated dynamic constraint boundary is determined. The probability of exceeding the limits of the environmental prediction mean and uncertainty variance relative to the dedicated dynamic constraint boundary is calculated. The probability of exceeding the limits is normalized and weighted and fused to obtain a dedicated continuous navigation risk field. The exclusive continuous navigation risk field, the real-time position and speed of the target decision vessel are input into the state space into a preset deep reinforcement learning multi-objective optimization model, and the optimal navigation decision scheme including the navigation path and speed is output under the condition of satisfying the navigation physical constraints. The system acquires real-time environmental observation data and actual actions of the target decision-making vessel during its actual navigation process. It then compares the deviation of the real-time environmental observation data with the environmental prediction results of the corresponding spatiotemporal location in the continuous three-dimensional waterway environment field, and compares the deviation of the actual actions with the theoretically guided actions in the optimal navigation decision scheme. Based on the environmental perception error and decision execution error, it constructs a joint loss function and updates the parameters of the deep reinforcement learning multi-objective optimization model online through the joint loss function. Based on the updated parameters and the multi-source navigation element data incorporated at the next moment, it continuously updates the continuous three-dimensional waterway environment field and the dedicated continuous navigation risk field, and generates the navigation decision scheme for the next moment.

[0006] According to a second aspect of the present invention, a ship navigation intelligent decision-making system is provided, comprising: The acquisition module is used to acquire multi-source navigation element data of the target waterway area based on a multi-ship cooperative network, perform spatiotemporal preprocessing on the multi-source navigation element data, and allocate confidence weights according to the physical state indicators of the preprocessed multi-source navigation element data to obtain a navigation element dataset with confidence weight identifiers. The calculation module is used to calculate the mean and uncertainty variance of the waterway environment at the spatial grid and time nodes using the navigation element dataset with confidence weights as known samples and a spatiotemporal joint Gaussian probability mapping model, so as to obtain a continuous three-dimensional waterway environment field. The processing module is used to determine the exclusive dynamic constraint boundary based on the continuous three-dimensional waterway environment field and the individual attributes and real-time motion state of the target decision vessel, calculate the probability of exceeding the limit of the environmental prediction mean and uncertainty variance relative to the exclusive dynamic constraint boundary, and normalize and weightedly fuse the probability of exceeding the limit to obtain the exclusive continuous navigation risk field. The output module is used to input the exclusive continuous navigation risk field, the real-time position and speed of the target decision vessel as the state space into a preset deep reinforcement learning multi-objective optimization model, and output the optimal navigation decision scheme including the navigation path and speed under the condition of satisfying the navigation physical constraints. The update module is used to acquire real-time environmental observation data and actual actions of the target decision-making vessel during actual navigation. It compares the deviation of the real-time environmental observation data with the environmental prediction results of the corresponding spatiotemporal position in the continuous three-dimensional waterway environment field, and compares the deviation of the actual actions with the theoretical guidance actions in the optimal navigation decision scheme. It constructs a joint loss function based on environmental perception error and decision execution error, and updates the parameters of the deep reinforcement learning multi-objective optimization model online through the joint loss function. Based on the updated parameters and the multi-source navigation element data brought in at the next moment, it continuously updates the continuous three-dimensional waterway environment field and the dedicated continuous navigation risk field, and generates the navigation decision scheme for the next moment.

[0007] According to a third aspect of the present invention, an electronic device is provided, comprising: a communication interface, a processor, and a memory; The memory is used to store program instructions, which, when executed by the processor that is connected to the memory via the communication interface, implement any of the above-described intelligent ship navigation decision-making methods.

[0008] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored, which, when executed by a computer (e.g., a processor in a computer), implement any of the above-described intelligent decision-making methods for ship navigation.

[0009] In summary, this invention provides a method and system for intelligent decision-making in ship navigation. The method includes: acquiring multi-source navigation element data of a target waterway based on a multi-ship collaborative network; forming a navigation element dataset through spatiotemporal preprocessing and confidence weighting; constructing a continuous three-dimensional waterway environment field using a spatiotemporal joint Gaussian probability mapping model; and generating a dedicated continuous navigation risk field by combining the individual attributes and motion state of the target decision-making vessel; further inputting the risk field, position, and speed into a deep reinforcement learning multi-objective optimization model, outputting path and speed decisions, and constructing a joint loss function based on actual navigation observation data and execution deviations; updating the parameters of the deep reinforcement learning multi-objective optimization model online; and achieving rolling updates of the continuous three-dimensional waterway environment field, the dedicated continuous navigation risk field, and the navigation decision scheme. The technical solution of this invention improves the accuracy of waterway environment modeling through multi-ship collaborative perception, confidence weighting, and spatiotemporal Gaussian mapping; achieves joint path and speed decision-making by combining ship-specific constraints and reinforcement learning optimization; and enhances decision-making security, real-time performance, and adaptability through online updating of the joint loss function.

[0010] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and drawings.

[0011] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0012] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0013] Figure 1 A flowchart of a ship navigation intelligent decision-making method provided as an embodiment of the present invention; Figure 2 A flowchart of another intelligent decision-making method for ship navigation provided as an embodiment of the present invention; Figure 3 A flowchart of yet another intelligent decision-making method for ship navigation provided as an embodiment of the present invention; Figure 4 A flowchart of yet another intelligent decision-making method for ship navigation provided as an embodiment of the present invention; Figure 5 A flowchart of yet another intelligent decision-making method for ship navigation provided as an embodiment of the present invention; Figure 6 A flowchart of yet another intelligent decision-making method for ship navigation provided as an embodiment of the present invention; Figure 7 A structural diagram of a ship navigation intelligent decision-making system provided as an embodiment of the present invention; Figure 8 A structural diagram of an electronic device provided as an embodiment of the present invention; Figure 9 A schematic diagram illustrating ship perception and decision-making in a real navigation environment, provided for embodiments of the present invention; Figure 10 A schematic diagram of risk field partitioning provided for embodiments of the present invention. Detailed Implementation

[0014] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present invention and not to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present invention by illustrating examples of the invention.

[0015] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0016] like Figure 1 As shown, the present invention provides a ship navigation intelligent decision-making method, which includes: In step S11, multi-source navigation element data of the target waterway area is obtained based on the multi-ship cooperative network, the multi-source navigation element data is preprocessed in time and space, and confidence weights are assigned according to the physical state indicators of the preprocessed multi-source navigation element data to obtain a navigation element dataset with confidence weight identifiers. In step S12, the navigation element dataset with confidence weights is used as a known sample, and a spatiotemporal joint Gaussian probability mapping model is used to calculate the mean and uncertainty variance of the waterway environment at the spatial grid and time nodes to obtain a continuous three-dimensional waterway environment field. In step S13, a dedicated dynamic constraint boundary is determined based on the continuous three-dimensional waterway environment field and the individual attributes and real-time motion state of the target decision vessel. The probability of exceeding the limits of the environmental prediction mean and uncertainty variance relative to the dedicated dynamic constraint boundary is calculated, and the probability of exceeding the limits is normalized and weighted to obtain a dedicated continuous navigation risk field. In step S14, the exclusive continuous navigation risk field, the real-time position and speed of the target decision vessel are input as state space into a preset deep reinforcement learning multi-objective optimization model, and the optimal navigation decision scheme including the navigation path and speed is output under the condition of satisfying the navigation physical constraints. In step S15, real-time environmental observation data and actual actions of the target decision-making vessel during actual navigation are acquired. The deviation of the real-time environmental observation data is compared with the environmental prediction results of the corresponding spatiotemporal location in the continuous three-dimensional waterway environment field, and the deviation of the actual actions is compared with the theoretical guidance actions in the optimal navigation decision scheme. A joint loss function is constructed based on the environmental perception error and the decision execution error. The parameters of the deep reinforcement learning multi-objective optimization model are updated online through the joint loss function. Based on the updated parameters and the multi-source navigation element data incorporated at the next moment, the continuous three-dimensional waterway environment field and the dedicated continuous navigation risk field are updated on a rolling basis. A schematic diagram of the risk field partitioning is attached. Figure 10 As shown, it generates a navigation decision scheme for the next moment.

[0017] This embodiment provides an intelligent decision-making method for ship navigation, applicable to target waterway areas such as inland waterways, port areas, and near-shore waterways. Based on a multi-ship collaborative network, it acquires multi-source navigation element data for the target waterway area. This multi-source navigation element data includes the ship's real-time three-dimensional positioning coordinates, six-degree-of-freedom attitude data, real-time draft, speed, heading, main engine speed, rudder angle information, as well as water depth around the hull, water surface clearance, channel current velocity, current direction, obstacle distribution, visibility, local wind field, wave elements, channel boundaries, and navigation mark positions and status information. The multi-source navigation element data undergoes timestamp alignment, time-series resampling, dynamic attitude compensation, draft correction, coordinate transformation, outlier removal, and missing value imputation to obtain navigation element data with a unified spatiotemporal reference.

[0018] For navigation element data with unified spatiotemporal references, reliability weights are assigned based on their physical state indicators. Specifically, m discrete observation samples and their corresponding n physical state indicators are extracted to construct an original evaluation matrix; extreme value standardization is performed based on the positive or negative attributes of each physical state indicator to obtain a standardized matrix Y; and the feature weight P of each observation sample under each physical state indicator is calculated. ij Information entropy value E of each physical state index j and objective entropy weight w j And calculate the comprehensive confidence weight W for the i-th discrete observation sample. i The overall confidence weight W will be used. i By assigning corresponding discrete observation samples, a navigation element dataset with confidence weight labels is obtained.

[0019] Using a navigation element dataset with confidence weights as known samples, a continuous three-dimensional waterway environment field is constructed using a spatiotemporal joint Gaussian probability mapping model. Static water-blocking boundaries of the target waterway region are extracted and a discrete spatial grid is constructed. The spatial distance is determined by the shortest waterway distance D that bypasses the static water-blocking boundary. g (x i ,x j The time distance is calculated using the sample collection time difference Δt, and the x-axis is used to calculate any two known sample points. i and x j The spatiotemporal covariance K between ij The formula is shown below:

[0020] Where, σ f ² represents the signal variance, l s l is the spatial length hyperparameter. t This is a hyperparameter for time duration. Based on W... i Construct the observation noise diagonal matrix O and determine the covariance vector Q. a and self-covariance Q b and according to

[0021]

[0022] Calculate the environmental prediction mean μ(x,t) and uncertainty variance σ²(x,t), where Z is a column vector composed of the measured values ​​of environmental elements from all known samples.

[0023] The exclusive dynamic constraint boundary is determined based on the continuous three-dimensional waterway environment field and the individual attributes and real-time motion state of the target decision-making vessel. The individual attributes of the target decision-making vessel include length, beam, draft, deadweight tonnage, vessel type, maneuvering parameters, and safety level. The real-time motion state includes speed, heading, rudder angle, main engine power, and dynamic changes in draft. For the k-th environmental risk factor, the exclusive dynamic constraint boundary C... k Satisfy the following formula:

[0024] Among them, S k Let Δ be the static baseline value of the k-th environmental risk factor. k (v t ,ω t ) is determined by the real-time speed v t and attitude angle ω t The resulting dynamic correction. Extract the predicted mean μ. k (x,t) and uncertainty σ kSubstitute (x,t) into the standard normal cumulative distribution function Φ to calculate the probability of violation risk, and obtain the comprehensive navigation risk value R(x,t) according to the following formula:

[0025] By setting a first risk threshold τ1 and a second risk threshold τ2, the grid nodes are divided into safe passage zones, risk warning zones, and no-navigation zones, forming a dedicated continuous navigation risk field.

[0026] The dedicated continuous navigation risk field and the real-time position x of the target decision-making vessel at time t. t and speed v t As part of the state space input. With x t The risk matrix is ​​extracted from the dedicated continuous navigation risk field centered on the region, forming the state space input s. t The action space output a is composed of continuous rate of change of heading and acceleration. t Construct a composite reward function r t Composite reward function r t This includes costs for navigation safety, traffic efficiency, power consumption, and control smoothness, with an over-limit penalty term added when R(x,t)≥τ2. Within the Soft Actor-Critic framework, the policy information entropy H is introduced as a regularization term into the policy optimization objective, enabling the deep neural network decision policy π to maximize both the expected cumulative reward and the random entropy of the action probability distribution. The optimal deep neural network decision policy π* is obtained through iterative training, and the output of π* theoretically guides the actions, generating an optimal navigation decision scheme that includes the navigation path and speed.

[0027] Real-time environmental observation data and actual actions of the target decision-making vessel during actual navigation are acquired. The real-time environmental observation data is compared with the environmental prediction results for the corresponding spatiotemporal location in the continuous three-dimensional waterway environmental field to obtain the environmental perception error. The actual actions are compared with the theoretically guided actions in the optimal navigation decision scheme to obtain the decision execution error. A joint loss function L is constructed based on the environmental perception error and the decision execution error.

[0028] Where λ1 is the environmental perception error feedback weight, and λ2 is the decision execution error feedback weight. Using the global joint parameter set before fine-tuning as the initial parameters, backpropagation fine-tuning is performed according to the joint loss function L to obtain the fine-tuned global joint parameter set. The fine-tuned global joint parameter set is loaded into the model at time t+1. Based on the multi-source navigation element data imported at time t+1, the continuous three-dimensional waterway environment field and the dedicated continuous navigation risk field are updated, and the updated state space is input to s. t+1 Output the navigation decision for the next moment.

[0029] In a preferred embodiment, a ship navigation intelligent decision-making method is provided, comprising: S1: Based on a multi-ship collaborative network, acquire multi-source navigation element data for the target waterway area.

[0030] Specifically, the multi-source navigation element data includes real-time perception data acquired by sensors carried by each cooperating vessel, and a schematic diagram of vessel perception and decision-making is attached. Figure 9 As shown, the data includes the ship's real-time three-dimensional positioning coordinates, six-degree-of-freedom attitude data, real-time draft, ship speed, heading, main engine speed and rudder angle; and relative observation data based on different ships, including water depth around the hull, water surface clearance, channel current speed, current direction, distribution information of obstacles on shoals and beaches, location of fixed and moving obstacles, visibility, local wind field and wave elements, as well as channel boundary, navigation mark location and status information.

[0031] S2: Preprocess the multi-source navigation element data.

[0032] S2.1: Since the sampling frequencies of multiple ships and multiple sensors are inconsistent, cubic spline interpolation is used to process them into consistent time series data.

[0033] Suppose the known time series data points are (t0, y0), (t1, y1), ... (t... n ,y n ), at any two adjacent times (t i ,t i+1 Construct a cubic polynomial function S between ) i (t):

[0034] Among them, coefficient , , , Let represent the coefficients of a cubic polynomial, which are solved by simultaneously solving the conditions of continuity, continuity of the first derivative, and continuity of the second derivative. This represents the observed values ​​of navigation elements.

[0035] S2.2: Eliminate measurement projection errors caused by ship motion through the transformation between the ship's coordinate system and the global geographic coordinate system. First, based on the ship's six degrees of freedom attitude angles (roll... A rotation matrix is ​​constructed using pitch θ and bow roll ψ. Then, the vertical offset of the measurement points is corrected according to the real-time draft d of different ships. Finally, the relative observation data is projected from the ship's coordinates to the global geographic coordinate system, thereby achieving spatial unification of multi-source data.

[0036]

[0037] Where R represents a three-dimensional rotation matrix; , , Represents global coordinates; , , Represents the relative observation coordinates of the ship's hull.

[0038] S2.3: Set reasonable threshold intervals (Vmin, Vmax) for each physical quantity of the observed values. If an observed value at a certain time does not belong to the threshold interval, it is determined to be an outlier and removed. Subsequently, for the missing sequences after removal, the time-series moving average method is used for imputation. The calculation process of the time-series moving average method is as follows: assuming the time point of the missing data is t, select the valid observation data with a window size of k before and after this point, and calculate their arithmetic mean as the imputation value.

[0039] S3: The entropy weight method is used to assign confidence weights to the multi-source navigation element data after S2 preprocessing, generating a navigation element dataset with confidence weight labels.

[0040] S3.1: Obtain the preprocessed navigation element dataset, extract m discrete observation samples, and select n key physical state indicators to construct the original evaluation matrix. For the positive (larger is better) and negative (smaller is better) physical attributes of each indicator, the extreme value standardization method is used to perform dimensionless processing on the original data, finally obtaining the standardized matrix Y. The calculation formula is as follows:

[0041] S3.2: For the standardized matrix Y, first calculate the feature weight of the i-th observation sample under the j-th index. Then Substituting into the information entropy model, we obtain the information entropy value of the j-th physical state index. This quantifies the degree of information dispersion of the indicator within the overall sample. The calculation formula is as follows:

[0042] in, This represents the value of the i-th observation sample in the standardized matrix Y under the j-th physical state index; express The corresponding feature proportions; This represents the information entropy value generated by the j-th physical state index in the information entropy model; S3.3: Based on information entropy value Calculate the objective entropy weight of the j-th indicator. The calculation process and formula are as follows:

[0043] Among them, w j Let represent the objective entropy weight of the j-th physical state index.

[0044] S3.4: Combining the standardized matrix Y with the objective weights w j The comprehensive confidence weight W for each discrete observation sample is obtained through linear weighted fusion calculation. i Then calculate the weight W i Assign the data to the corresponding data. The calculation formula is as follows:

[0045] Among them, W i represents the comprehensive confidence weight of the i-th discrete observation sample calculated in the final calculation, where i∈[1,m] and j∈[1,n].

[0046] S4: A navigation element dataset with confidence weights. A spatiotemporal Gaussian probability mapping model is used to construct a spatiotemporal distribution model of the waterway environment. Discrete observation data are transformed into a continuous three-dimensional waterway environment field containing the prediction mean and uncertainty variance. Through the continuous inflow of multi-ship collaborative data, a time sliding window and data augmentation mechanism are used to globally update and share the constructed continuous three-dimensional waterway environment field to obtain the waterway environment status that evolves over time.

[0047] S4.1: Using the data with confidence weights produced in S3 as known samples, an exponential covariance function is used to calculate the attenuation weights of spatial and temporal distances between different samples, and then the results are multiplied and fused. The calculation formula is as follows:

[0048] Among them, K ij The elements of the spacetime kernel matrix K are represented by x. i and x j These represent different sample data; D represents the signal variance; g (x i ,x j () indicates the shortest waterway distance by which a sample point bypasses the water-blocking boundary; t represents the time difference between sample collection; l s and l t These represent the length hyperparameters for smoothing spatial and temporal fluctuations, respectively.

[0049] S4.2: Construct the diagonal matrix of noise, construct the diagonal matrix of observed noise, and assign the confidence weights W to each sample. iThe reciprocal of the product is added to the matrix as a penalty term to suppress low-confidence data. The calculation formula is as follows:

[0050] in, This represents the observation noise diagonal matrix that incorporates confidence weights; diag represents the diagonal matrix construction symbol. This represents the variance of the basic observation noise.

[0051] S4.3: Determine the covariance vector Q between the blank grid node (x,t) to be tested and all known samples using the covariance function. a And the autocovariance Q of the node to be tested b ; S4.4: Based on the Gaussian probability mapping model, regression is performed to calculate the environmental prediction mean μ(x,t) and uncertainty variance σ of the node to be tested. 2 (x,t) generates a continuous three-dimensional waterway environment field. The calculation formula is as follows:

[0052] Where K is the spatiotemporal covariance matrix calculated in step S4.1; Z is a column vector composed of the measured values ​​of all known sample environmental elements.

[0053] S4.5: Set a time window sliding strategy with a preset duration. When new navigation element observation data is incorporated into the multi-ship collaborative network, extract its confidence weight and measured value, and dynamically augment the dimensions of the observation noise diagonal matrix and the measured value column vector. At the same time, remove historical sample points that exceed the time window from the observation noise diagonal matrix O and the measured value column vector Z. Finally, re-execute the above mapping calculation formula using the updated matrix and vector to achieve global dynamic rolling update of the continuous three-dimensional waterway environment field.

[0054] S5: A navigation risk quantification model is constructed using individual dynamic boundary constraints and probability cumulative distribution functions. The model combines the individual attributes of the target decision vessel with the probability cumulative normalization of multiple factors such as water depth margin, current velocity disturbance, channel clearance and obstacle distribution, and performs weighted fusion to generate a continuous navigation risk field specific to the target decision vessel. The channel area is then classified according to the risk threshold. S5.1: For the target decision vessel, extract its individual attribute characteristics (including length, beam, draft, deadweight tonnage, vessel type, maneuvering parameters, and safety level) and real-time motion status (including speed, heading, rudder angle, main engine power, and dynamic changes in draft). Combine the vessel's maneuvering kinematics model and navigation regulations to calculate the vessel's specific dynamic constraint boundary C for various navigation environment elements (such as water depth, current speed, wind, waves, and clearance). kThis ensures that the navigation safety requirements of different ship types are fully met. The calculation formula is as follows:

[0055] Among them, C k S represents the specific dynamic constraint boundary calculated by the target decision-making vessel for the k-th environmental factor; k This represents the static baseline value of the target decision vessel for the k-th environmental factor; This indicates that the ship is subject to its real-time speed v. t With attitude angle The dynamic correction amount resulting from the influence.

[0056] S5.2: Predicted mean μ(x,t) and uncertainty variance σ of environmental elements at each grid node in the 3D waterway environment field generated by S4 2 Using the dynamic constraint boundary (x,t) as the risk assessment benchmark, and substituting it into the probability cumulative distribution function, combined with the predicted mean and uncertainty variance, the probability of violation risk faced by the target decision-making vessel under various environmental factors is calculated. The multi-dimensional violation risk probabilities are then weighted and fused to calculate and output the comprehensive navigation risk value under this grid node, thereby generating a continuous navigation risk field R(x,t). The calculation formula is as follows:

[0057] Where R(x,t) represents the comprehensive navigation risk value of the target decision vessel at grid node (x,t); N represents the dimension of the environmental risk factor; C represents the normalized fusion weight of the k-th environmental risk factor; k Φ represents the exclusive dynamic constraint boundary calculated for the target decision vessel for the k-th environmental factor; Φ represents the standard normal cumulative distribution function, which is used to transform the uncertainty of environmental prediction into the probability of the target decision vessel exceeding the exclusive dynamic constraint boundary.

[0058] S5.3: Set multi-level navigation thresholds τ to map the generated comprehensive navigation risk value into discrete risk level labels, thereby classifying the risk levels of the waterway area. The overall calculation formula for risk quantification is as follows:

[0059] Among them, L safe L warn and L danger These correspond to designated safe passage zones, risk warning zones, and no-navigation zones.

[0060] S6: Based on the continuous navigation risk field generated in S5, a deep reinforcement learning multi-objective optimization model is constructed with navigation safety, traffic efficiency and energy consumption as objectives. Under the condition of satisfying the navigation physical constraints, the model performs joint optimization decision-making on the navigation path and speed of the target decision-making ship by maximizing the expected cumulative reward and policy information entropy, so as to obtain the optimal navigation decision scheme.

[0061] S6.1: The real-time location information of the target decision vessel (such as latitude, longitude, or plane coordinates x) t y t Real-time speed v t Real-time heading ψ t And by stitching together the local region risk matrix generated based on the continuous navigation risk field R(x,t) with the current ship position as the center, a high-dimensional state space input s is constructed. t Then, the ship's maneuvering and control variables are defined as a continuous motion space, with the specific outputs being the rate of change of heading and longitudinal acceleration, forming the motion space vector α. t .

[0062] S6.2: To balance safety, efficiency, and energy consumption, a navigation risk cost term α1R is introduced to minimize navigation risk; The model is penalized for longer travel time by quantifying negative rewards, which encourages the model to shorten the journey at a reasonable speed and improve traffic efficiency. By using the cube of speed to amplify the energy cost of high-speed navigation, the ship's blind acceleration is suppressed, and fuel consumption is reduced. The L2 norm squared of the action vector is used to penalize aggressive maneuvers, avoiding frequent large rudder angles and sudden acceleration / deceleration, thus improving sailing comfort and equipment reliability. Based on this, an indicator function-based approach is added. The boundary violation penalty term, which is formed by the penalty coefficient k, indicates that when the comprehensive navigation risk value R of the ship's current position is greater than or equal to the high-risk warning threshold τ set in S5, a maximum negative reward is triggered, forcing the model to output an evasive action to ensure that the ship does not enter the high-risk restricted navigation area.

[0063] S6.3: To avoid the model getting trapped in local optima in complex waters, such as a single route in a narrow channel, policy information entropy H is introduced as a regularization term in the model's optimization objective. By maximizing entropy, the model is encouraged to maintain the randomness of action distribution during the decision-making process, exploring different combinations of heading and speed throughout the journey. The calculation formula is as follows:

[0064] in, argmax represents the optimal deep neural network decision-making strategy obtained after training the SAC model. The symbol represents the solution operator; E represents the operator for statistical expectation. The model indicates that it follows the current neural network strategy. The simulated sampled state-action trajectory sequence; t represents the current discrete time step; α1-α4 represent the time discount factor; α1-α4 represent the positive normalized weight coefficients for different costs; R represents the value obtained through the S5 mapping at time t and position x. t The overall navigation risk probability value at the location; α t represents the action space vector output by the model at time t; k represents the maximum penalty constant set by the user. This represents an indicator function, which takes the value 1 if the condition is true and 0 if the condition is false. This represents the high-risk warning threshold set in S5; β represents the weighting coefficient used to adjust the relative weight ratio between information entropy reward and basic reward; H represents the information entropy calculator. This represents the probability distribution function of the action output by the current model under the given state st.

[0065] S7: Obtain real-time environmental observation data and actual actions of the target decision-making vessel during actual navigation, compare the deviations with the continuous three-dimensional waterway environment field and the optimal navigation decision scheme, construct a joint loss function based on the extracted environmental perception error and decision execution error as a decision error evaluation model, and use a backpropagation fine-tuning mechanism to update the weights of the deep reinforcement learning multi-objective optimization model according to the error evaluation results, thereby achieving adaptive correction of the decision.

[0066] S7.1: Obtain the real-time environmental observation data set and actual action vector of the target decision-making vessel during actual navigation, and extract the environmental perception error by subtracting the real-time environmental observation data from the environmental prediction mean μ(x,t). The actual action vector is then compared with the theoretical guidance action output by the optimal navigation decision scheme in S6 under the current state. Extracting decision execution error by difference The calculation formula is as follows:

[0067] Where: c represents the dimension index of the environmental element; This represents the target decision-making vessel at discrete time step t and real-time position x. t The actual environmental element observations measured using sensors; This represents the mean of the environmental predictions made by the S4 spatiotemporal distribution model; This represents the space vector of the actions actually performed by the ship at time t; This represents the optimal deep neural network decision policy in a given state s. t The theoretically guided action space vector output under the given conditions.

[0068] S7.2: The feedback weight constants λ1 and λ2 are used to weight and fuse the two factors, constructing a joint loss function as a decision error evaluation model to quantify the current performance deviation of the entire decision-making process. The calculation formula is as follows:

[0069] Where N represents the total dimension of environmental elements, and λ1 and λ2 represent the positive feedback weight constants of environmental perception error and decision execution error, respectively.

[0070] S7.3: Using the adaptive learning rate and gradient calculation mechanism, differentiate the joint loss function and calculate the loss function with respect to the model's global parameters. The gradient is calculated. Parameters are updated along the reverse direction of the gradient, thereby affecting the joint parameters of the deep reinforcement learning multi-objective optimization model. Perform online fine-tuning. Obtain the updated parameter set through iterative calculations. This enables the system to adaptively correct itself in long-term complex environments.

[0071]

[0072] Among them, among them, and η represents the set of global joint parameters before and after fine-tuning; η represents the adaptive learning rate for online reverse fine-tuning of parameters; This represents the gradient differentiation operator.

[0073] S8: Based on the revised model, it performs rolling updates and continuously outputs dynamic navigation risk fields and real-time optimized navigation decision schemes, thereby realizing intelligent decision-making throughout the entire life cycle of ship navigation.

[0074] S8.1, Obtain the updated joint parameter set generated by online fine-tuning in S7. Then, the system's time stepping and state transition mechanism is triggered to load the updated joint parameters into the model at time t+1, completing the iterative replacement of the underlying parameters; S8.2, based on the spatiotemporal distribution model driven by the updated hyperparameters, continuously outputs the dynamic navigation risk field at the next time step. Simultaneously, based on the updated neural network weight parameters, the deep reinforcement learning multi-objective optimization model is driven, inputting s according to the new state space at time t+1. t+1 It outputs updated theoretical guidance action vectors in real time as navigation decision-making schemes.

[0075] In this embodiment, the joint decision-making of navigation path and speed improves the accuracy of waterway environment modeling through multi-ship collaborative perception, confidence weighting, and spatiotemporal Gaussian mapping; it achieves joint decision-making of path and speed by combining ship-specific constraints and reinforcement learning optimization; and it enhances the safety, real-time performance, and adaptability of decision-making through online updates of joint losses.

[0076] In one embodiment, such as Figure 2 As shown, the navigation element dataset with confidence weight identifiers includes the following steps S21-S24: In step S21, the real-time three-dimensional positioning coordinates, six-degree-of-freedom attitude data, real-time draft, speed, heading, main engine speed and rudder angle information of the ship are obtained from the sensors carried by each cooperating vessel. The water depth, water surface clearance height, channel current speed, current direction, obstacle distribution, visibility, local wind field, wave elements, channel boundary, navigation mark position and status information around the hull are also obtained. In step S22, the multi-source navigation element data is timestamped based on a unified time reference, and time series resampling is performed to address the differences in sampling frequencies of different sensors, thereby obtaining navigation element data with a unified time reference. In step S23, based on the six-degree-of-freedom attitude data and real-time draft, and using the ship's coordinate system as a transition, dynamic attitude compensation and draft correction are performed on the relative observation data in the time-referenced navigation element data. The data is then mapped to a unified global geospatial reference frame through a coordinate system transformation matrix, and outlier removal and missing value filling are performed on the mapped data to obtain time-referenced navigation element data. In step S24, extreme value standardization is performed based on the positive or negative attributes of each physical state index to obtain the standardization matrix Y, and the characteristic weight P of each observed sample under each physical state index is calculated. ij Information entropy value E of each physical state index j and objective entropy weight w j The corresponding discrete observation samples are then labeled according to the comprehensive confidence weight Wi to obtain a navigation element dataset with confidence weight labels.

[0077] In this embodiment, the navigation element dataset with confidence weights is generated from multi-source navigation element data jointly provided by various cooperating vessels within the target channel area. Sensors onboard each cooperating vessel collect real-time three-dimensional positioning coordinates, six-DOF attitude data, real-time draft, speed, heading, main engine speed, and rudder angle information. They also collect information on water depth, surface clearance, channel current velocity, current direction, obstacle distribution, visibility, local wind field, wave elements, channel boundaries, and the position and status of navigation marks around the hull. This multi-source navigation element data includes both the motion state data of the target decision-making vessel itself and the relative observation data of other cooperating vessels regarding the target channel area.

[0078] Timestamp alignment of multi-source navigation element data is performed based on a unified time reference, ensuring that data collected by different cooperating vessels and different sensors corresponds to the same time reference. To address the differences in sampling frequencies among different sensors, time-series resampling of the multi-source navigation element data is performed to obtain navigation element data with a unified time reference. Time-series resampling can employ interpolation, downsampling, or window aggregation methods to ensure comparability of data such as speed, heading, water depth, current velocity, current direction, visibility, and local wind field at a unified time point, avoiding deviations in channel environment judgments due to different sampling frequencies.

[0079] Based on six-degree-of-freedom attitude data and real-time draft, and using the ship's coordinate system as an intermediary, dynamic attitude compensation and draft correction are performed on the relative observation data in the time-consolidated navigation element data. Dynamic attitude compensation corrects the effects of roll, pitch, bow, heave, sway, and surge on the observation direction and distance, while draft correction corrects the effects of changes in ship draft on water depth, surface clearance, and the judgment of the relative position of obstacles. The relative observation data after dynamic attitude compensation and draft correction are mapped to a unified global geospatial reference frame through a coordinate system transformation matrix. Outlier removal and missing value imputation are then performed on the mapped data to obtain navigation element data with a unified spatiotemporal reference.

[0080] Based on the positive or negative attributes of each physical state index, extreme value standardization is performed on the navigation element data with unified spatiotemporal reference to obtain the standardization matrix Y, where Y... ij Let represent the standardized value of the i-th discrete observation sample under the j-th physical state index. The characteristic weight P of each observation sample under each physical state index is calculated using the following formula. ij :

[0081] The information entropy value E of each physical state index is calculated using the following formula. j :

[0082] Calculate the objective entropy weight w using the following formula. j :

[0083] The comprehensive confidence weight W for the i-th discrete observation sample is calculated using the following formula. i :

[0084] The comprehensive confidence weight W i By assigning corresponding discrete observation samples, a navigation element dataset with confidence weight labels is obtained.

[0085] In one embodiment, such as Figure 3 As shown, the continuous three-dimensional waterway environment field includes the following steps S31-S33: In step S31, calculate any two known sample points x i and x j The spatiotemporal covariance K between ij The spatial distance is defined as the shortest waterway distance D that bypasses the static water-blocking boundary. g (x i ,x j The time distance is calculated using the sample collection time difference Δt; In step S32, based on the comprehensive confidence weight W i Construct an observation noise diagonal matrix O, wherein the observation noise diagonal matrix O will W i The reciprocal of is used as a noise penalty term for low-confidence samples, and the covariance vector Q between the grid node (x,t) to be tested and all known samples is determined. a And determine the autocovariance Q of the grid node (x,t) to be tested. b ; In step S33, the environmental prediction mean μ(x,t) and uncertainty variance σ²(x,t) of the grid node (x,t) to be tested are calculated according to the following formulas.

[0086] The continuous three-dimensional waterway environment field is generated. When new navigation element data is incorporated into the multi-ship collaborative network, the O and measured value column vector Z are dynamically augmented through a time sliding window, and historical samples that exceed the time window are removed to update the continuous three-dimensional waterway environment field.

[0087] In this embodiment, the continuous three-dimensional waterway environment field is constructed based on a navigation element dataset with confidence weights. For the target waterway area, shorelines, shoals, prohibited navigation areas, fixed obstacles, and non-navigable areas are extracted to form static water-blocking boundaries. A discrete spatial grid is then constructed within the navigable waters defined by these static water-blocking boundaries. The navigation element dataset with confidence weights is used as a known sample set, where each known sample corresponds to a spatial location, a collection time, a measured value, and a comprehensive confidence weight W. i .

[0088] For any two known sample points x i and x j Calculate the spatiotemporal covariance K between the two. ij The spatial distance is not calculated using Euclidean linear distance, but rather using the shortest waterway distance D that bypasses the static water-blocking boundary. g (x i ,x j To avoid misjudgments of spatial correlation caused by crossing shorelines, shoals, or prohibited navigation waters; the temporal distance is represented by the sample collection time difference Δt, used to characterize the degree of correlation of the waterway environment over time. The spatiotemporal covariance K... ij satisfy:

[0089] Where, σ f ² represents the signal variance, l s l is the spatial length hyperparameter. t This is the time length hyperparameter. The spatiotemporal covariance K is derived from the above. ij It can simultaneously depict the continuous changes in the waterway environment across spatial grids and time nodes.

[0090] Based on the comprehensive confidence weight W i Construct the observation noise diagonal matrix O. The observation noise diagonal matrix O will be W i The reciprocal of the product is used as a noise penalty term for low-confidence samples, which increases the overall confidence weight W. i The influence of lower-quality known samples in environmental prediction is weakened, thus reducing the overall confidence weight W. i The contribution of higher-quality known samples to environmental prediction is enhanced. For the grid node to be tested (x,t), the covariance vector Q between the grid node to be tested (x,t) and all known samples is determined. a And determine the autocovariance Q of the grid node (x,t) to be tested. b .

[0091] The environmental prediction mean μ(x,t) and uncertainty variance σ²(x,t) of the grid node (x,t) to be measured are calculated according to the following formulas:

[0092] Among them, K is the spatio-temporal covariance matrix between known samples, and Z is the column vector composed of the measured values of environmental elements of known samples. The continuous three-dimensional channel environment field is jointly generated by the environmental prediction mean value μ(x,t) and the uncertainty variance σ²(x,t) corresponding to each待测 grid node (x,t). When the multi-ship collaborative network incorporates new multi-source navigation element data, the dynamic dimension augmentation of O and the measured value column vector Z is performed through a time sliding window, and the historical samples beyond the time window are removed, so that the continuous three-dimensional channel environment field is updated with the real-time state of the target channel area rolling.

[0093] In one embodiment, as Figure 4 shown, the exclusive continuous navigation risk field includes the following steps S41 - S44: In step S41, extract the individual attributes and real-time motion state of the target decision-making ship. The individual attributes include ship length, ship width, draft, deadweight tonnage, ship type, maneuverability parameters, and safety level. The real-time motion state includes ship speed, course, rudder angle, main engine power, and dynamic change of draft; In step S42, according to the individual attributes, the real-time motion state, the ship maneuvering kinematic model, and the navigation regulations, calculate the exclusive dynamic constraint boundary C of the target decision-making ship for the k-th environmental risk element k ; In step S43, extract the prediction mean value μ k (x,t) and uncertainty σ k (x,t) of the k-th environmental risk element at each grid node (x,t) from the continuous three-dimensional channel environment field. Substitute C k , μ k (x,t) and σ k (x,t) into the standard normal cumulative distribution function Φ, calculate the violation risk probability of the target decision-making ship relative to each environmental risk element, and obtain the comprehensive navigation risk value R(x,t) at the grid node (x,t) through weighted fusion; In step S44, set the first risk threshold τ1 and the second risk threshold τ2. When 0 ≤ R(x,t) ≤ τ1, divide the corresponding grid node into the safe passage area. When τ1 < R(x,t) < τ2, divide the corresponding grid node into the risk warning area. When R(x,t) ≥ τ2, divide the corresponding grid node into the no-go area. The exclusive continuous navigation risk field is composed of the comprehensive navigation risk values and risk levels of each grid node.

[0094] In this embodiment, the dedicated continuous navigation risk field is generated based on the continuous three-dimensional waterway environment field, the individual attributes of the target decision-making vessel, and its real-time motion state. The individual attributes of the target decision-making vessel include length, beam, draft, deadweight tonnage, vessel type, maneuvering parameters, and safety level. The real-time motion state includes speed, heading, rudder angle, main engine power, and dynamic changes in draft. The individual attributes characterize the navigation requirements of the target decision-making vessel under static conditions, while the real-time motion state characterizes the dynamic response characteristics of the target decision-making vessel under current navigation conditions.

[0095] Based on individual attributes, real-time motion status, ship maneuvering kinematics model, and navigation regulations, calculate the specific dynamic constraint boundary C of the target decision-making ship for the k-th environmental risk factor. k The dedicated dynamic constraint boundary C k satisfy:

[0096] Among them, S k Let Δ be the static baseline value of the k-th environmental risk factor. k (v t ,ω t ) is determined by the real-time speed v t and attitude angle ω t The resulting dynamic correction. The k-th environmental risk factor may include at least one of the following: water depth margin, channel clearance, current velocity disturbance, obstacle distance, visibility, and wave elements. This is achieved by introducing Δ... k (v t ,ω t ), making the dedicated dynamic constraint boundary C k It can adjust to changes in the real-time motion state of the target decision vessel.

[0097] The predicted mean μ of the k-th environmental risk element at each grid node (x,t) is extracted from the continuous three-dimensional waterway environmental field. k (x,t) and uncertainty σ k (x,t). Let C... k μ k (x,t) and σ k Substituting (x,t) into the standard normal cumulative distribution function Φ, we calculate the probability of violation risk of the target decision vessel relative to each environmental risk factor. Since μ k (x,t) reflects the prediction level of the k-th environmental risk factor at grid node (x,t), σ k (x,t) reflects the uncertainty of the k-th environmental risk factor at grid node (x,t). Therefore, the probability of violation risk can simultaneously characterize the environmental prediction result and the specific dynamic constraint boundary C. k The degree of deviation between them and the credibility of environmental prediction results.

[0098] The violation risk probabilities of each environmental risk factor are weighted and fused according to the following formula to obtain the comprehensive navigation risk value R(x,t) at the grid node (x,t):

[0099] where ω k is the normalized fusion weight of the k-th environmental risk factor. Set the first risk threshold τ1 and the second risk threshold τ2. When 0 ≤ R(x,t) ≤ τ1, the corresponding grid node is classified as a safe passage area; when τ1 < R(x,t) < τ2, the corresponding grid node is classified as a risk warning area; when R(x,t) ≥ τ2, the corresponding grid node is classified as a no-go area. The comprehensive navigation risk value R(x,t) and risk level of each grid node jointly constitute an exclusive continuous navigation risk field.

[0100] In one embodiment, as Figure 5 shown, the optimal navigation decision scheme includes the following steps S51 - S54: In step S51, with the real-time position x t of the target decision-making ship at time t, the speed v t , and the regional risk matrix intercepted from the exclusive continuous navigation risk field centered on x t constitute the state space input s t , and the continuous heading change rate and acceleration constitute the action space output a t ; In step S52, construct a composite reward function r t , the composite reward function r t includes the cost of navigation safety, the cost of passage efficiency, the cost of power energy consumption, and the cost of control smoothness, and superimposes an out-of-bounds penalty term when R(x,t) ≥ τ2; In step S53, in the Soft Actor-Critic framework, introduce the policy information entropy H as a regularization term into the policy optimization objective, so that the optimal deep neural network decision-making policy π maximizes the random entropy of the action probability distribution while maximizing the expected cumulative reward; In step S54, obtain the optimal deep neural network decision-making policy π* through iterative training, and use the optimal deep neural network decision-making policy π* to output theoretical guidance actions under the current state space input st, and convert the theoretical guidance actions at consecutive moments into an optimal navigation decision scheme including the navigation path and speed.

[0101] In this embodiment, the optimal navigation decision scheme is generated based on the exclusive continuous navigation risk field and the real-time motion state of the target decision-making ship. The real-time position of the target decision-making ship at time t is denoted as x t , and the speed is denoted as vt , with x t A regional risk matrix is ​​extracted from a pre-defined range within a dedicated continuous navigation risk field, centered on the x-axis, and the real-time position x is used as the reference. t Speed ​​v t Together with the regional risk matrix, they constitute the state space input s t Action space output a t Composed of continuous rate of change of course and acceleration, it is used to characterize the amount of course adjustment and speed change that the target decision-making vessel can perform at the current moment.

[0102] Construct a composite reward function r t Output a to the action space t Evaluation is performed. The composite reward function r... t This includes navigation safety costs, traffic efficiency costs, power consumption costs, and control smoothness costs. Navigation safety costs constrain the target decision-making vessel to stay away from high-risk grid nodes; traffic efficiency costs constrain the target decision-making vessel to approach the predetermined navigation target; power consumption costs constrain the magnitude of acceleration and speed changes; and control smoothness costs constrain the continuous motion space output 'a'. t The degree of change. When the grid node where the target decision ship is located satisfies R(x,t)≥τ2, in the composite reward function r t Overlapping boundary violation penalties helps discourage actions that would lead to entering a restricted area.

[0103] Within the Soft Actor-Critic framework, policy information entropy H is introduced as a regularization term into the policy optimization objective, enabling the deep neural network decision-making policy π to maximize both the expected cumulative reward and the stochastic entropy of the action probability distribution. By introducing policy information entropy H, the deep neural network decision-making policy π can maintain its exploration capability among multiple executable actions, avoiding premature convergence to a local navigation scheme in complex navigation environments, thereby improving its adaptability to dynamic risk changes.

[0104] The optimal deep neural network decision strategy π* is obtained through iterative training. The current state space is input into s. t The input is fed into the optimal deep neural network decision strategy π*, which outputs theoretically guided actions. The theoretically guided actions include continuous rates of change of course and acceleration. Based on the theoretically guided actions at consecutive time intervals, the course, speed, and position of the target decision vessel are updated to obtain the optimal navigation decision scheme that includes the navigation path and speed.

[0105] In one embodiment, such as Figure 6As shown, the online updating of the parameters of the deep reinforcement learning multi-objective optimization model through the joint loss function, and the rolling updating of the continuous three-dimensional waterway environment field and the dedicated continuous navigation risk field based on the updated parameters and the multi-source navigation element data injected at the next time step, includes the following steps S61-S64: In step S61, real-time environmental observation data and actual actions performed by the target decision-making vessel during actual navigation are obtained, and the difference between the real-time environmental observation data and the mean of the environmental prediction is calculated to obtain the environmental perception error. In step S62, the actual executed action is compared with the theoretically guided action output by the optimal deep neural network decision strategy π under the state space input st to obtain the decision execution error; In step S63, a joint loss function is constructed, using the global joint parameter set before fine-tuning as the initial parameters. The parameters of the deep reinforcement learning multi-objective optimization model are then fine-tuned through backpropagation according to the joint loss function to obtain the global joint parameter set after fine-tuning. In step S64, the fine-tuned global joint parameter set is loaded into the model at time t+1. The continuous three-dimensional waterway environment field and the dedicated continuous navigation risk field are updated based on the multi-source navigation element data imported at time t+1, and the updated state space is input s. t+1 Output the navigation decision for the next moment.

[0106] In this embodiment, the parameters of the deep reinforcement learning multi-objective optimization model are updated online through a joint loss function. Based on the updated parameters and the multi-source navigation element data incorporated at the next time step, the continuous three-dimensional channel environment field and the dedicated continuous navigation risk field are continuously updated. Real-time environmental observation data and actual actions performed by the target decision-making vessel during actual navigation are acquired. The real-time environmental observation data includes at least one of the following elements at the target decision-making vessel's current position: water depth, water surface clearance, channel current velocity, current direction, obstacle distribution, visibility, local wind field, and wave elements. The actual actions performed include the target decision-making vessel's actual rate of change of course and acceleration.

[0107] The environmental perception error is obtained by subtracting the mean of the environmental predictions for the corresponding spatiotemporal locations in the continuous three-dimensional waterway environmental field from the real-time environmental observation data. For the c-th environmental element, the real-time environmental observation data can be represented as Z. real,c (x t The environmental prediction mean can be expressed as μ(t). c (x t The environmental perception error can be expressed as E(t). sense,c =Z real,c (x t ,t)-μ c (x tThe actual executed action is compared with the theoretically guided action output by the optimal deep neural network decision strategy π under the state space input st, to obtain the decision execution error, which can be expressed as E. exec =a real (t)-π*(s t ), where area(t) represents the actual action performed at time t.

[0108] A joint loss function L is constructed based on environmental perception error and decision execution error, wherein the joint loss function L satisfies:

[0109] Wherein, λ1 is the environmental perception error feedback weight, λ2 is the decision execution error feedback weight, and N is the number of environmental risk factors. The environmental perception error feedback weight λ1 is used to adjust the influence of the prediction deviation of the continuous three-dimensional waterway environment field on parameter updates, and the decision execution error feedback weight λ2 is used to adjust the influence of the deviation between theoretically guided actions and actual executed actions on parameter updates. Using the global joint parameter set before fine-tuning as the initial parameters, the parameters of the deep reinforcement learning multi-objective optimization model are fine-tuned through backpropagation according to the joint loss function L, resulting in the fine-tuned global joint parameter set.

[0110] The fine-tuned global joint parameter set is loaded into the model at time t+1, and multi-source navigation element data is received at time t+1. Based on the multi-source navigation element data received at time t+1, the mean and uncertainty variance of the environmental prediction in the continuous three-dimensional channel environment field are updated, and the exclusive continuous navigation risk field is updated by combining the individual attributes and real-time motion state of the target decision vessel at time t+1. The updated exclusive continuous navigation risk field, the real-time position and speed of the target decision vessel at time t+1 constitute the updated state space input s. t+1 The updated deep reinforcement learning multi-objective optimization model outputs the navigation decision scheme for the next moment.

[0111] In one embodiment, Figure 7 This is a block diagram of a ship navigation intelligent decision-making system according to an exemplary embodiment. Figure 7 As shown, the intelligent decision-making system for ship navigation includes an acquisition module 71, a calculation module 72, a processing module 73, an output module 74, and an update module 75.

[0112] The acquisition module 71 is used to acquire multi-source navigation element data of the target waterway area based on a multi-ship cooperative network, perform spatiotemporal preprocessing on the multi-source navigation element data, and allocate confidence weights according to the physical state indicators of the preprocessed multi-source navigation element data to obtain a navigation element dataset with confidence weight identifiers. The calculation module 72 is used to calculate the mean and uncertainty variance of the waterway environment at the spatial grid and time nodes using the navigation element dataset with confidence weight as a known sample and a spatiotemporal joint Gaussian probability mapping model to obtain a continuous three-dimensional waterway environment field. The processing module 73 is used to determine the exclusive dynamic constraint boundary based on the continuous three-dimensional waterway environment field and the individual attributes and real-time motion state of the target decision vessel, calculate the probability of exceeding the limit of the environmental prediction mean and uncertainty variance relative to the exclusive dynamic constraint boundary, and normalize and weightedly fuse the probability of exceeding the limit to obtain the exclusive continuous navigation risk field. The output module 74 is used to input the exclusive continuous navigation risk field, the real-time position and speed of the target decision vessel as the state space into a preset deep reinforcement learning multi-objective optimization model, and output the optimal navigation decision scheme including the navigation path and speed under the condition of satisfying the navigation physical constraints. The update module 75 is used to acquire real-time environmental observation data and actual actions of the target decision-making vessel during actual navigation. It compares the deviation of the real-time environmental observation data with the environmental prediction results of the corresponding spatiotemporal position in the continuous three-dimensional waterway environment field, and compares the deviation of the actual actions with the theoretical guidance actions in the optimal navigation decision scheme. It constructs a joint loss function based on environmental perception error and decision execution error, updates the parameters of the deep reinforcement learning multi-objective optimization model online through the joint loss function, and updates the continuous three-dimensional waterway environment field and the dedicated continuous navigation risk field on a rolling basis based on the updated parameters and the multi-source navigation element data incorporated in the next moment, and generates the navigation decision scheme for the next moment.

[0113] The acquisition module 71, calculation module 72, processing module 73, output module 74, and update module 75 included in the block diagram of the intelligent ship navigation decision-making system are controlled to execute the intelligent ship navigation decision-making method described in any of the above embodiments.

[0114] like Figure 8 As shown, the present invention provides an electronic device 800, which includes: a communication interface, a processor 801, and a memory 802; The memory 802 stores program instructions. When executed by the processor 801, which is connected to the memory 802 via the communication interface, the program instructions acquire multi-source navigation element data of the target channel based on a multi-ship collaborative network. After spatiotemporal preprocessing and confidence weighting, a navigation element dataset is formed. A continuous three-dimensional channel environment field is constructed using a spatiotemporal joint Gaussian probability mapping model. A dedicated continuous navigation risk field is generated by combining the individual attributes and motion state of the target decision-making vessel. The risk field, position, and speed are further input into a deep reinforcement learning multi-objective optimization model, which outputs path and speed decisions. A joint loss function is constructed based on actual navigation observation data and execution deviations. The parameters of the deep reinforcement learning multi-objective optimization model are updated online, realizing the rolling update of the continuous three-dimensional channel environment field, the dedicated continuous navigation risk field, and the navigation decision scheme.

[0115] This invention provides a computer-readable storage medium storing computer program instructions. When executed by a processor, these instructions acquire multi-source navigation element data of a target waterway based on a multi-ship cooperative network. After spatiotemporal preprocessing and confidence weighting, a navigation element dataset is formed. A continuous three-dimensional waterway environment field is constructed using a spatiotemporal joint Gaussian probability mapping model. A dedicated continuous navigation risk field is generated by combining the individual attributes and motion states of the target decision-making vessel. Furthermore, the risk field, position, and speed are input into a deep reinforcement learning multi-objective optimization model, which outputs path and speed decisions. A joint loss function is constructed based on actual navigation observation data and execution deviations, and the parameters of the deep reinforcement learning multi-objective optimization model are updated online, enabling the continuous three-dimensional waterway environment field, the dedicated continuous navigation risk field, and the navigation decision scheme to be updated continuously.

[0116] It should be understood that the specific features, operations, and details described above regarding the method of the present invention can also be similarly applied to the apparatus and system of the present invention, or vice versa. Furthermore, each step of the method of the present invention described above can be performed by a corresponding component or unit of the apparatus or system of the present invention.

[0117] It should be understood that the various modules / units of the device of the present invention can be implemented wholly or partially through software, hardware, firmware, or a combination thereof. Each module / unit can be embedded in the processor of a computer device in hardware or firmware form or independent of the processor, or it can be stored in the memory of a computer device in software form for the processor to call to execute the operation of each module / unit. Each module / unit can be implemented as an independent component or module, or two or more modules / units can be implemented as a single component or module.

[0118] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores computer instructions executable by the processor, which, when executed by the processor, instruct the processor to perform steps of the methods of embodiments of the present invention. The computer device can be broadly categorized as a server, terminal, or any other electronic device with the necessary computing and / or processing capabilities. In one embodiment, the computer device may include a processor, memory, network interface, communication interface, etc., connected via a system bus. The processor of the computer device can be used to provide the necessary computing, processing, and / or control capabilities. The memory of the computer device may include a non-volatile storage medium and internal memory. The non-volatile storage medium may store an operating system, computer programs, etc. The internal memory can provide an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface and communication interface of the computer device can be used to connect and communicate with external devices via a network. When the computer program is executed by the processor, it performs the steps of the methods of the present invention.

[0119] This invention can be implemented as a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, causes the steps of the methods of embodiments of the invention to be performed. In one embodiment, the computer program is distributed across multiple network-coupled computer devices or processors, such that the computer program is stored, accessed, and executed in a distributed manner by one or more computer devices or processors. A single method step / operation, or two or more method steps / operations, may be executed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations may be executed by one or more computer devices or processors, and one or more other method steps / operations may be executed by one or more other computer devices or processors. One or more computer devices or processors may execute a single method step / operation, or execute two or more method steps / operations.

[0120] It will be understood by those skilled in the art that the method steps of the present invention can be performed by a computer program instructing related hardware, such as a computer device or processor. The computer program may be stored in a non-transitory computer-readable storage medium, and its execution causes the steps of the present invention to be performed. Depending on the context, any references herein to memory, storage, databases, or other media may include non-volatile and / or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0121] The technical features described above can be combined arbitrarily. Although not all possible combinations of these technical features are described, any combination of these technical features should be considered to be covered by this specification, provided that such combination does not contain contradictions.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A smart decision-making method for ship navigation, characterized in that, include: Multi-source navigation element data of the target waterway area is obtained based on a multi-ship cooperative network. The multi-source navigation element data is preprocessed in time and space. Confidence weights are assigned according to the physical state indicators of the preprocessed multi-source navigation element data to obtain a navigation element dataset with confidence weights. Using the navigation element dataset with confidence weights as known samples, a spatiotemporal joint Gaussian probability mapping model is used to calculate the mean and uncertainty variance of the waterway environment at spatial grids and time nodes, thus obtaining a continuous three-dimensional waterway environment field. Based on the continuous three-dimensional waterway environment field and the individual attributes and real-time motion state of the target decision-making vessel, a dedicated dynamic constraint boundary is determined. The probability of exceeding the limits of the environmental prediction mean and uncertainty variance relative to the dedicated dynamic constraint boundary is calculated. The probability of exceeding the limits is normalized and weighted and fused to obtain a dedicated continuous navigation risk field. The exclusive continuous navigation risk field, the real-time position and speed of the target decision vessel are input into the state space into a preset deep reinforcement learning multi-objective optimization model, and the optimal navigation decision scheme including the navigation path and speed is output under the condition of satisfying the navigation physical constraints. The system acquires real-time environmental observation data and actual actions of the target decision-making vessel during its actual navigation process. It then compares the deviation of the real-time environmental observation data with the environmental prediction results of the corresponding spatiotemporal location in the continuous three-dimensional waterway environment field, and compares the deviation of the actual actions with the theoretically guided actions in the optimal navigation decision scheme. Based on the environmental perception error and decision execution error, it constructs a joint loss function and updates the parameters of the deep reinforcement learning multi-objective optimization model online through the joint loss function. Based on the updated parameters and the multi-source navigation element data incorporated at the next moment, it continuously updates the continuous three-dimensional waterway environment field and the dedicated continuous navigation risk field, and generates the navigation decision scheme for the next moment.

2. The intelligent decision-making method for ship navigation as described in claim 1, characterized in that, The navigation element dataset with confidence weight identifiers includes: The system acquires real-time three-dimensional positioning coordinates, six-degree-of-freedom attitude data, real-time draft, speed, heading, main engine speed and rudder angle information from the sensors carried by each cooperating vessel, as well as water depth, water surface clearance, channel current speed, current direction, obstacle distribution, visibility, local wind field, wave elements, channel boundary, navigation mark position and status information around the hull. The multi-source navigation element data is timestamped based on a unified time reference, and time series resampling is performed to address the differences in sampling frequencies of different sensors, resulting in navigation element data with a unified time reference. Based on the six-degree-of-freedom attitude data and real-time draft, using the ship's coordinate system as a transition, dynamic attitude compensation and draft correction are performed on the relative observation data in the time-referenced navigation element data. The data is then mapped to a unified global geospatial reference frame through a coordinate system transformation matrix. Outlier removal and missing value filling are performed on the mapped data to obtain time-referenced navigation element data. Based on the positive or negative attributes of each physical state index, extreme value standardization is performed to obtain the standardization matrix Y. Then, the characteristic weight P of each observed sample under each physical state index is calculated. ij Information entropy value E of each physical state index j and objective entropy weight w j The corresponding discrete observation samples are then labeled according to the comprehensive confidence weight Wi to obtain a navigation element dataset with confidence weight labels.

3. The intelligent decision-making method for ship navigation as described in claim 2, characterized in that, The continuous three-dimensional waterway environment field includes: Extract the static water-blocking boundary of the target waterway area and construct a discrete spatial grid, and use the dataset of navigation elements with confidence weight identification as the known sample set; Calculate any two known sample points x i and x j The spatiotemporal covariance K between ij The spatial distance is defined as the shortest waterway distance D that bypasses the static water-blocking boundary. g (x i ,x j The time distance is represented by the sample collection time difference Δt, and the spatiotemporal covariance K is... ij satisfy: Where, σ f ² represents the signal variance, l s l is the spatial length hyperparameter. t This is a hyperparameter for time duration; Based on the comprehensive confidence weight W i Construct an observation noise diagonal matrix O, wherein the observation noise diagonal matrix O will W i The reciprocal of is used as a noise penalty term for low-confidence samples, and the covariance vector Q between the grid node (x,t) to be tested and all known samples is determined. a And determine the autocovariance Q of the grid node (x,t) to be tested. b ; Calculate the environmental prediction mean μ(x,t) and uncertainty variance σ²(x,t) of the待测 grid node (x,t) according to the following formula to generate the continuous three-dimensional waterway environmental field, When new navigation element data is incorporated into the multi-ship cooperation network, perform dynamic dimension augmentation on O and the measured value column vector Z through a time sliding window, and剔除 historical samples that exceed the time window to update the continuous three-dimensional waterway environmental field.

4. The intelligent decision-making method for ship navigation as described in claim 1, characterized in that, The exclusive continuous navigation risk field includes: Extract the individual attributes and real-time motion state of the target decision-making ship. The individual attributes include ship length, ship width, draft, deadweight tonnage, ship type, maneuverability parameters, and safety level. The real-time motion state includes ship speed, course, rudder angle, main engine power, and dynamic change of draft; Based on the individual attributes, the real-time motion state, the ship maneuvering kinematics model, and navigation regulations, calculate the specific dynamic constraint boundary C of the target decision-making ship for the k-th environmental risk factor. k , Among them, S k Let Δ be the static baseline value of the k-th environmental risk factor. k (v t ,ω t ) is determined by the real-time speed v t and attitude angle ω t The resulting dynamic correction amount; The predicted mean μ of the k-th environmental risk element at each grid node (x,t) is extracted from the continuous three-dimensional waterway environmental field. k (x,t) and uncertainty σ k (x,t), C k μ k (x,t) and σ k Substituting (x,t) into the standard normal cumulative distribution function Φ, the probability of violation risk of the target decision vessel relative to each environmental risk factor is calculated, and the comprehensive navigation risk value R(x,t) at the grid node (x,t) is obtained by weighted fusion according to the following formula. Where, ω k The normalized fusion weight of the k-th environmental risk factor; Set the first risk threshold τ1 and the second risk threshold τ2. When 0≤R(x,t)≤τ1, divide the corresponding grid node into a safe passage area. When τ! < R(x,t) < τ2, divide the corresponding grid node into a risk warning area. When R(x,t)≥τ2, divide the corresponding grid node into a no-go area. The exclusive continuous navigation risk field is composed of the comprehensive navigation risk values and risk levels of each grid node.

5. The intelligent decision-making method for ship navigation as described in claim 4, characterized in that, The optimal navigation decision-making plan includes: The target decision is the real-time position x of the ship at time t. t Speed ​​v t And with x t The state space input s is formed by the regional risk matrix extracted from the dedicated continuous navigation risk field centered on the risk matrix. t The action space output a is composed of continuous rate of change of heading and acceleration. t ; Construct a composite reward function r t The composite reward function r t It includes navigation safety costs, traffic efficiency costs, power consumption costs, and control smoothness costs, and adds an over-boundary penalty term when R(x,t)≥τ2; Under the Soft Actor-Critic framework, introduce the policy information entropy H as a regularization term into the policy optimization objective, so that the optimal deep neural network decision-making policy π maximizes the expected cumulative reward while maximizing the random entropy of the action probability distribution; Obtain the optimal deep neural network decision-making policy π* through iterative training, and use the optimal deep neural network decision-making policy π* to output the theoretical guidance action under the current state space input st, and convert the theoretical guidance actions at consecutive times into an optimal navigation decision-making plan including the navigation path and ship speed.

6. The intelligent decision-making method for ship navigation as described in claim 5, characterized in that, The online update of the parameters of the deep reinforcement learning multi-objective optimization model through the joint loss function, and the rolling update of the continuous three-dimensional waterway environmental field and the exclusive continuous navigation risk field based on the updated parameters and the multi-source navigation element data incorporated at the next moment, include: Obtain the real-time environmental observation data and actual execution actions of the target decision-making ship during actual navigation, subtract the real-time environmental observation data from the environmental prediction mean to obtain the environmental perception error; Compare the actual execution action with the theoretical guidance action output by the optimal deep neural network decision-making policy π under the state space input st to obtain the decision execution error; Construct a joint loss function according to the following formula, where λ1 is the environmental perception error feedback weight, λ2 is the decision execution error feedback weight, use the global joint parameter set before fine-tuning as the initial parameter, and perform backpropagation fine-tuning on the parameters of the deep reinforcement learning multi-objective optimization model according to the joint loss function to obtain the fine-tuned global joint parameter set; The fine-tuned global joint parameter set is loaded into the model at time t+1. Based on the multi-source navigation element data imported at time t+1, the continuous three-dimensional channel environment field and the dedicated continuous navigation risk field are updated, and the updated state space is input s. t+1 Output the navigation decision for the next moment.

7. A ship navigation intelligent decision-making system, characterized in that, Include: The acquisition module is used to acquire multi-source navigation element data of the target waterway area based on a multi-ship cooperative network, perform spatiotemporal preprocessing on the multi-source navigation element data, and allocate confidence weights according to the physical state indicators of the preprocessed multi-source navigation element data to obtain a navigation element dataset with confidence weight identifiers. The calculation module is used to calculate the mean and uncertainty variance of the waterway environment at the spatial grid and time nodes using the navigation element dataset with confidence weights as known samples and a spatiotemporal joint Gaussian probability mapping model, so as to obtain a continuous three-dimensional waterway environment field. The processing module is used to determine the exclusive dynamic constraint boundary based on the continuous three-dimensional waterway environment field and the individual attributes and real-time motion state of the target decision vessel, calculate the probability of exceeding the limit of the environmental prediction mean and uncertainty variance relative to the exclusive dynamic constraint boundary, and normalize and weightedly fuse the probability of exceeding the limit to obtain the exclusive continuous navigation risk field. The output module is used to input the exclusive continuous navigation risk field, the real-time position and speed of the target decision vessel as the state space into a preset deep reinforcement learning multi-objective optimization model, and output the optimal navigation decision scheme including the navigation path and speed under the condition of satisfying the navigation physical constraints. The update module is used to acquire real-time environmental observation data and actual actions of the target decision-making vessel during actual navigation. It compares the deviation of the real-time environmental observation data with the environmental prediction results of the corresponding spatiotemporal position in the continuous three-dimensional waterway environment field, and compares the deviation of the actual actions with the theoretical guidance actions in the optimal navigation decision scheme. It constructs a joint loss function based on environmental perception error and decision execution error, and updates the parameters of the deep reinforcement learning multi-objective optimization model online through the joint loss function. Based on the updated parameters and the multi-source navigation element data brought in at the next moment, it continuously updates the continuous three-dimensional waterway environment field and the dedicated continuous navigation risk field, and generates the navigation decision scheme for the next moment.

8. The intelligent decision-making system for ship navigation as described in claim 7, characterized in that: The acquisition module, the calculation module, the processing module, the output module, and the update module are controlled to execute the intelligent decision-making method for ship navigation as described in any one of claims 2 to 6.

9. An electronic device, characterized in that, include: Communication interface, processor, memory; The memory is used to store program instructions, which, when executed by the processor that is communicatively connected to the memory via the communication interface, enable the electronic device to implement the intelligent decision-making method for ship navigation as described in any one of claims 1 to 6.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by a computer, the computer enables the computer to implement the intelligent decision-making method for ship navigation as described in any one of claims 1 to 6.

Images