A method for autonomous path planning of a dredging vessel based on reinforcement learning

Abstract

The present application relates to the technical field of dredging engineering, and particularly relates to a kind of autonomous path planning method of dredging ship based on reinforcement learning, comprising: real-time acquisition multidimensional state space vector;Construct digital twin scene, through the incremental environment complexity, adopt three-stage course learning strategy training strategy network, and pass through hierarchical reward function dynamic adjustment path planning priority;The trained strategy network is migrated to ship entity;Based on target multimodal perception system, obstacles are identified, collision risk is evaluated through dynamic probability model, and local path reconstruction is carried out when there is collision risk;In the multi-ship cooperative operation scene, a global optimization model is constructed, distributed iterative solution and incremental updating strategy are used to dynamically redistribute tasks for each ship until convergence, and the cooperative optimization scheme of each ship is output.The present application can improve the path planning efficiency in the multi-ship cooperative operation scene, and reduce the training cost.

Description

Technical Field

[0001] This invention relates to the field of dredging engineering technology, and more specifically to an autonomous path planning method for dredging vessels based on reinforcement learning. Background Technology

[0002] Traditional dredging operations have significant drawbacks. For example, path planning methods based on human experience are difficult to adapt to the dynamic changes in complex underwater environments, resulting in low operational efficiency. Redundant navigation caused by fixed operating modes means that equipment energy consumption accounts for more than 40% of the total operating cost, making it uneconomical. Disturbance of bottom sediment by mechanical operations can easily cause the spread of suspended matter, seriously threatening the integrity of sensitive ecological areas such as coral reefs and seagrass beds. At the same time, environmental perception solutions based on single sensors have insufficient data dimensions. The false detection rate of obstacles caused by environmental interference of single sensors (such as sonar) is as high as 25%, and the lack of detection capabilities for underwater reefs, shipwrecks, and other obstacles further exacerbates the risk of collision.

[0003] Furthermore, traditional path planning algorithms such as A* and Dijkstra can only achieve single-objective optimization and cannot effectively balance improving dredging efficiency with controlling ecological disturbances. Meanwhile, reinforcement learning path planning schemes generally lack embedded fluid dynamics equation constraints, and the policy network exhibits a transfer error exceeding 30% between virtual simulation environments like Unity-Mujoco and real ship control systems, making it difficult to meet operational requirements under complex hydrodynamic conditions. These issues collectively lead to multiple challenges in efficiency, energy consumption, ecology, and safety for traditional dredging operations, necessitating an intelligent path planning method that integrates multi-source sensor data and possesses physical constraint awareness capabilities. Summary of the Invention

[0004] In view of this, the present invention provides a method for autonomous path planning of dredging vessels based on reinforcement learning, which can improve the path planning efficiency in multi-vessel collaborative operation scenarios and reduce training costs.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for autonomous path planning of dredging vessels based on reinforcement learning, characterized by the following steps:

[0007] S1. Real-time acquisition of three-dimensional terrain data, water body reflectance spectrum data, dredging pump power data and three-dimensional flow velocity data, and after preprocessing, a multi-dimensional state space vector is obtained.

[0008] S2. Construct a digital twin scenario based on multi-dimensional state space vectors. In the digital twin scenario, train the policy network by increasing the environmental complexity and adopting a three-stage course learning strategy. Introduce Navier-Stokes residual constraints to enable the policy network to generate mixed control commands including dredging pump control, hull steering angle and speed. Dynamically adjust the path planning priority through a hierarchical reward function. Transfer the trained policy network to the ship entity.

[0009] S3. Generate discrete-continuous hybrid control commands and global path points for the ship based on the policy network, identify obstacles based on the target multimodal perception system, assess collision risk through a dynamic probability model, and reconstruct the local path when there is a collision risk.

[0010] S4. In the scenario of multi-ship collaborative operation, a global optimization model is constructed, an initial sub-region is allocated to each ship, and a distributed iterative solution is adopted. When the triggering condition is reached, an incremental update strategy is adopted to dynamically redistribute tasks to each ship until convergence, and the collaborative optimization scheme of each ship is output.

[0011] Furthermore, the process of acquiring various data in S1 includes:

[0012] Deploy multibeam side-scan sonar to the mid-bottom area of ​​the ship to generate three-dimensional terrain data of the work area in real time.

[0013] The hyperspectral imaging system was mounted on the bow deck to collect water reflectance spectral data of a preset area. The water reflectance spectral data was clustered using the DBSCAN algorithm to obtain several clusters. Each cluster represents an area with similar water reflectance spectral characteristics. The sediment density level corresponding to each cluster was determined according to a predefined sediment density level standard.

[0014] The power data of the dredging pump is collected in real time using a Hall sensor;

[0015] Three-dimensional flow velocity data within a preset water depth range are measured using an ADCP flow meter.

[0016] Furthermore, in S1, the methods for preprocessing various types of data include:

[0017] After spatiotemporal registration of the 3D terrain data and water reflectance spectral data, noise suppression is then performed.

[0018] The dredging pump power was normalized, and the three-dimensional flow velocity data was vector decomposed.

[0019] Based on the spatiotemporal convolutional neural network, feature extraction is performed on the processed 3D terrain data, water body reflectance spectrum data, dredging pump power and 3D flow velocity data. A multi-head parallel attention mechanism is introduced to fuse the data of each dimension, resulting in a multi-dimensional state space vector containing terrain complexity, sediment density level, energy consumption value, water flow component and environmental dynamic parameters.

[0020] Furthermore, in S2, the process of constructing a digital twin scenario includes:

[0021] A fluid dynamics model based on the Navier-Stokes equations is constructed to simulate the flow velocity range of 0-10kN and supports dynamic water flow fields for laminar and turbulent multimodal flows.

[0022] A sediment settling model was established using the Stokes settling formula and a two-phase flow coupling algorithm, defining the sediment particle size as 0.1-5 mm and the settling velocity as 0.1-0.5 m / s.

[0023] A six-degree-of-freedom ship dynamics model was constructed to match the thrust-power curve of the propeller to the parameters of the actual ship.

[0024] A three-dimensional terrain model was constructed, and the terrain roughness was subdivided into 10 levels. Level 0 is a smooth and hard substrate, and Level 10 has an unevenness range of ≥2m and a slope standard deviation of >15°.

[0025] Real-time multidimensional data is input into the corresponding model to construct a real-time digital twin scenario.

[0026] Furthermore, in S2, the process of training the policy network using a three-stage curriculum learning strategy includes:

[0027] Basic terrain training: Configure the digital twin scene as flat terrain with no water flow interference. Only enable the dredging volume reward in the reward function. The training target is a dredging efficiency of at least 800m³. 3 / h;

[0028] Directional water flow adaptation: A constant flow velocity field of 5kN is introduced into the digital twin scenario, with randomized initialization of the flow direction. The terrain complexity is increased to level 3. An energy consumption negative penalty term and an energy consumption baseline are added to the reward function. The training objective is to maintain a dredging efficiency of no less than 600m³ under directional water flow conditions. 3 / h and energy consumption fluctuation rate less than 15%;

[0029] Complex Disturbance Enhancement: Random eddies and undercurrents are introduced into the digital twin scenario to increase ecological constraints and activate the ecological disturbance penalty term. The training objective is to maintain a dredging efficiency of no less than 400m under disturbance conditions. 3 / h, and the area of ​​ecological disturbance is less than 30%.

[0030] Furthermore, the expression for the hierarchical reward function is:

[0031]

[0032] Among them, V actual E represents the actual dredged volume. current For real-time energy consumption, A polluted V represents the area of ​​ecological disturbance. target E baseline A total These represent the target dredging volume, baseline energy consumption, and total area of ​​the work area, respectively; A, B, and C are adjustable weighting coefficients ranging from 0 to 1.

[0033] Furthermore, the target multimodal perception system in S3 includes a phased sonar array and a binocular vision module; the phased sonar array is used to model the spatial position of hard targets on the seabed; the binocular vision module is used to identify flexible obstacles using a YOLOv8s network.

[0034] Furthermore, in S3, the expression for the dynamic probability model is:

[0035]

[0036] Where d is the real-time detected distance between the ship and the obstacle, d safe The distance is the safety threshold, k is the sensitivity adjustment coefficient; P collision Let P be the collision probability. collision The value triggered a level three warning;

[0037] When collision risk exists, a dynamic window method is used to generate candidate obstacle avoidance paths with continuous curvature, and the optimal obstacle avoidance path is selected by optimizing the objective function, where the expression of the objective function is:

[0038] f(v,ω)=α·J dist +β·J vel +γ·J obs

[0039] Where α, β, and γ are weighting coefficients; J dist J is the path distance cost. vel For the cost of speed smoothing, J obs The cost of avoiding obstacles.

[0040] Furthermore, in S4, the objective function of the global optimization model is to minimize the sum of the times spent by all ships in completing their respective tasks;

[0041] The constraints include area coverage constraints, safety distance constraints, and load balancing constraints. Among them, the area coverage constraint requires that the union of the operating areas of all vessels must completely cover the total operating area, with a coverage rate of 100%. The safety distance constraint requires that the distance between any two different vessels must be no less than 50 meters. The load balancing constraint requires that the difference between the maximum and minimum dredging volume of a single vessel does not exceed 15% of the average dredging volume of all vessels.

[0042] Furthermore, S4 includes:

[0043] The work area is divided into n initial sub-regions, and initial task boundaries are assigned to each vessel.

[0044] Each vessel plans the optimal path within its assigned sub-region, calculates local variables, and collects global constraint violation information. The task boundaries of each vessel are updated using Lagrange multipliers until convergence. The convergence criteria are: task change between adjacent iterations <1%, maximum number of iterations reaches a preset value, or global objective function change rate <0.5%.

[0045] When a ship changes its path by more than 20% due to obstacle avoidance, or when an unmarked obstacle is detected in the task area, the trigger condition is considered to have been met. At this time, an incremental update strategy is used to redistribute the affected sub-regions, and a hot start strategy is used to inherit the multiplier parameters from the previous iteration until convergence, and a collaborative optimization scheme is output.

[0046] As can be seen from the above technical solution, compared with the prior art, the present invention has the following beneficial effects:

[0047] 1. This invention constructs a high-fidelity digital twin scenario based on multi-dimensional real-time data and trains the strategy network using a three-stage course learning strategy. During the training process, a hierarchical reward function mechanism is constructed by combining fluid dynamics constraint optimization and path energy consumption penalty mechanism to achieve dynamic weight adjustment of dredging efficiency, energy consumption and ecological protection. This reduces the unit dredging energy consumption from 0.45L / ton to 0.35L / ton, a reduction of 22%, saving 150 tons of diesel consumption per ship per year, corresponding to a CO2 emission reduction of 480 tons, resolving multi-objective optimization conflicts and improving path planning efficiency.

[0048] 2. This invention trains the policy network in a virtual simulation scenario and then transfers the trained policy network to a real ship. By using virtual-real transfer learning, the training cost of the real ship is reduced and the policy deployment cycle is shortened by up to 60%.

[0049] 3. This invention identifies obstacles through a multimodal target perception system, enabling the identification of both rigid targets on the seabed and flexible targets such as fishing nets and submarine pipelines. This forms a multimodal perception system that complements rigid and flexible targets, achieving an identification accuracy of over 95%. It can significantly reduce collision risks, decreasing the collision accident rate from 2.1 times / month to 0.2 times / month, a 90% reduction in risk. Combined with millimeter-level trajectory adjustment technology, it enables safe operation in all scenarios in complex underwater environments.

[0050] 4. In multi-ship collaborative operation scenarios, this invention allocates the operation area for each ship based on the ADMM distributed optimization algorithm. When the triggering condition is met, an incremental update strategy is used to dynamically redistribute tasks for each ship. This avoids the high cost and low efficiency of global recalculation, focuses only on the affected local area, and quickly responds to changes (such as path changes or the appearance of obstacles). While ensuring system flexibility, it significantly reduces the amount of computation, improves overall operating efficiency, and ensures that the system can adapt to new situations in a timely manner without wasting resources in unaffected areas, ensuring uninterrupted multi-ship collaborative operation. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0052] Figure 1 The overall architecture diagram of the reinforcement learning-based autonomous path planning method for dredging vessels provided by this invention is shown below.

[0053] Figure 2 A schematic diagram of multi-dimensional state space modeling provided by the present invention;

[0054] Figure 3 A schematic diagram illustrating the digital twin scenario construction, course learning, and transfer process provided by this invention;

[0055] Figure 4 A flowchart of the reinforcement learning algorithm provided by this invention;

[0056] Figure 5 The flowchart for multi-ship collaborative optimization provided by this invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] like Figure 1 As shown in the figure, this invention discloses an autonomous path planning method for dredging vessels based on reinforcement learning, including the following steps:

[0059] S1. Real-time acquisition of three-dimensional terrain data, water body reflectance spectrum data, dredging pump power data and three-dimensional flow velocity data, and after preprocessing, a multi-dimensional state space vector is obtained.

[0060] S2. Construct a digital twin scenario based on multi-dimensional state space vectors. In the digital twin scenario, train the policy network by increasing the environmental complexity and adopting a three-stage course learning strategy. Introduce Navier-Stokes residual constraints to enable the policy network to generate mixed control commands including dredging pump control, hull steering angle and speed. Dynamically adjust the path planning priority through a hierarchical reward function. Transfer the trained policy network to the ship entity.

[0061] S3. Generate discrete-continuous hybrid control commands and global path points for the ship based on the policy network, identify obstacles based on the target multimodal perception system, assess collision risk through a dynamic probability model, and reconstruct the local path when there is a collision risk.

[0062] S4. In the scenario of multi-ship collaborative operation, a global optimization model is constructed, an initial sub-region is allocated to each ship, and a distributed iterative solution is adopted. When the triggering condition is reached, an incremental update strategy is adopted to dynamically redistribute tasks to each ship until convergence, and the collaborative optimization scheme of each ship is output.

[0063] The following provides further explanation of each of the above steps.

[0064] S1. Initialization and data acquisition.

[0065] 1) Sonar System Configuration: The Reson SeaBat T50 multibeam side-scan sonar was deployed in the mid-bottom area of ​​the ship, set to a high-precision terrain scanning mode (resolution 0.5m, scanning range ±60°), generating a 3D elevation grid of the work area in real time, with an elevation accuracy error ≤ ±0.1m and a sampling frequency of 10Hz. Sonar data was transmitted to the main control unit via Ethernet at a sampling frequency of 10Hz.

[0066] 2) Multispectral Sensing Module: A Hyperspec III hyperspectral imaging system is mounted on the bow deck, covering the spectral range of 400-2500 nm (spectral resolution 5 nm), acquiring water reflectance spectral data at a rate of 0.5 seconds per frame. After clustering using the DBSCAN algorithm, several clusters are obtained, each representing a region with similar water reflectance spectral characteristics, which are closely related to sediment density. Based on a predefined sediment density classification standard (Level 1: loose, density 1.2-1.5 g / cm³), the sediment is then analyzed. 3 Grade 5: Dense, density > 1.8 g / cm³ 3 ), and analyze each cluster: 2

[0067] Feature statistics: Calculate the mean, variance, and other statistical measures of the spectral characteristics of sample points within each cluster, and analyze the correlation between these statistics and sediment density. 2

[0068] Density calibration: By comparing with samples of known sediment density (e.g., obtaining spectral data of sediment with different densities under laboratory conditions and establishing a mapping relationship between spectral characteristics and sediment density), a corresponding sediment density range is calibrated for each cluster. For example, if the spectral characteristics of a certain cluster correspond to a sediment density of 1.2-1.5 g / cm³ in the laboratory... 3 If the spectral characteristics of the samples are most similar, then the cluster is classified as Level 1 (loose); if the spectral characteristics of another cluster are most similar to those of the sediment density > 1.8 g / cm³, then the cluster is classified as Level 1 (loose); 3 Based on the matching of sample spectral characteristics, they are divided into 5 levels (dense). 2

[0069] Dynamic Adjustment: Because the aquatic environment is dynamic, sediment density also changes with factors such as water flow and sediment input. Therefore, the DBSCAN algorithm can process newly acquired spectral data in real time and dynamically adjust clusters based on the latest data distribution, thereby achieving dynamic classification of sediment density levels.

[0070] Through this closed-loop processing flow of "data acquisition - feature statistics - density calibration - dynamic adjustment", the DBSCAN algorithm achieves dynamic and adaptive classification of sediment density levels. Actual measurements have verified that its classification accuracy remains stable at over 93%, providing a reliable quantitative analysis tool for water sediment movement monitoring and ecological environment assessment.

[0071] 3) Energy consumption monitoring unit: A high-precision Hall sensor (accuracy ±0.5%) is used to collect the dredging pump power (range 50-500kW) in real time. The data is transmitted via CAN bus and the sampling frequency is 20Hz.

[0072] 4) ADCP current meter: integrates Nortek Signature 1000ADCP, measures three-dimensional flow velocity (u, v, w components, accuracy ±0.05m / s) at a water depth of 0-50m, with a data output frequency of 2Hz.

[0073] 5) Data preprocessing:

[0074] ① Spatiotemporal Registration and Noise Suppression: The spatiotemporal registration stage relies on the SLAM (Simultaneous Localization and Mapping) algorithm to construct a unified coordinate system for multiple sensors. First, seabed feature points (such as reef outline feature points and terrain undulation inflection points) are accurately extracted from sonar point cloud data. Simultaneously, water spectral feature regions in multispectral images are acquired. The optimal rigid body transformation matrix between the two modal data is solved through the Iterative Closest Point (ICP) algorithm, achieving sub-pixel-level precise alignment between the sonar 3D point cloud coordinate system and the multispectral image pixel coordinate system. For attitude deviations caused by hull motion, such as roll (±15°) and pitch (±10°), the SLAM algorithm uses an extended Kalman filter (EKF) to calculate the hull's six-degree-of-freedom attitude parameters in real time. Based on the Euler angle compensation model, the coordinate system transformation relationship is dynamically corrected to ensure that the spatial consistency error of multi-source data in the UTM projection coordinate system is controlled within ≤5cm.

[0075] This process effectively eliminates the spatial misalignment of cross-modal data caused by ship swaying, providing a unified spatial benchmark for high-precision fusion of terrain grids and sediment density distribution, thereby improving the accuracy of subsequent feature fusion by more than 40%.

[0076] In the noise suppression stage, Kalman filtering algorithm is used to recursively reduce noise based on the temporal dynamic characteristics of sonar data. Specifically, median filtering with a window size of 5×5 and a step size of 1m is used to process the original sonar elevation data to effectively eliminate isolated noise points with errors exceeding 0.3m, while fully preserving key terrain details such as gullies and steep slopes.

[0077] To address the frequency domain characteristics of the spectral data, the Savitzky-Golay algorithm with a window length of 11 and a polynomial order of 3 is used for smoothing. A differentiated filtering strategy is then employed to simultaneously improve the signal-to-noise ratio of the two types of data to over 20 dB.

[0078] ② Energy consumption data standardization: The raw power data of the sensor is normalized based on the energy conservation model.

[0079] (Unit: kW·h)

[0080] Where P actual The real-time power is P (kW), and the sampling interval is Δt = 10s. actual =500kW is the rated power.

[0081] ③ Flow vector analysis: The three-dimensional velocity vector measured by ADCP is decomposed into:

[0082] Flow direction angle: 0°-360° (0° is due north, increasing clockwise), resolution 1°;

[0083] Flow rate: 0-10 kN (1 kN ≈ 0.514 m / s), quantization accuracy 0.1 kN;

[0084] Vertical component: velocity range in the w direction ±2m / s, used to evaluate the stratification effect of water flow.

[0085] 6) Multidimensional feature extraction and fusion:

[0086] The preprocessed multidimensional data is input into a spatiotemporal convolutional neural network. This network architecture includes: an input layer receiving a 4-channel tensor composed of topographic elevation, sediment density, energy consumption, and water flow; a spatiotemporal convolutional layer extracting spatial-temporal correlation features from the data using 3×3×3 3D convolutional kernels; and a feature fusion layer employing a multi-head parallel attention mechanism to weightedly fuse the four-dimensional data (topography, sediment, energy consumption, and water flow), ultimately outputting a multidimensional feature vector to provide a highly robust feature representation for subsequent analysis. The specific implementation process of the multi-head parallel attention mechanism in the four-dimensional data fusion is as follows:

[0087] Input Feature Grouping and Mapping: The feature tensor output by the spatiotemporal convolutional layer covers four dimensions: terrain (including elevation, slope, and curvature), sediment density, energy consumption (characterized by pump power), and water flow (including three velocity components). These dimensions are C×H×W×T (corresponding to the number of channels, height, width, and time step, respectively). The grouping strategy divides the four-dimensional data into N parallel heads (N=8 here), with each parallel head independently processing all four dimensions of features. Specifically, different linear transformations (using weight matrices respectively) are applied. The input feature X is mapped to different subspaces, thereby generating the corresponding query vector. Key vector Value vector Where i represents the i-th parallel header.

[0088] Attention score calculation: Within each parallel head, the correlation weights between the four-dimensional features are calculated using a specific formula, as follows:

[0089]

[0090] Where, d k The dimension of the key vector is used for scaling operations to ensure gradient stability and avoid problems such as gradient explosion or vanishing during calculation, making model training more stable and efficient.

[0091] Feature Focus Logic: Different parallel heads emphasize different aspects of the correlation between features across various dimensions. For example, abrupt changes in elevation in terrain features may be strongly correlated with flow velocity direction in water flow features through a few parallel heads; while other parallel heads will focus on the synergistic relationship between sediment density and energy consumption. Through this differentiated focus mechanism, the model can more comprehensively capture the complex relationships between multi-dimensional features, achieving a deep fusion and understanding of features such as terrain, sediment density, energy consumption, and water flow, thereby improving the accuracy of the overall environmental state representation.

[0092] Weighted aggregation of value vectors: applying the calculated attention weights to the value vector V i This achieves weighted aggregation of value vectors, thereby generating fused features for each parallel head. The specific formula is as follows:

[0093] Head i =Attention i ·V i

[0094] This process enables feature fusion to fully consider the correlation weights between features of different dimensions, highlight key information, and enhance the effectiveness of feature representation.

[0095] Multi-dimensional fusion example: In multi-dimensional fusion, different parallel heads exhibit specific synergistic optimization effects. For example, in the coordination of terrain and water flow, one parallel head establishes a close correlation between "steep slope terrain" and "vertical flow velocity" by leveraging high attention weights, thereby optimizing the energy consumption of climbing in path planning and achieving more efficient operation. In the optimization of sediment and energy consumption, another parallel head focuses on strengthening the correspondence between "dense sediment areas" and "high pump power," thereby improving the efficiency of dredging operations and ensuring the rational use of resources and efficient task execution. Through this multi-dimensional weighted fusion mechanism, the model can integrate the advantages of features from various dimensions to achieve more accurate and efficient decision support.

[0096] In the multi-head concatenation stage, the outputs of all parallel heads are concatenated along the channel dimension, specifically represented as (MultiHead = Concat(Head1, Head2, ..., dots, Head...). N Then, a linear projection operation is performed, and the dimensionality of the stitched features is reduced to 128 dimensions through a fully connected layer. The mathematical expression is (Z =

[0097] MultiHead·W O ), of which (W O ) is the projection matrix, and the final output is... This process reduces the feature dimension while retaining key information, which improves the model's computational efficiency and ensures the compactness and effectiveness of feature representation.

[0098] 7) State Vector Construction and Update: Generate a multi-dimensional state vector containing terrain complexity (levels 0-10), sediment density (levels 1-5), energy consumption, water flow u / v / w components, and environmental dynamic parameters, such as... Figure 2 As shown, the environment status is updated every 10 seconds.

[0099] The topographic complexity is represented by a 4-dimensional vector, including topographic complexity score, elevation variance (degree of topographic relief), mean slope (overall tilt trend), and curvature extrema (local concavity and convexity features); sediment density is represented by a 3-dimensional vector, including density level (1-5), density gradient (difference between adjacent areas), and density stability (time series fluctuation); internal friction is represented by a 3-dimensional vector, including normalized instantaneous power value, average energy consumption and standard deviation over the past 10 seconds. The flow component contains a 6-dimensional vector with real-time values ​​of u, v, and w components, velocity change rate (acceleration), and flow direction angle stability (variance); environmental dynamic parameters contain a 2-dimensional vector including ecological disturbance index and obstacle proximity.

[0100] Finally, in the data encapsulation stage, feature engineering techniques are used to organically integrate the preprocessed terrain grid, sediment density labels, flow velocity vectors, and other multi-dimensional information into a state vector containing multi-dimensional features. With the help of the nanosecond-level transmission capability of gigabit Ethernet, this state vector is transmitted to the reinforcement learning decision module in real time.

[0101] S2, reinforcement learning training and online decision-making, the specific process is as follows: Figures 3-4 As shown.

[0102] 1) Digital Twin Scenario Construction:

[0103] Digital twin scenario construction is based on multi-source real-time data to build a high-fidelity simulation environment, providing physical constraints and dynamic inputs for reinforcement learning.

[0104] This invention constructs a high-fidelity physical constraint space using real-time data to ensure the realism of the simulation environment. During the phased training phase of the course, the policy network is optimized in a controllable progression of complexity by actively configuring environmental parameters. The two are decoupled through phase separation, which not only ensures the rigor of the training process but also lays the foundation for dynamic adaptation after deployment on the actual ship.

[0105] Specifically, this invention employs an architecture of "pre-setting environmental parameters during the training phase + calibrating the physical model with real-time data" to ensure decoupling between the progressively increasing environmental complexity of the phased training and the dynamic changes in real-time data. The core value of real-time data lies in defining the physically feasible domain and initial state of the simulation environment; it does not directly drive the training process, thereby ensuring the rigor of training while improving the generalization ability of the policy network to real-world scenarios.

[0106] This invention ensures the consistency between control commands generated in the simulation environment and the control of the actual ship's dredging pump. The core logic is: to calibrate the physical rules of the simulation model with real-time data, to bridge the simulation-reality gap with transfer learning, and to dynamically correct execution errors with closed-loop feedback, ultimately achieving a precise mapping between "virtual commands and actual ship execution".

[0107] When constructing a high-fidelity digital twin scene in the Unity-Mujoco co-simulation platform, a series of key technologies were adopted: fluid dynamics modeling based on the Navier-Stokes equations was used to construct a system capable of simulating a flow velocity range of 0-10 kN (Reynolds number Re = 1 × 10⁻⁶). 4 ~1×10 6 It supports dynamic flow fields for laminar and turbulent multimodal flows; it establishes a sediment settling model using the Stokes settling formula and a two-phase flow coupling algorithm, accurately defining sediment particle size as 0.1-5mm and settling velocity as 0.1-0.5m / s; it integrates a six-degree-of-freedom rigid body motion model to conduct ship dynamics simulation, making the thrust-power curve of the propeller match the parameters of the actual ship (maximum thrust 50kN, power response delay ≤0.2s); it realizes configurable terrain design, subdividing terrain roughness into 10 levels (level 0 is a smooth hard bottom, level 10 has an unevenness amplitude ≥2m and a slope standard deviation >15°), thereby meeting diverse simulation application needs.

[0108] All model input parameters are derived from real-time sensor data streams: ADCP flow velocity is injected into the fluid model after synchronization via the PTP protocol (timing error <10ms); hyperspectral sediment density is preprocessed using Savitzky-Golay filtering (window 11, order 3); and sonar topographic data undergoes noise removal via Kalman filtering (Q=0.01, R=0.1). This model-driven architecture based on measured data forms a closed loop of "real-time sensor acquisition - data preprocessing - multi-physics coupling - digital twin evolution," enabling the simulation scenario to respond to real-world environmental changes with a full-link latency of <200ms, providing multi-dimensional dynamic decision-making support for ship path planning, including flow field, settlement, and topography.

[0109] 2) Three-Stage Learning Strategy: The goal of this learning strategy is to train the policy network to adapt to various assignment requirements by progressively increasing the environmental complexity in stages. This module trains the policy network in three progressive stages, gradually increasing the environmental complexity:

[0110] ①Basic terrain training:

[0111] The scenario is configured as flat terrain (slope less than 5°) with no water flow interference; the reward function only enables the dredging volume reward (α = 1.0) to quickly establish basic path planning capabilities; the training target is a dredging efficiency of no less than 800m. 3 / h, the convergence steps do not exceed 500,000.

[0112] ② Adaptation to directional water flow:

[0113] The scenario was upgraded to include a constant flow velocity field of 5kN (randomly initialized flow direction), and the terrain complexity was increased to level 3. An energy consumption negative penalty term (β = 0.3) and an energy consumption baseline value were added to the reward function. The training objective was to maintain a dredging efficiency of no less than 600m³ under directional water flow conditions. 3 / h and energy consumption fluctuation rate less than 15%.

[0114] ③ Complex interference enhancement:

[0115] The scene enhancement involves superimposing random eddies (10 to 50 m in diameter, rotation speed 0.5 to 2 m / s) and undercurrents (velocity gradient 2 to 5 kN / m); ecological constraints are added, an ecological disturbance penalty term (γ = 0.1) is activated, and a pollution area threshold A is set. total =10 4 m 2 When conducting robustness verification, the strategy is required to maintain a dredging efficiency of no less than 400m³ under disturbance conditions. 3 / h, and the area of ​​ecological disturbance is less than 30%.

[0116] 3) Strategy Network Inference Stage: First, the Actor network generates discrete-continuous hybrid control commands, including dredging pump start / stop status, hull turning angle with an accuracy of ±0.5°, and travel speed with a resolution of 0.1kn, and outputs global path points every 5 seconds. Based on this, the Critic network introduces Navier-Stokes residuals (Navier-Stokes residuals reflect the degree of deviation between the current fluid motion state and the basic mechanical laws of viscous fluid motion) to simultaneously correct the fluid dynamics feasibility when evaluating the state value, thereby ensuring the physical consistency between the heading angle and the water flow vector, and providing state inputs that conform to the real physical characteristics for the reward function calculation.

[0117] 4) In the tiered reward calculation process, the dredging volume V is collected in real time. actual Energy consumption E current Ecological disturbance area a polluted Multi-dimensional data is then substituted into the reward function, and the expression is:

[0118]

[0119] Among them, V actual E represents the actual dredged volume. current For real-time energy consumption, A polluted V represents the area of ​​ecological disturbance. target E baseline A total These represent the target dredging volume, baseline energy consumption, and total operating area, respectively; A, B, and C are adjustable weighting coefficients ranging from 0 to 1. A represents the weighting of the actual dredging volume V. actual With the target dredging volume V target The weighting of the ratios reflects the importance attached to dredging efficiency; the larger the value of A, the more significant the positive impact of dredging efficiency on the total reward. B is used to measure current energy consumption E. current Compared with the baseline energy consumption E baseline The degree of penalty for the ratio: the larger the B value, the stronger the negative impact of energy consumption exceeding the benchmark value on the total reward; C reflects the impact on the polluted area A. polluted With total area A total The penalty weight of the ratio is such that the larger the C value, the more prominent the negative impact of ecological disturbance on the total reward.

[0120] This formula quantifies and weights dredging efficiency, energy consumption costs, and ecological impact, ultimately enabling dynamic adaptive adjustment of path planning priorities and forming a closed-loop progressive decision-making process of "state perception - physical constraint assessment - multi-objective reward calculation - control strategy optimization".

[0121] 5) Domain Randomized Transfer: The goal of domain randomized transfer is to transfer the policy network trained in simulation to the actual ship, solving the problem of "simulation-reality" domain differences. Random perturbations are applied to water transparency (attenuation coefficient of 0.5-5m) and sensor noise (Gaussian white noise σ=0.05). Through transfer learning, the simulation policy parameters are transferred to the actual ship, achieving a success rate of ≥85%.

[0122] This invention employs a two-stage architecture of "pre-trained model transfer + online dynamic calibration" to achieve efficient reuse and fine-tuning of policy parameters:

[0123] ① Initial parameter migration (cold start phase):

[0124] Model structure alignment: Maintain consistency between the simulation environment and the policy network architecture of the actual ship controller (both are 3-layer fully connected networks with 128 neurons per layer) to ensure parameter dimension matching.

[0125] Key parameter mapping: Physically relevant parameters such as fluid resistance coefficient and sediment grabbing efficiency learned in the simulation are directly transferred to the Mujoco engine parameter table of the actual ship dynamics model; the weights (α / β / γ) of the hierarchical reward function are normalized and the initial values ​​are dynamically adjusted based on the ecological sensitivity level of the actual ship operation area.

[0126] Zero-sample testing: During the first deployment on the actual vessel, the automatic control mode was turned off, and the historical sensor data of the actual vessel (including 1,000 sets of complex water condition samples) was played back offline to verify whether the control commands output by the strategy network (such as steering angle and pump power) conformed to the principles of hydrodynamics (error ≤15%).

[0127] ② Online calibration optimization (adaptive phase):

[0128] Physical constraint closed loop: The ship's roll / pitch angle is measured in real time by the IMU mounted on the actual ship and compared with the predicted value of the simulation model. If the deviation is >5°, the boundary condition correction of the Navier-Stokes equation is triggered (such as increasing the ship's drag coefficient).

[0129] The reward function is closed-loop: the actual dredging volume, energy consumption, and ecological disturbance area are statistically analyzed every hour, the deviation from the target value is calculated, and the hierarchical reward weights α / β / γ are automatically adjusted to form a "measurement-evaluation-re-optimization" cycle.

[0130] ③ Ship-to-ship compatibility:

[0131] Spatiotemporal calibration technology: PTP (Precise Time Protocol) is used to achieve nanosecond-level synchronization between the simulation clock and the actual ship sensor clock, ensuring that the timing error of the 32-dimensional state vector (including terrain, flow velocity, and energy consumption data) input to the strategy network is <10ms; the mapping relationship between the actual ship coordinate system and the simulation global coordinate system is established through the SLAM algorithm, and the spatial alignment error between the measured sonar point cloud and the simulated terrain grid is ≤10cm.

[0132] Hardware-in-the-loop testing: Before deployment on the actual ship, the simulation scenario is connected to the actual ship controller through the ROS2 middleware to conduct a 12-hour continuous hardware-in-the-loop test (HIL Test) to verify the bidirectional transmission reliability of sensor data streams and control command streams (packet loss rate <0.1%).

[0133] Fault tolerance mechanism design: If the migrated strategy fails to perform in the actual ship (such as failing to avoid obstacles 3 times in a row), it will automatically switch to "simulation prediction mode": using the Unity-Mujoco model to predict the ship's state in the next 5 seconds in real time and generate emergency control commands, with a maximum fault tolerance time of 30 seconds.

[0134] S3, Dynamic Obstacle Avoidance Execution:

[0135] 1) Generate discrete-continuous hybrid control commands (including dredging pump control, hull steering angle and speed) and global path points for ships based on the strategy network.

[0136] 2) Obstacle detection and classification: At the environmental perception level, the phased array sonar array scans the area 200m ahead, constructing a 3D point cloud model of rigid obstacles such as reefs and shipwrecks at a resolution of 0.5m, thus achieving accurate spatial modeling of the hard targets on the seabed; at the same time, the binocular vision module acquires image data with a resolution of 1920×1080, which is integrated into the YOLOv8s model to identify flexible targets such as fishing nets and seabed pipelines. Its average accuracy (mAP@0.5) at an IOU threshold of 0.5 reaches more than 95%, thus forming a multimodal perception system that complements rigid and flexible targets.

[0137] 3) Collision risk assessment: A dynamic probability model is constructed based on the logistic function, with the following expression:

[0138]

[0139] Among them, the safe distance threshold d safe =50m, sensitivity adjustment coefficient k=0.5, parameter d represents the real-time detected distance between the ship and the obstacle (unit: meters), and the calculation results trigger a level three early warning mechanism (yellow warning: 0.3≤P collision <0.6; Orange alert: 0.6≤P collision <0.8; Red alert: P collision ≥0.8).

[0140] 4) Path replanning phase:

[0141] When collision risk exists, a dynamic window method is used to generate candidate obstacle avoidance paths with continuous curvature, and the optimal obstacle avoidance path is selected by optimizing the objective function, where the expression of the objective function is:

[0142] f(v,ω)=α·J dist +β·J vel +γ·J obs

[0143] Where α, β, and γ are weighting coefficients; J dist J is the path distance cost. vel For the cost of speed smoothing, J obs The obstacle avoidance cost is defined as follows: α is the path distance cost weight, reflecting the importance of the distance between the current path and the target path (or destination); β is the speed smoothing cost weight, reflecting the smoothness of speed changes; and γ is the obstacle avoidance cost weight, representing the priority of obstacle avoidance behavior.

[0144] Local path reconstruction is completed within 200ms, ensuring that the ship performs obstacle avoidance maneuvers with a speed fluctuation of ≤0.1kn and a steering error of ±1°, achieving millimeter-level precision trajectory adjustment and smooth connection of the global path.

[0145] The process of optimizing the objective function is as follows:

[0146] Candidate solution generation stage: First, based on the physical constraints of the current speed and angular velocity (including maximum acceleration, braking distance, etc.), feasible speed and angular velocity combination windows need to be generated; then, the cost calculation stage begins, where for each candidate speed combination, the corresponding path distance cost (such as the Euclidean distance from the current path to the target point or path tracking deviation), speed smoothing cost (i.e., the rate of change of speed, such as the square of acceleration or angular acceleration, used to penalize non-smooth motion) and obstacle avoidance cost (usually inversely proportional to the distance to the nearest obstacle, such as the reciprocal of the square of the distance) are calculated.

[0147] Next, each cost term is substituted into the objective function for weighted summation to calculate the total cost of each candidate solution, and the combination of velocity and angular velocity with the minimum total cost is selected as the final control command.

[0148] In addition, parameter tuning strategies, as optimization methods, include empirical methods (manually adjusting the weights of each cost item according to the needs of the scenario, such as increasing the weight of obstacle avoidance cost in obstacle avoidance scenarios), automatic optimization (using algorithms such as genetic algorithms and particle swarm optimization to search for the optimal weight combination to maximize dredging efficiency or minimize energy consumption), and adaptive adjustment (adjusting the weights in real time according to dynamic environmental changes such as increased water flow or increased obstacle density, such as increasing the weight of obstacle avoidance cost as the collision risk increases), thereby ensuring that the entire control process maintains optimal performance in different scenarios.

[0149] S4. Optimization of multi-vehicle collaborative operations; specific process as follows: Figure 5 As shown.

[0150] 1) Global optimization model building:

[0151] Objective function: Minimize the sum of times all ships take to complete their respective tasks, expressed as:

[0152] (n is the number of ships, T) i )

[0153] Where T i This represents the time taken for the i-th ship to complete a specific task (such as dredging, navigation, etc.).

[0154] Constraints:

[0155] ①Regional coverage constraints: (100% coverage)

[0156] This means that the union of all vessel operation areas must completely cover the total operation area, ensuring no omissions and achieving 100% coverage. Represents the respective operating areas of n ships. i The set of ; TotalArea represents the total operating area; n represents the number of vessels participating in the operation.

[0157] ②Safety distance constraints:

[0158] For any two different vessels i and j, the distance between them must be no less than 50 meters to ensure operational safety and avoid the risk of collision.

[0159] ③ Load balancing constraints: [max i V i -min i V i ≤15%×Mean(V)(V i (Dredging volume per vessel)

[0160] This indicates that the difference between the maximum and minimum dredging volume of a single vessel must not exceed 15% of the average dredging volume of all vessels to ensure a relatively balanced dredging workload for each vessel and avoid some vessels being overworked or idle. Where V i This represents the amount of dredging done by the i-th vessel.

[0161] 2) Distributed iterative solution:

[0162] Initialization: The central controller divides the work area into n initial sub-regions (based on Voronoi diagrams), and each vessel receives the initial task boundary (the initial task boundary refers to the geometric boundary of the initial work sub-region assigned to each dredging vessel by the Voronoi diagram partitioning algorithm).

[0163] Iterative optimization: Each vessel solves its subproblem locally, plans the optimal path within its allocated area, and calculates local variables (operation time, dredging volume). The central controller collects information on violations of global constraints (such as overlapping areas or insufficient safety distance). It updates the task boundaries of each vessel using Lagrange multipliers (in distributed optimization of multi-vessel collaborative operations, the Lagrange multiplier method transforms global constraints into penalty terms for local optimization problems through a decomposition and coordination mechanism, gradually adjusting the task boundaries to meet safety and efficiency requirements) until convergence (change in task between adjacent iterations < 1%). Convergence criterion: Termination occurs when the maximum number of iterations is 50 or the rate of change of the global objective function is < 0.5%.

[0164] 3) Dynamic task reallocation:

[0165] Triggering conditions: When a ship changes its path by more than 20% due to obstacle avoidance (deviation from the original planned path > 100m), or when unmarked obstacles (such as temporary fishing nets) are detected in the task area, an incremental update strategy is adopted to redistribute the affected sub-regions. A hot start strategy is adopted to inherit the multiplier parameters of the previous iteration until convergence, and a collaborative optimization scheme is output.

[0166] Incremental updates are a highly efficient local optimization strategy. When faced with task changes, the central controller only reallocates and adjusts resources within the affected sub-region (e.g., within a 500m radius), rather than recalculating the entire global region. This approach avoids the high cost and inefficiency of global recalculation, focusing only on the affected local area. It allows for rapid response to changes (such as path alterations or obstacle appearances), significantly reducing computational load and improving overall operational efficiency while maintaining system flexibility. This ensures the system can adapt to new situations promptly without wasting resources in unaffected areas.

[0167] By employing a hot-start strategy that inherits the multiplier parameters from the previous iteration, the convergence time is reduced by 70% compared to a full update. In a 10-ship scenario, the redistribution time is ≤1.5 seconds, ensuring uninterrupted multi-ship collaborative operations. The hot-start strategy is an efficient method for optimizing the iteration process. Its core lies in inheriting key parameters (such as multiplier parameters) from the previous iteration, avoiding the tedious process of calculating all parameters from scratch during a full update. In multi-ship collaborative operation scenarios, this strategy manifests as directly using the multiplier parameters calculated in previous iterations, without needing to re-derive their initial values ​​or perform a full recalculation.

[0168] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0169] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for autonomous path planning of a dredging vessel based on reinforcement learning, characterized in that, Includes the following steps: S1. Real-time acquisition of three-dimensional terrain data, water body reflectance spectrum data, dredging pump power data and three-dimensional flow velocity data, and after preprocessing, a multi-dimensional state space vector is obtained. S2. Construct a digital twin scenario based on multi-dimensional state space vectors. In the digital twin scenario, train the policy network by increasing the environmental complexity and adopting a three-stage course learning strategy. Introduce Navier-Stokes residual constraints and dynamically adjust the path planning priority through a hierarchical reward function. Transfer the trained policy network to the ship entity. S3. Generate discrete-continuous hybrid control commands and global path points for the ship based on the policy network, identify obstacles based on the target multimodal perception system, assess collision risk through a dynamic probability model, and reconstruct the local path when there is a collision risk. S4. In the scenario of multi-ship collaborative operation, a global optimization model is constructed, an initial sub-region is allocated to each ship, and a distributed iterative solution is adopted. When the triggering condition is reached, an incremental update strategy is adopted to dynamically redistribute tasks to each ship until convergence, and the collaborative optimization scheme of each ship is output.

2. The dredging vessel autonomous path planning method based on reinforcement learning according to claim 1, characterized in that, The process of acquiring various data in S1 includes: Deploy multibeam side-scan sonar to the mid-bottom area of ​​the ship to generate three-dimensional terrain data of the work area in real time. The hyperspectral imaging system was mounted on the bow deck to collect water reflectance spectral data of a preset area. The water reflectance spectral data was clustered using the DBSCAN algorithm to obtain several clusters. Each cluster represents an area with similar water reflectance spectral characteristics. The sediment density level corresponding to each cluster was determined according to a predefined sediment density level standard. The power data of the dredging pump is collected in real time using a Hall sensor; Three-dimensional flow velocity data within a preset water depth range are measured using an ADCP flow meter.

3. The dredging vessel autonomous path planning method based on reinforcement learning according to claim 1, characterized in that, In S1, the methods for preprocessing various types of data include: After spatiotemporal registration of the 3D terrain data and water reflectance spectral data, noise suppression is then performed. The dredging pump power was normalized, and the three-dimensional flow velocity data was vector decomposed. Based on the spatiotemporal convolutional neural network, feature extraction is performed on the processed 3D terrain data, water body reflectance spectrum data, dredging pump power and 3D flow velocity data. A multi-head parallel attention mechanism is introduced to fuse the data of each dimension, resulting in a multi-dimensional state space vector containing terrain complexity, sediment density level, energy consumption value, water flow component and environmental dynamic parameters.

4. The dredging vessel autonomous path planning method based on reinforcement learning according to claim 1, characterized in that, In S2, the process of constructing a digital twin scenario includes: A fluid dynamics model based on the Navier-Stokes equations is constructed to simulate the flow velocity range of 0-10kN and supports dynamic water flow fields for laminar and turbulent multimodal flows. A sediment settling model was established using the Stokes settling formula and a two-phase flow coupling algorithm, defining the sediment particle size as 0.1-5 mm and the settling velocity as 0.1-0.5 m / s. A six-degree-of-freedom ship dynamics model was constructed to match the thrust-power curve of the propeller to the parameters of the actual ship. A three-dimensional terrain model was constructed, and the terrain roughness was subdivided into 10 levels. Level 0 is a smooth and hard substrate, and Level 10 has an unevenness range of ≥2m and a slope standard deviation of >15°. Real-time multidimensional data is input into the corresponding model to construct a real-time digital twin scenario.

5. The dredging vessel autonomous path planning method based on reinforcement learning according to claim 1, characterized in that, In S2, the process of training the policy network using a three-stage curriculum learning strategy includes: Basic terrain training: Configure the digital twin scene as flat terrain with no water flow interference. Only enable the dredging volume reward in the reward function. The training target is a dredging efficiency of at least 800m³. 3 / h; Directional water flow adaptation: A constant flow velocity field of 5kN is introduced into the digital twin scenario, with randomized initialization of the flow direction. The terrain complexity is increased to level 3. An energy consumption negative penalty term and an energy consumption baseline are added to the reward function. The training objective is to maintain a dredging efficiency of no less than 600m³ under directional water flow conditions. 3 / h and energy consumption fluctuation rate less than 15%; Complex Disturbance Enhancement: Random eddies and undercurrents are introduced into the digital twin scenario to increase ecological constraints and activate the ecological disturbance penalty term. The training objective is to maintain a dredging efficiency of no less than 400m under disturbance conditions. 3 / h, and the area of ​​ecological disturbance is less than 30%.

6. The dredging vessel autonomous path planning method based on reinforcement learning according to claim 5, characterized in that, The expression for the tiered reward function is: Among them, V actual E represents the actual dredged volume. current For real-time energy consumption, A polluted V represents the area of ​​ecological disturbance. target E baseline A total These represent the target dredging volume, baseline energy consumption, and total area of ​​the work area, respectively; A, B, and C are adjustable weighting coefficients ranging from 0 to 1.

7. The dredging vessel autonomous path planning method based on reinforcement learning according to claim 1, characterized in that, The target multimodal perception system in S3 includes a phased sonar array and a binocular vision module; the phased sonar array is used to model the spatial position of hard targets on the seabed; the binocular vision module is used to identify flexible obstacles using a YOLOv8s network.

8. The autonomous path planning method for dredging vessels based on reinforcement learning according to claim 1, characterized in that, In S3, the expression for the dynamic probability model is: Where d is the real-time detected distance between the ship and the obstacle, d safe The distance is the safety threshold, k is the sensitivity adjustment coefficient; P collision Let P be the collision probability. collision The value triggered a level three warning; When collision risk exists, a dynamic window method is used to generate candidate obstacle avoidance paths with continuous curvature, and the optimal obstacle avoidance path is selected by optimizing the objective function, where the expression of the objective function is: f(v,ω)=α·J dist +β·J vel +γ·J obs Where α, β, and γ are weighting coefficients; J dist J is the path distance cost. vel For the cost of speed smoothing, J obs The cost of avoiding obstacles.

9. The dredging vessel autonomous path planning method based on reinforcement learning according to claim 1, characterized in that, In S4, the objective function of the global optimization model is to minimize the sum of the times that all ships take to complete their respective tasks; The constraints include area coverage constraints, safety distance constraints, and load balancing constraints. Among them, the area coverage constraint requires that the union of the operating areas of all vessels must completely cover the total operating area, with a coverage rate of 100%. The safety distance constraint requires that the distance between any two different vessels must be no less than 50 meters. The load balancing constraint requires that the difference between the maximum and minimum dredging volume of a single vessel does not exceed 15% of the average dredging volume of all vessels.

10. The autonomous path planning method for dredging vessels based on reinforcement learning according to claim 1, characterized in that, S4 include: The work area is divided into n initial sub-regions, and initial task boundaries are assigned to each vessel. Each vessel plans the optimal path within its assigned sub-region, calculates local variables, and collects global constraint violation information. The task boundaries of each vessel are updated using Lagrange multipliers until convergence. The convergence criteria are: task change between adjacent iterations <1%, maximum number of iterations reaches a preset value, or global objective function change rate <0.5%. When a ship changes its path by more than 20% due to obstacle avoidance, or when an unmarked obstacle is detected in the task area, the trigger condition is considered to have been met. At this time, an incremental update strategy is used to redistribute the affected sub-regions, and a hot start strategy is used to inherit the multiplier parameters from the previous iteration until convergence, and a collaborative optimization scheme is output.

Images