Dredging process control optimization method based on data driving

Abstract

The invention discloses a drag suction dredger dredging process control optimization method based on data driving. The method specifically comprises the steps that dredging state parameters, dredging action parameters and environment parameters are obtained through a ship monitoring system; designing a reward function to be associated with the historical data set to form a historical experience playback memory bank, and extracting data from the historical experience playback memory bank to form a historical data training set; mixing and extracting data in the historical data training set and the current data training set, and further training the meta reinforcement learning model to obtain a greedy reinforcement model; and carrying out dredging control on the ship by combining the greedy enhancement model with the current dredging state parameters. According to the method, state parameters and environment parameters of the trailing suction dredger are obtained, and dredging process control optimization parameters are obtained through a trained model; and according to the obtained parameters, related equipment of the trailing suction dredger is controlled so as to drive dredging equipment to reach the optimal working condition point, and the purpose of dredging optimization intelligent control is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent dredging, and in particular relates to a dredging process control optimization method based on data-driven. Background Art

[0002] Dredging is an underwater operation with many process control parameters. At present, the degree of automation of domestic dredging is not high, and manual operation is still the main method. However, due to the complex factors affecting the flow state of mud, such as mud concentration, mud flow rate, sediment particle size, sedimentation rate of different sediments and pipeline performance, even experienced operators cannot make targeted and efficient dredging actions quickly, resulting in low production and low efficiency, high energy consumption and high emissions. Therefore, it is particularly urgent to improve the degree of automation of dredging today. Intelligent dredging is an advanced technology that integrates modern information technology, automatic control technology, sensor technology, artificial intelligence algorithms, etc. and applies them to dredging projects. It aims to improve the efficiency, accuracy and safety of dredging operations, but the existing intelligent dredging methods are still not effective in actual dredging processes due to problems such as model overfitting caused by a single data set. Summary of the invention

[0003] In order to overcome the problems of model overfitting caused by a single data set in the prior art intelligent dredging method, the present invention provides the following technical solutions:

[0004] A data-driven dredging process control optimization method for a trailing suction dredger comprises the following steps:

[0005] S1- Acquisition of initial data:

[0006] Obtain dredging status parameters, dredging action parameters and environmental parameters through the ship monitoring system;

[0007] S2-Formation of historical data training set:

[0008] The dredging state parameters of adjacent time intervals and the dredging action parameters within the time interval are integrated as a historical data set, a reward function is designed to be associated with the historical data set to form a historical experience playback memory library, and data is extracted from the historical experience playback memory library to form a historical data training set;

[0009] S3-Meta-RL model training:

[0010] Build an initial reinforcement learning model, and use the historical data training set to train the initial reinforcement learning model to obtain a meta-reinforcement learning model;

[0011] S4- Exploration in the early stage of dredging:

[0012] In the early stage of dredging, the meta-reinforcement learning model is used to conduct environmental exploration of the actual dredging site to obtain the current data training set;

[0013] S5- Training of Greedy Reinforcement Model:

[0014] Mix and extract data from the historical data training set and the current data training set to further train the meta-reinforcement learning model and obtain the greedy reinforcement model;

[0015] S6-Controlled Field Dredging:

[0016] The greedy reinforcement model is used to control the dredging of ships in combination with the current dredging state parameters.

[0017] Furthermore, in the step S1-initial data acquisition, the dredging state parameters include the raking lip angle, the mud pump speed and the ship speed; the dredging action parameters are the increase, decrease or unchanged of the dredging state parameters; and the environmental parameters include soil quality and digging depth.

[0018] Furthermore, the step S2-forming a historical data training set comprises the following steps:

[0019] S2.1- Setting Geo-Fence for Dredging Status Parameters:

[0020] According to the different types of ship power systems and dredging equipment, set up electronic fences for dredging status parameters;

[0021] S2.2- Discretization of dredging state parameters:

[0022] According to the electronic fence of dredging state parameters, the dredging state parameters are discretized;

[0023] S2.3- Discretization of dredging action parameters:

[0024] The dredging action parameters correspond to the increase, decrease or unchanged of the dredging state parameters, and the action set is obtained accordingly;

[0025] S2.4-Integration of historical data sets:

[0026] Integrate the discretized dredging state parameters and action sets to form a historical data set;

[0027] S2.5- Design of reward function:

[0028] Design reward functions based on the absolute value of the vacuum;

[0029] S2.6-Formation of historical experience playback memory bank:

[0030] Based on the dredging state parameters, action sets and reward function values ​​corresponding to the same moment and the dredging state parameters of the next moment, a historical experience playback memory library is formed;

[0031] S2.7-Formation of historical data training set:

[0032] Extract historical experience and replay data in the memory bank to form a historical data training set.

[0033] Furthermore, the step S3-training of the meta-reinforcement learning model comprises the following steps:

[0034] S3.1- Construction of initial reinforcement learning model:

[0035] Building an initial reinforcement learning model based on a neural network, wherein the initial reinforcement learning model includes a current Q network and a target Q network;

[0036] S3.2- Initialization of the initial reinforcement learning model:

[0037] Let θ = θ′ to complete the initialization of the initial reinforcement learning model;

[0038] S3.3-Setting of preset conditions:

[0039] The first preset condition is the number of iterations K of updating the θ′ parameter;

[0040] The second preset condition is the maximum update times x or y of the θ parameter or the θ′ parameter;

[0041] S3.4-Model training:

[0042] Based on the historical data training set, the θ parameter is calculated and updated by a back propagation algorithm; when the first preset condition is met, the θ parameter is copied to the θ′ parameter;

[0043] When the second preset condition is met, the model stops training and the current values ​​of the θ parameter and the θ′ parameter are determined as the parameters of the meta-reinforcement learning model.

[0044] Furthermore, the step S4-exploration in the early stage of dredging includes the following steps:

[0045] S4.1-Meta-RL model output actions:

[0046] Select the action set corresponding to the largest action value as the output action of the meta-reinforcement learning model;

[0047] S4.2- Setting of exploration function:

[0048] Design the exploration function μ based on the greedy algorithm;

[0049] S4.3-Determination of dredging actions:

[0050] Selecting the output action as the dredging action with μ probability;

[0051] A random action in the action space is used as a dredging action with a probability of 1-μ;

[0052] Control the dredging machine movement through the selected dredging action;

[0053] S4.4-Formation of current data training set:

[0054] The current data training set is formed based on the dredging action, the dredging state parameters before and after the dredging action, and the reward function value corresponding to the dredging action.

[0055] Furthermore, in the step S5-greedy reinforcement model training, data from the current data training set and the historical data training set are mixed and extracted to further train the meta-reinforcement learning model to obtain the greedy reinforcement model.

[0056] Furthermore, the step S2-forming of the training set further includes the following steps:

[0057] S2.8-Data classification:

[0058] According to the environmental parameters obtained in step S1, different working conditions corresponding to different soil types and excavation depths are divided, and in combination with different working conditions and different ship types, the historical experience replay memory library obtained in step S2.6 is divided into data. In step S2.7, the data of the same working condition of the same ship type in the historical experience replay memory library are extracted for model training, so that the model can be pre-adjusted according to different working conditions and ship types, thereby achieving rapid convergence of the model.

[0059] The present invention obtains the state parameters and environmental parameters of the trailing suction hopper dredger and uses the trained model to obtain the optimization parameters of the dredging process control; according to the obtained parameters, the trailing suction hopper dredger related equipment is controlled to drive the dredging equipment to reach the optimal operating point, thereby achieving the purpose of dredging optimization intelligent control. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a schematic diagram of the overall flow of the data-driven dredging process control optimization method of the present invention. DETAILED DESCRIPTION

[0061] The technical solution of the present invention is described clearly and completely with specific embodiments in conjunction with the accompanying drawings. It should be understood that the described embodiments are only used to illustrate the present invention, but not to limit the scope of the present invention.

[0062] Embodiment 1:

[0063] A data-driven optimization method for dredging process control of a trailing suction hopper dredger Figure 1 As shown, the following steps are included:

[0064] S1- Acquisition of initial data:

[0065] Obtain dredging status parameters, dredging action parameters and environmental parameters through the ship monitoring system;

[0066] S2-Formation of historical data training set:

[0067] The dredging state parameters of adjacent time intervals and the dredging action parameters within the time interval are integrated as a historical data set, a reward function is designed to be linked to the historical data set to form a historical experience replay memory library, and data is extracted from the historical experience replay memory library to form a historical data training set.

[0068] S3-Meta-RL model training:

[0069] Build an initial reinforcement learning model, use the historical data training set to train the initial reinforcement learning model, and obtain the meta-reinforcement learning model.

[0070] S4- Exploration in the early stage of dredging:

[0071] In the early stage of dredging, the meta-reinforcement learning model is used to conduct environmental exploration of the actual dredging site to obtain the current data training set.

[0072] S5- Training of Greedy Reinforcement Model:

[0073] The data from the historical data training set and the current data training set are mixed and further trained on the meta-reinforcement learning model to obtain the greedy reinforcement model.

[0074] S6-Controlled Field Dredging:

[0075] The greedy reinforcement model is used to control the dredging of ships in combination with the current dredging state parameters.

[0076] Further, in the step S1-initial data acquisition, the dredging state parameters include the raking lip angle, the mud pump speed and the ship speed; the dredging action parameters are the increase, decrease or unchanged of the dredging state parameters; the environmental parameters include soil quality and digging depth;

[0077] Furthermore, the step S2-forming a historical data training set comprises the following steps:

[0078] S2.1- Setting Geo-Fence for Dredging Status Parameters:

[0079] According to the different types of ship power systems and dredging equipment, set up electronic fences for dredging status parameters;

[0080] For example: the maximum value of the rake lip angle is 15 degrees and the minimum value is 0 degrees, then the electronic fence of the rake lip angle is set to 0-15 degrees;

[0081] S2.2- Discretization of dredging state parameters:

[0082] According to the electronic fence of dredging state parameters, the dredging state parameters are discretized;

[0083] For example: the electronic fence with rake lip angle is 0-15 degrees, which is discretized into {0,1,2…15};

[0084] S2.3- Discretization of dredging action parameters:

[0085] The dredging action parameters correspond to the increase, decrease or unchanged of the raking lip angle, the mud pump speed, and the ship speed. The action set obtained accordingly includes three different actions of the three dredging state parameters, that is, it contains 27 different actions;

[0086] The action set is {000; 001; 00-1; 010; 011; 01-1; ... 111; -1-1-1};

[0087] Among them, 0 represents the action remains unchanged, 1 represents the increase of the corresponding parameter, and -1 represents the decrease of the corresponding parameter.

[0088] S2.4-Integration of historical data sets:

[0089] Integrate the discretized dredging state parameters and action sets to form a historical data set (s t , a t ,s t+1 ),

[0090] Among them, s t is the dredging state parameter at time t, a t is the action set at time t, s t+1 for a t Dredging state parameters at the next moment after the action;

[0091] S2.5- Design of reward function:

[0092] According to the vacuum absolute value Vam, the following reward function r is designed:

[0093]

[0094] The vacuum absolute value Vam is chosen instead of the output commonly used in other methods because the installation position of the output meter and the suction port are far apart in actual operation, resulting in data delay. To eliminate this effect, the vacuum absolute value Vam, which is proportional to the output, is chosen as the basis for designing the reward function r.

[0095] The meaning of the reward function r is: when the absolute value of vacuum is greater than a1, it means that the production has approached the maximum capacity of the ship, and the higher the number, the greater the production; when the absolute value of vacuum is greater than a2, r = 0, as a protection measure; a1 and a2 are determined according to the capacity and mechanism model of the trailing suction hopper ship, and are not restricted in this method.

[0096] S2.6-Formation of historical experience playback memory bank:

[0097] The reward function value r corresponding to time t t Add the historical data set (s t , a t ,s t+1 ), forming a historical experience playback memory bank (s t , a t , r t ,s t+1 ).

[0098] S2.7-Formation of historical data training set:

[0099] Extract the data in the historical experience playback memory library to form a historical data training set (s t , a t , r t ,s t+1 ).

[0100] Furthermore, the step S3-training of the meta-reinforcement learning model comprises the following steps:

[0101] S3.1- Construction of initial reinforcement learning model:

[0102] Building an initial reinforcement learning model based on a neural network, wherein the initial reinforcement learning model includes a current Q network and a target Q network;

[0103] The current Q network uses the θ parameter to predict the action Q value under the current strategy, and the target Q network uses the θ′ parameter to provide a stable target Q value; the θ′ parameter is a regularly updated copy of the θ parameter;

[0104] S3.2- Initialization of the initial reinforcement learning model:

[0105] Let θ = θ′ to complete the initialization of the initial reinforcement learning model;

[0106] S3.3-Setting of preset conditions:

[0107] In order to prevent the model from overfitting and ensure the learning depth of the model, the first preset condition and the second preset condition are set as stop signals for model training. Once the first preset condition and the second preset condition are met at the same time, the model stops training and is saved;

[0108] The first preset condition is the number of iterations K. Once the iteration of the model reaches the preset number of iterations K, it is considered that the first preset condition is met, and the θ parameter is copied to the θ′ parameter to achieve regular update of the θ′ parameter;

[0109] The second preset condition is: the θ parameter is updated x times / θ′ parameter is updated y times; wherein x is the maximum update number of the θ parameter; y is the maximum update number of the θ′ parameter; once the second preset condition is met, that is, any one of the maximum update numbers of the θ parameter or the θ′ parameter meets the condition, the model stops training.

[0110] S3.4-Model training:

[0111] Based on the historical data training set, the θ parameter is calculated and updated by a back propagation algorithm; when the first preset condition is met, the θ parameter is copied to the θ′ parameter; the back propagation algorithm is a conventional technical means for training models in the prior art and will not be described in detail;

[0112] When the second preset condition is met, the model stops training, and the current values ​​of the θ parameter and the θ′ parameter are determined as the parameters of the meta-reinforcement learning model to obtain the meta-reinforcement learning model.

[0113] Furthermore, the step S4-exploration in the early stage of dredging includes the following steps:

[0114] S4.1-Meta-RL model output actions:

[0115] Calculate the target Q network s t+1 The Q values ​​of all action sets in the state, select the action set at corresponding to the maximum Q value as the output action a of the meta-reinforcement learning model max ; The calculation formula of the Q value is as follows:

[0116] Q=r+δmaxQ(s t+1 , a t )

[0117] in:

[0118] Q is the action set a t The action value of r is the action set a t The corresponding reward value; δ is the discount factor, the value range is [0,1], which can be set by yourself; maxQ(s t+1 , a t ) is the next state s t+1 The set of all possible actions a t The expected value of is given by the target Q network;

[0119] S4.2- Setting of exploration function:

[0120] Based on the greedy algorithm, the following exploration function μ is designed:

[0121]

[0122] Where, μ is the exploration degree, and its value range is [0, 1]; μ0 is the initial exploration degree; R is the current round number; R max is the maximum number of exploration rounds;

[0123] S4.3-Determination of dredging actions:

[0124] Under the current situation, select the output action a with probability μ max As a dredging action t ′, a random action in the action space is taken as the dredging action a with a probability of 1-μ t ′; through the selected dredging action a t ′Control the movement of dredging equipment.

[0125] S4.4-Formation of current data training set:

[0126] The dredging state parameter before the dredging action at′ is taken as s′, and the dredging state parameter before the dredging action a t The dredging state parameter after the action is taken as s t+1 ′, with dredging action a t ′The corresponding reward function value is taken as r t ′, forming the current data training set (s′, a t ′,r t ′,s t+1 ′).

[0127] Furthermore, in the step S5-greedy reinforcement model training, the current data training set (s′, r t ′,a t ′,s t+1 ′) and historical data training set (s t , a t , r t ,s t+1 ) is used to further train the meta-reinforcement learning model to obtain a greedy reinforcement model. The training steps of the greedy reinforcement model are the same as those of the meta-reinforcement learning model and will not be described in detail.

[0128] Furthermore, the step S2-forming of the training set further includes the following steps:

[0129] S2.8-Data classification:

[0130] According to the environmental parameters obtained in step S1, different working conditions corresponding to different soil types and excavation depths are divided, and in combination with different working conditions and different ship types, the historical experience replay memory library obtained in step S2.6 is divided into data. In step S2.7, the data of the same working condition of the same ship type in the historical experience replay memory library are extracted for model training, so that the model can be pre-adjusted according to different working conditions and ship types, thereby achieving rapid convergence of the model.

[0131] Example 2: Engineering example based on the dredging process control optimization method described in Example 1:

[0132] 1. Implementation process:

[0133] S1- Acquisition of initial data:

[0134] Obtaining dredging state parameters, dredging action parameters and environmental parameters;

[0135] The dredging state parameters include the raking lip angle, the mud pump speed, and the ship speed; the dredging action parameters include: increase, unchanged, and decrease; the environmental parameters include soil quality and digging depth;

[0136] S2-Formation of training set:

[0137] S2.1- Set up the electronic fence of dredging state parameters and discretize them:

[0138] The maximum value of the rake lip angle is 15 degrees and the minimum value is 0 degrees. The electronic fence of the rake lip angle is set to 0-15 degrees and discretized into {0,1,2…15};

[0139] The maximum speed of the mud pump is 95% of the maximum speed, and the minimum speed is 80% of the maximum speed. The electronic fence of the speed is set to 80%-95% of the maximum speed, and it is discretized into {80,81,82…93,94,95};

[0140] The maximum speed is 3 knots and the minimum is 2 knots, so the speed electronic fence is set to 2-3 knots and discretized into {2.0, 2.1, 2.2, …, 2.7, 2.8, 2.9, 3.0};

[0141] S2.2-Action set settings:

[0142] The dredging action parameters correspond to the increase, decrease or unchanged of the raking lip angle, mud pump speed and ship speed. The action set obtained thereby includes three different actions of three dredging state parameters, that is, it contains 27 different actions; the action set is {0,00; 001; 00-1; 010; 011; 01-1; ...111; -1-1-1}; wherein 0 represents unchanged action, 1 represents an increase in the corresponding parameter, and -1 represents a decrease in the corresponding parameter.

[0143] S2.3-Integration of historical data:

[0144] Integrate the discretized dredging state parameters and action sets to form a historical data set (s t , a t ,s t+1 );

[0145] Among them, s t is the dredging state parameter at time t, a t For t Dredging action parameters under state, s t+1 for a t Dredging state parameters at the next moment after the action;

[0146] S2.4- Design of reward function:

[0147] Let a1 = 0.7, a2 = 0.9; design reward function r:

[0148]

[0149] S2.5-Formation of historical experience playback memory bank:

[0150] The reward function value r corresponding to time t t Add historical data sets (s t , a t ,s t+1 ), forming a historical experience playback memory bank (s t , a t , r t ,s t+1 ).

[0151] S2.6-Formation of historical data training set:

[0152] Extract the data of different ships under the same ship type and the same working conditions from the historical experience playback memory library to form a historical data training set (s t , a t , r t ,s t+1 ).

[0153] S3-Meta-RL model training:

[0154] S3.1- Construction of initial reinforcement learning model:

[0155] Build an initial reinforcement learning model based on a neural network. The current Q network includes an input layer, an output layer, and a hidden layer. The network structure is set as follows:

[0156] The input layer includes 3 state parameter nodes;

[0157] The hidden layer includes two layers, the first layer includes 128 neurons and uses the ReLU activation function; the second layer includes 64 neurons and uses the ReLU activation function;

[0158] The output layer includes 27 neurons and uses a sigmoid activation function;

[0159] The current Q network and the target Q network have the same network structure.

[0160] S3.2- Initialization of the initial reinforcement learning model:

[0161] Let θ = θ′ to complete the initialization of the initial reinforcement learning model;

[0162] S3.3-Setting of preset conditions:

[0163] Set the first preset condition to the number of iterations K=400;

[0164] The second preset condition is set as the maximum update times of the θ parameter x=4000 / the maximum update times of the θ′ parameter y=30;

[0165] S3.4-Model training:

[0166] Based on the historical data training set, the θ parameter and θ′ parameter are calculated and updated through the back propagation algorithm to perform model training.

[0167] S4- Exploration in the early stage of dredging:

[0168] S4.1-Meta-RL model output actions:

[0169] Calculate the Q value in the target Q network and select the action a with the largest Q value t As the output action a of the meta-RL model max ;

[0170] S4.2- Setting of exploration function:

[0171] Let the initial exploration degree μ0 = 0.7, the maximum number of rounds R max =500, set the exploration function μ;

[0172] S4.3-Determination of dredging actions:

[0173] Select the output action a of the meta-reinforcement learning model with μ probability max As a dredging action t ′;

[0174] A random action in the action space is taken as the dredging action a with a probability of 1-μ t ′;

[0175] Use dredging action t ′Control the movement of dredging equipment.

[0176] S4.4-Getting the current data training set:

[0177] Dredging action t The state before the dredging action is s′, and the dredging action a t The state after the action is s t+1 ′, taking the reward function value corresponding to time t as r t ′, forming the current data training set (s′, a t ′,r t ′,s t+1 ′).

[0178] S5- Training of Greedy Reinforcement Model:

[0179] Randomly extract data from the historical data training set and the current data training set, and further train the meta-reinforcement learning model to obtain the greedy reinforcement model.

[0180] S6-Controlled Field Dredging:

[0181] The greedy reinforcement model is used to control the dredging of ships in combination with the current dredging state parameters.

[0182] 2. Beneficial effects:

[0183] 2.1- Construction effect of silt geology:

[0184] The conventional control method and the control method of the present invention are used to control the same dredger to perform 30 operations on the silt geology alternately, and the comparative data in Table 1 are obtained as follows:

[0185] Table 1- Comparison of construction effects of silt geology

[0186]

[0187] By adopting the method of the present invention in construction in muddy geology, the speed of retracting and releasing the rake is significantly improved, the dredging efficiency is increased by an average of 7.85%, and the fuel consumption per 10,000 cubic meters of soil is reduced by an average of 7.43%.

[0188] 2.2- Construction effect of composite geological structure of silt and fine sand:

[0189] The conventional control method and the control method of the present invention are used to control the same dredger to perform 20 operations on the silt and fine sand composite geology alternately, and the comparative data in Table 2 are obtained as follows:

[0190] Table 2-Comparison of composite geological construction effects

[0191]

[0192]

[0193] By adopting the control method of the present invention to carry out construction in the composite geology of silt and fine silt sand, the mud blowing efficiency is improved by an average of 5.94%, and the oil consumption per 10,000 cubic meters of soil is reduced by an average of 5.87%.

[0194] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are within the scope of the technical solution of the present invention.

Claims

1. A data-driven dredging process control optimization method for a trailing suction hopper dredger, characterized in that: The steps include: S1- Acquisition of initial data: Obtain dredging status parameters, dredging action parameters and environmental parameters through the ship monitoring system; S2-Formation of historical data training set: The dredging state parameters of adjacent time intervals and the dredging action parameters within the time interval are integrated as a historical data set, a reward function is designed to be associated with the historical data set to form a historical experience playback memory library, and data is extracted from the historical experience playback memory library to form a historical data training set; S3-Meta-RL model training: Build an initial reinforcement learning model, and use the historical data training set to train the initial reinforcement learning model to obtain a meta-reinforcement learning model; S4- Exploration in the early stage of dredging: In the early stage of dredging, the meta-reinforcement learning model is used to conduct environmental exploration of the actual dredging site to obtain the current data training set; S5- Training of Greedy Reinforcement Model: Mix and extract data from the historical data training set and the current data training set to further train the meta-reinforcement learning model and obtain the greedy reinforcement model; S6-Controlled Field Dredging: The greedy reinforcement model is used to control the dredging of ships in combination with the current dredging state parameters.

2. The data-driven dredging process control optimization method for a trailing suction hopper dredger according to claim 1, characterized in that: In the step S1-initial data acquisition, the dredging state parameters include the raking lip angle, the mud pump rotation speed and the ship speed; the dredging action parameters are the increase, decrease or unchanged of the dredging state parameters; and the environmental parameters include soil quality and digging depth.

3. The data-driven dredging process control optimization method for a trailing suction hopper dredger according to claim 2 is characterized in that: The step S2-forming a historical data training set comprises the following steps: S2.1- Setting Geo-Fence for Dredging Status Parameters: According to the different types of ship power systems and dredging equipment, set up electronic fences for dredging status parameters; S2.2- Discretization of dredging state parameters: According to the electronic fence of dredging state parameters, the dredging state parameters are discretized; S2.3- Discretization of dredging action parameters: The dredging action parameters correspond to the increase, decrease or unchanged of the dredging state parameters, and the action set is obtained accordingly; S2.4-Integration of historical data sets: Integrate the discretized dredging state parameters and action sets to form a historical data set; S2.5- Design of reward function: Design reward functions based on the absolute value of the vacuum; S2.6-Formation of historical experience playback memory bank: Based on the dredging state parameters, action sets and reward function values ​​corresponding to the same moment and the dredging state parameters of the next moment, a historical experience playback memory library is formed; S2.7-Formation of historical data training set: Extract historical experience and replay data in the memory bank to form a historical data training set.

4. The data-driven dredging process control optimization method for a trailing suction hopper dredger according to claim 3 is characterized in that: The step S3-meta reinforcement learning model training includes the following steps: S3.1- Construction of initial reinforcement learning model: Building an initial reinforcement learning model based on a neural network, wherein the initial reinforcement learning model includes a current Q network and a target Q network; S3.2- Initialization of the initial reinforcement learning model: Let θ = θ′ to complete the initialization of the initial reinforcement learning model; S3.3-Setting of preset conditions: The first preset condition is the number of iterations K of updating the θ′ parameter; The second preset condition is the maximum update times x or y of the θ parameter or the θ′ parameter; S3.4-Model training: Based on the historical data training set, the θ parameter is calculated and updated by a back propagation algorithm; when the first preset condition is met, the θ parameter is copied to the θ′ parameter; When the second preset condition is met, the model stops training and the current values ​​of the θ parameter and the θ′ parameter are determined as the parameters of the meta-reinforcement learning model.

5. The data-driven dredging process control optimization method for a trailing suction hopper dredger according to claim 4 is characterized in that: The step S4-exploration in the early stage of dredging includes the following steps: S4.1-Meta-RL model output actions: Select the action set corresponding to the largest action value as the output action of the meta-reinforcement learning model; S4.2- Setting of exploration function: Design the exploration function μ based on the greedy algorithm; S4.3-Determination of dredging actions: Selecting the output action as the dredging action with μ probability; A random action in the action space is used as a dredging action with a probability of 1-μ; Control the dredging machine movement through the selected dredging action; S4.4-Formation of current data training set: The current data training set is formed based on the dredging action, the dredging state parameters before and after the dredging action, and the reward function value corresponding to the dredging action.

6. The data-driven dredging process control optimization method for a trailing suction hopper dredger according to claim 5, characterized in that: In the step S5-greedy reinforcement model training, data from the current data training set and the historical data training set are mixed and extracted to further train the meta-reinforcement learning model to obtain the greedy reinforcement model.

7. The data-driven dredging process control optimization method for a trailing suction hopper dredger according to claim 3, characterized in that: The step S2-forming of the training set further comprises the following steps: S2.8-Data classification: According to the environmental parameters obtained in step S1, different working conditions corresponding to different soil types and excavation depths are divided, and in combination with different working conditions and different ship types, the historical experience replay memory library obtained in step S2.6 is divided into data. In step S2.7, the data of the same working condition of the same ship type in the historical experience replay memory library are extracted for model training, so that the model can be pre-adjusted according to different working conditions and ship types, thereby achieving rapid convergence of the model.

Images