Deepwater laying intelligent control method and device based on water flow environment perception

Abstract

The invention provides a deepwater layout intelligent control method and device based on water flow environment perception, and relates to the technical field of turnover plate layout control, and the method comprises the steps: building an initial deep reinforcement learning model based on an Actor-Critic framework, and the initial deep reinforcement learning model comprises an Actor network and a Critic network; an experience pool and a reward function are constructed, sample data are acquired from the experience pool, parameters of the Actor network and the Critic network are iteratively adjusted through the reward function according to the sample data, and a final deep reinforcement learning model is generated; collecting a water flow environment data set at the current moment through a sensor network on the laying ship, and preprocessing the water flow environment data set to obtain a state vector at the current moment; inputting the state vector at the current moment into the final deep reinforcement learning model for forward propagation to obtain an action vector at the current moment; and adjusting the laying operation of the laying ship at the next moment according to the action vector at the current moment.

Description

Technical Field

[0001] This application relates to the field of flip-board paving control technology, and in particular to a deep-water paving intelligent control method and device based on water flow environment perception. Background Technology

[0002] Deep-water raft laying is a crucial construction phase in marine engineering and river management, primarily used for projects such as submarine pipeline protection, riverbed reinforcement, and ecological bank protection. The quality of this work directly impacts the stability and lifespan of the project. Existing underwater flexible raft laying methods are widely used in straight waterways with good flow conditions. However, with the development of waterways, laying underwater flexible rafts in waters with complex flow conditions still faces numerous challenges. These include sharp bends and deep water areas with high water pressure, fast currents, and unpredictable flow directions; the speed and distance of vessel movement rely heavily on manual experience; and the fixed length of the flaps cannot be adjusted during laying. These problems expose underwater flexible rafts to risks such as tearing or slippage, significantly increasing project uncertainty.

[0003] Existing deployment control methods cannot effectively optimize the decision network for complex and variable water flow environments, resulting in the decision network being unable to output the optimal operation strategy under different water flow scenarios, thus affecting the adjustment accuracy of deployment operations.

[0004] Existing paving control methods cannot translate operational strategies into specific, quantifiable adjustment commands. As a result, it is still necessary to rely on the experience of the crew to judge the water flow environment and adjust the course, speed, and flap parameters of the paving vessel, resulting in low precision in paving operations. Summary of the Invention

[0005] To address the aforementioned problems, in a first aspect, the present invention provides an intelligent control method for deep-water laying based on water flow environment perception, comprising: An initial deep reinforcement learning model is constructed based on the Actor-Critic framework. The initial deep reinforcement learning model includes an Actor network and a Critic network. An experience pool and a reward function are constructed. Sample data is obtained from the experience pool. The parameters of the Actor network and the Critic network are iteratively adjusted according to the sample data through the reward function to generate the final deep reinforcement learning model. The current water flow environment data set is collected by the sensor network on the laying vessel. The water flow environment data set is preprocessed to obtain the current state vector. The state vector includes the average water flow velocity, water depth, soft body strain value, flap angle, flap length and water flow force. The current state vector is input into the final deep reinforcement learning model for forward propagation to obtain the current action vector, which includes heading adjustment, speed adjustment, flap angle adjustment and flap length adjustment. Adjust the deployment operation of the deployment ship in the next moment based on the action vector at the current moment.

[0006] Optionally, the reward function is a weighted fusion of the software strain value ratio, flap angle deviation, flap length deviation, speed, and strain over-limit indication function.

[0007] Optionally, the parameters of the Actor network and Critic network are iteratively adjusted using a reward function based on sample data to generate the final deep reinforcement learning model, including: S11: Obtain the sample state vector s at time t from the sample data. t and label action vector a' t ; S12: Transfer the sample state vector s t Input the Actor network and perform forward propagation to obtain the predicted action vector a at time t. t ; S13: Transfer the label action vector a' t and predicted action vector a t Substitute the reward function to calculate the reward value r at time t. t ; S14: Transfer the sample state vector s t and predicted action vector a t Input the Critic network for state evaluation and generate the value Q at time t; adjust the parameters of the Actor network and Critic network according to the value Q to obtain the optimized Actor network and Critic network. S15: Obtain the sample state vector s at time t+1 from the sample data. t+1 According to the sample state vector s t+1 The optimized value Q' at time t is generated using the optimized Actor network and the optimized Critic network, based on the reward value r. t The TD target value y at time t is obtained by calculating the optimized value Q'. t ; S16: Based on value Q and TD target value y t Calculate the loss value at time t; S17: Repeat steps S11-S16 until the loss value converges to the preset value, generating the final deep reinforcement learning model.

[0008] Optionally, the calculation process for the reward value includes: The sample speed is obtained from the sample data, the predicted software strain value, the predicted flap angle and the predicted flap length are obtained from the predicted action vector, and the tag software strain value, the tag flap angle and the tag flap length are obtained from the tag action vector. The ratio of the predicted software strain value to the tag software strain value is substituted into the software strain value ratio term. The difference between the predicted flap angle and the tag flap angle is substituted into the flap angle deviation term. The difference between the predicted flap length and the tag flap length is substituted into the flap length deviation term. The sample speed is substituted into the speed term. The predicted software strain value is substituted into the strain over-limit indication function to calculate the reward value.

[0009] Optionally, the process of obtaining the state vector includes: The average flow velocity, water depth, soft sprue strain value, flap angle, flap length, water density, soft sprue flow resistance coefficient, average flow velocity within the draft range of the sprue-laying vessel, angle between the flow direction and the longitudinal axis of the sprue-laying vessel, soft sprue width, and water depth at the soft sprue laying location are obtained from the water flow environment data set. The projected area of ​​the soft raft perpendicular to the water flow direction is calculated based on the width of the soft raft, the water depth at the soft raft laying location, and the angle between the water flow direction and the longitudinal axis of the rafting vessel. The hydrodynamic force is calculated based on the density of water, the hydrodynamic drag coefficient of the soft sprue, the average flow velocity within the draft range of the sprue vessel, and the projected area of ​​the soft sprue perpendicular to the water flow direction. The state vector consists of the average flow velocity, water depth, soft body strain value, flap angle, flap length, and flow force.

[0010] Optionally, the step of inputting the current state vector into the final deep reinforcement learning model for forward propagation to obtain the current action vector includes: Obtain the weight matrix, bias vector, activation function, and hyperbolic tangent function of the Actor network in the final deep reinforcement learning model; Substitute the current state vector, weight matrix, and bias vector into the activation function to generate the first result; Substitute the first result into the hyperbolic tangent function to generate the action vector at the current moment.

[0011] Optionally, adjusting the deployment operation of the deployment vessel in the next moment based on the action vector at the current moment includes: Get the current heading, current speed, current flap angle, and current flap length; Calculate the heading for the next moment based on the current heading and heading adjustment amount; The ship's speed at the next moment is calculated based on the current ship speed and the speed adjustment amount. The flip angle for the next moment is calculated based on the current flip angle and the amount of flip angle adjustment. The flap length at the next moment is calculated based on the current flap length and the flap length adjustment amount. The laying operation of the laying vessel is adjusted according to the course, speed, tilting angle, and tilting length at the next moment.

[0012] Secondly, the present invention provides an intelligent control device for deep-water laying based on water flow environment perception, used to implement the aforementioned intelligent control method for deep-water laying based on water flow environment perception, the device comprising: The initial deep reinforcement learning model building module is used to build an initial deep reinforcement learning model based on the Actor-Critic framework. The initial deep reinforcement learning model includes an Actor network and a Critic network. The final deep reinforcement learning model acquisition module is used to construct the experience pool and reward function, obtain sample data from the experience pool, and iteratively adjust the parameters of the Actor network and Critic network based on the sample data through the reward function to generate the final deep reinforcement learning model. The state vector acquisition module is used to collect the current water flow environment data set through the sensor network on the laying vessel, preprocess the water flow environment data set, and obtain the current state vector. The state vector includes the average water flow velocity, water depth, soft body strain value, flap angle, flap length, and water flow force. The action vector acquisition module is used to input the current state vector into the final deep reinforcement learning model for forward propagation to obtain the action vector at the current moment. The action vector includes heading adjustment, speed adjustment, flap angle adjustment and flap length adjustment. The deployment operation adjustment module is used to adjust the deployment operation of the deployment ship in the next moment based on the motion vector at the current moment.

[0013] Thirdly, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the aforementioned intelligent control method for deep-water laying based on water flow environment perception.

[0014] Fourthly, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned intelligent control method for deep-water laying based on water flow environment perception.

[0015] The present invention has the following beneficial effects: 1. An initial deep reinforcement learning model is constructed based on the Actor-Critic framework. The initial deep reinforcement learning model is iteratively optimized using sample data and reward functions to continuously improve the decision accuracy of the Actor network. This allows the final deep reinforcement learning model obtained after optimization to dynamically adapt to changes in the water flow environment. Even in unfamiliar waters or under sudden water flow disturbances, it can quickly output adaptive adjustment quantities, improving the versatility of operations in different waters. The Actor network of the final deep reinforcement learning model outputs action vectors, which are used to adjust the deployment operation in a timely manner, significantly reducing soft material deployment losses and deployment deviations, and improving the quality and efficiency of deep-water deployment operations.

[0016] 2. The initial deep reinforcement learning model is trained based on the sample data in the experience pool. The parameters of the Actor network and Critic network are adjusted by the value Q. The reward value is calculated by the soft-body strain ratio, flap angle deviation, flap length deviation, speed, and strain over-limit indication function of the reward function. The target value is calculated based on the reward value and the optimized value. The optimization targets are strain not exceeding the limit, optimal flap parameters, and speed adapted to the environment. The Actor network continuously learns the optimal operation strategy under different water flow scenarios, guiding the action vector output by the Actor network to always move closer to the target of low loss and high precision, thereby improving the adjustment accuracy of the deployment operation.

[0017] 3. Based on the current heading, ship speed, flap angle, flap length, heading adjustment, speed adjustment, flap angle adjustment, and flap length adjustment, the system calculates the heading, ship speed, flap angle, and flap length for the next moment. This matches the operational requirements of the laying operation without exceeding the mechanical limits of the equipment, achieving specific quantitative operations for the laying operation and improving the accuracy of the laying operation. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a structural diagram of the device according to an embodiment of the present invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0021] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.

[0022] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0023] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0024] Reference Figure 1 This invention provides an intelligent control method for deep-water laying based on water flow environment perception, comprising: The S100 builds an initial deep reinforcement learning model based on the Actor-Critic framework, which includes an Actor network and a Critic network.

[0025] In some embodiments, the Actor network is a fully connected neural network. Its core function is to map a 6-dimensional state vector to a 4-dimensional continuous action vector, ensuring the output conforms to the physical constraints of the deployment operation. The input layer receives a 6-dimensional state vector, including the average flow velocity, water depth, soft material strain value, flap angle, flap length, and water force. A normalization layer is added after the input layer, using the mean and standard deviation of offline calculated state samples to map the state values ​​of each dimension to the [-1, 1] interval, avoiding the impact of differences in dimensional values ​​on network training stability. The input layer has 6 neurons, corresponding one-to-one with the input dimensions, and has no activation function, only responsible for receiving and preprocessing the state vector. The hidden layer adopts a two-layer fully connected structure. The first layer has 64 neurons, designed based on the complexity of the deep-water deployment state-action mapping, which can capture nonlinear relationships while avoiding training difficulties caused by too many initial parameters. The activation function is ReLU, calculated by taking the maximum value between the input and 0, alleviating the gradient vanishing problem and reducing computation through sparse activation. A Dropout layer is also added with a probability of 0.1. The first layer has provisions for overfitting prevention; the second layer has 32 neurons, which extract core information of state features through dimensionality reduction, realizing the transition from high-dimensional feature compression to low-dimensional action mapping. It also uses the ReLU activation function, and the inter-layer structure uses a combination of fully connected layers and batch normalization. The batch normalization momentum is set to 0.9 to avoid output distribution shift in hidden layers and accelerate gradient convergence. The output layer has 4 neurons, corresponding to 4-dimensional action adjustment amounts. The activation function is the hyperbolic tangent function, mapping the output to the [-1, 1] interval. Subsequently, a scaling factor is used to convert it to the actual operational range, such as heading adjustment [-10°, 10°] and speed adjustment [-0.5m / s, 0.5m / s]. There is no regularization layer to ensure continuous and real-time action output. For parameter initialization, the weights are initialized using He normality, based on the number of input neurons in the current layer according to specific rules, adapting to the gradient characteristics of the ReLU activation function to avoid abnormal initial weights causing training stagnation. The bias vector is initialized to 0 to reduce the complexity of initial parameters. The optimizer is Adam, with a learning rate of 0.001. , It adapts to continuous action space gradient updates and mitigates initial training oscillations through adaptive learning rate.

[0026] The Critic network is also a fully connected neural network. Its core function is to evaluate the value of state-action combinations, providing gradient directions for Actor action optimization. The input must include the Actor action output to form a closed loop. The input layer receives a 10-dimensional vector, which is formed by concatenating a 6-dimensional state vector and a 4-dimensional action vector to ensure the evaluation of the value of a specific action in the current state. The preprocessing is consistent with the Actor network. The state vector is normalized to [-1, 1] using the same mean and standard deviation, and the action vector is normalized using its own sample mean and standard deviation to avoid numerical interference from the input dimension. There are 10 neurons, matching the concatenated input dimension. There is no activation function; it is only responsible for receiving and preprocessing the state-action combination vector. The hidden layer adopts a two-layer fully connected structure symmetrical to the Actor network. The first layer has 64 neurons, consistent with the first layer of the Actor network to ensure uniformity of feature extraction dimensions. The activation function is R. The system uses eLU, adds Dropout and BatchNorm layers with the same parameters as the Actor to avoid evaluation bias affecting Actor training. The second layer has 32 neurons, symmetrical to the Actor's second layer, further compressing features and establishing value associations. It uses ReLU activation, and fully connected layers with BatchNorm are added between layers, with the same parameters as the Actor, ensuring stable mapping from state-action features to value output. The output layer has one neuron, outputting a 1-dimensional state-action value Q. It has no activation function and directly outputs a continuous value, which can be positive or negative (positive for action gain, negative for action loss), without requiring dimensional mapping. During parameter initialization, weights are initialized using Xavier normal distribution, based on the number of input neurons in the current layer according to rules, adapting to the continuous output requirement of value evaluation. The bias vector is initialized to 0. The optimizer is Adam, with a learning rate of 0.002. , Because the Critic needs to quickly respond to changes in the Actor's actions to correct the evaluation, a slightly higher learning rate can improve the evaluation accuracy and iteration speed.

[0027] The S200 constructs an experience pool and a reward function, obtains sample data from the experience pool, and iteratively adjusts the parameters of the Actor network and Critic network based on the sample data through the reward function to generate the final deep reinforcement learning model.

[0028] In some embodiments, the reward function is a weighted fusion of the software strain value ratio, flap angle deviation, flap length deviation, speed, and strain over-limit indication function.

[0029] In some embodiments, the parameters of the Actor network and the Critic network are iteratively adjusted using a reward function based on sample data to generate the final deep reinforcement learning model, including: S11: Obtain the sample state vector s at time t from the sample data. t and label action vector a't ; S12: Transfer the sample state vector s t Input the Actor network and perform forward propagation to obtain the predicted action vector a at time t. t ; S13: Transfer the label action vector a' t and predicted action vector a t Substitute the reward function to calculate the reward value r at time t. t ; In some embodiments, the calculation process of the reward value includes: The sample speed is obtained from the sample data, the predicted software strain value, the predicted flap angle and the predicted flap length are obtained from the predicted action vector, and the tag software strain value, the tag flap angle and the tag flap length are obtained from the tag action vector. The ratio of the predicted software strain value to the tag software strain value is substituted into the software strain value ratio term. The difference between the predicted flap angle and the tag flap angle is substituted into the flap angle deviation term. The difference between the predicted flap length and the tag flap length is substituted into the flap length deviation term. The sample speed is substituted into the speed term. The predicted software strain value is substituted into the strain over-limit indication function to calculate the reward value.

[0030] In some embodiments, the reward value r t The expression is: in, This represents the ratio of software strain values. This indicates the predicted strain value of the software. Indicates the strain value of the label software, | i opt -θ actual | indicates the flap angle deviation. i opt Indicates the predicted flip angle. i actual Indicates the label flip angle. L opt -L actual This indicates the deviation in flap length. L opt Indicates the predicted flap length. L actual Indicates the length of the label flap. v ship Indicates the speed item. This represents the strain over-limit indicator function.

[0031] S14: Transfer the sample state vector s tand predicted action vector a t Input the Critic network for state evaluation and generate the value Q at time t; adjust the parameters of the Actor network and Critic network according to the value Q to obtain the optimized Actor network and Critic network. In some embodiments, the Actor network updates its parameters using gradients, where the gradient is expressed as: in, The objective function J represents the policy parameter θ. μ The gradient; The value Q of the Critic network output is relative to action a. t The gradient; The output vector μ(s) of the Actor network is represented by t ) for its own parameter θ μ The gradient.

[0032] The expression for value Q is: Where Q(s) t ,a t ) represents the value Q at time t, [s t ;a t ] represents the concatenated vector of state and action; ReLU represents the activation function; W1, W2, and W3 represent the weight matrices; b1, b2, and b3 represent the bias vectors.

[0033] S15: Obtain the sample state vector s at time t+1 from the sample data. t+1 According to the sample state vector s t+1 The optimized value Q' at time t is generated using the optimized Actor network and the optimized Critic network, based on the reward value r. t The TD target value y at time t is obtained by calculating the optimized value Q'. t ; In some embodiments, the TD target value y t The expression is: in, c Indicates the discount factor; Q′ (s t+1 , m′ (s t+1 )) represents the optimized value at time t; m′ This represents the output vector of the optimized Actor network.

[0034] S16: Based on value Q and TD target value yt Calculate the loss value at time t; In some embodiments, the loss value L critic The expression is: Where N represents the total number of samples.

[0035] S17: Repeat steps S11-S16 until the loss value converges to the preset value, generating the final deep reinforcement learning model.

[0036] The S300 collects current water flow environment data through a sensor network on the laying vessel, preprocesses the water flow environment data to obtain the current state vector, which includes the average water flow velocity, water depth, soft body strain value, flap angle, flap length and water flow force.

[0037] In some embodiments, the hardware selection for the sensor network includes: a shipborne ADCP: a Doppler profiler is selected to measure the three-dimensional velocity and direction of flow in different water layers, with a sampling frequency ≥2Hz, installed at the bow to avoid areas disturbed by water flow; an ultrasonic depth sounding system: installed on the bottom of the ship to scan the terrain within a range of 30-50 meters in front of the bow in real time; mechanical sensors: waterproof tensile sensors with a range of 0-100kN and resistance strain gauges are installed at key stress points such as the soft anchor chain connection and the hinge point of the flap; tilt sensors: tilt sensors with an accuracy of 0.1° are installed at the flap pivot, and a rangefinder is installed at the edge of the flap to measure the actual extension length of the flap; a central processing unit: a shipborne industrial computer is used, equipped with a multi-channel data acquisition card and an integrated communication module to realize real-time data uploading and command reception; all sensors are calibrated before installation, and before construction, the system is debugged in calm water, all sensors are started, the synchronization and stability of the data channels are checked, and the measurement accuracy is verified by manually providing known conditions.

[0038] After collecting the water flow environment data set through the sensor network, the preprocessing of the water flow environment data set includes: applying a low-pass filter to the raw data in the water flow environment data set to remove high-frequency noise, using the 3σ criterion to remove outliers caused by brief collisions or bubbles, converting all sensor data to the ship's coordinate system with the center of gravity of the laying vessel as the origin by using the relative coordinates of the installation position, and using the GPS timing module to add a unified high-precision timestamp to all data.

[0039] In some embodiments, the process of obtaining the state vector includes: The average flow velocity, water depth, soft sprue strain value, flap angle, flap length, water density, soft sprue flow resistance coefficient, average flow velocity within the draft range of the sprue-laying vessel, angle between the flow direction and the longitudinal axis of the sprue-laying vessel, soft sprue width, and water depth at the soft sprue laying location are obtained from the water flow environment data set. The projected area of ​​the soft raft perpendicular to the water flow direction is calculated based on the width of the soft raft, the water depth at the soft raft laying location, and the angle between the water flow direction and the longitudinal axis of the rafting vessel. In some embodiments, the projected area of ​​the soft drain perpendicular to the water flow direction The calculation formula is: in, This refers to the width of the software layout, in meters (m). The water depth at the location where the soft sheet is laid is indicated in meters (m). The angle between the direction of water flow and the longitudinal axis of the rafting vessel, with a value ranging from 0° to 180°; The hydrodynamic force is calculated based on the density of water, the hydrodynamic drag coefficient of the soft sprue, the average flow velocity within the draft range of the sprue vessel, and the projected area of ​​the soft sprue perpendicular to the water flow direction. The state vector consists of the average flow velocity, water depth, soft body strain value, flap angle, flap length, and flow force.

[0040] In some embodiments, water flow force The calculation formula is: in, The density of water, in tons per cubic meter (t / m³). 3 ; This is the coefficient of fluid resistance in soft drainage systems. The average current velocity within the draft range of the laying vessel, in m / s.

[0041] The average flow velocity, water depth, soft body strain, flap angle, flap length, and flow force are normalized to obtain the state vector s. t It contains 6-dimensional physical observations: in, U The average flow velocity of the water is expressed in m / s. h Water depth, in meters (m). This represents the strain value of the soft material. i The angle of the flip panel; L The length of the flap is in meters (m). F cm The force is the force of the water flow, expressed in kN.

[0042] The S400 inputs the current state vector into the final deep reinforcement learning model for forward propagation to obtain the current action vector, which includes heading adjustment, speed adjustment, flap angle adjustment, and flap length adjustment.

[0043] In some embodiments, the step of inputting the current state vector into the final deep reinforcement learning model for forward propagation to obtain the current action vector includes: Obtain the weight matrix, bias vector, activation function, and hyperbolic tangent function of the Actor network in the final deep reinforcement learning model; Substitute the current state vector, weight matrix, and bias vector into the activation function to generate the first result; Substitute the first result into the hyperbolic tangent function to generate the action vector at the current moment.

[0044] In some embodiments, the state vector is mapped to a deterministic action vector using the DDPG (Deep Deterministic Policy Gradient) algorithm, and the expression for the action vector is: Among them, a t This represents the action vector at time t. m Represents a deterministic strategy function; i μ Represents the set of parameters of the Actor network; tanh Represents the hyperbolic tangent function; s t represents the state vector at time t; ReLU represents the activation function; W1, W2, and W3 represent the weight matrices; and b1, b2, and b3 represent the bias vectors.

[0045] S500 adjusts the deployment operation of the deployment ship in the next moment based on the motion vector at the current moment.

[0046] In some embodiments, adjusting the deployment operation of the deployment vessel in the next moment based on the motion vector at the current moment includes: Get the current heading, current speed, current flap angle, and current flap length; Calculate the heading for the next moment based on the current heading and heading adjustment amount; The ship's speed at the next moment is calculated based on the current ship speed and the speed adjustment amount. The flip angle for the next moment is calculated based on the current flip angle and the amount of flip angle adjustment. The flap length at the next moment is calculated based on the current flap length and the flap length adjustment amount. The laying operation of the laying vessel is adjusted according to the course, speed, tilting angle, and tilting length at the next moment.

[0047] In some embodiments, data is collected in real time by dedicated sensing equipment mounted on the laying vessel to ensure the timeliness and accuracy of parameters. The heading is collected by a combination of shipborne GPS positioning module and IMU inertial measurement unit. The GPS sampling frequency is 10Hz and the IMU sampling frequency is 50Hz. After the data is fused by Kalman filtering, the heading angle with true north as the reference, in degrees and in the range of 0°~360° is output with an accuracy of ±0.1°. The ship speed is collected by Doppler log with a sampling frequency of 10Hz. The ship speed relative to the ground is output in meters per second with an accuracy of ±0.01m / s. At the same time, moving average filtering is used to remove instantaneous outliers caused by wind and waves. The flap angle is acquired using a high-precision tilt sensor installed at the flap's pivot point, with a sampling frequency of 20Hz. The sensor outputs the angle between the flap and the horizontal direction, in degrees, ranging from 0° to 90°, with an accuracy of ±0.05°. This data is directly fed into the ship's central processing unit. The flap length is acquired using a laser rangefinder installed on the flap's edge, with a sampling frequency of 15Hz. This measures the actual extended length of the flap, in meters, with an accuracy of ±0.02m. The installation position deviation also needs to be corrected through ship coordinate system conversion. After all parameters are acquired, a unified timestamp with an accuracy of 1ms is added using a GPS timing module to ensure that the four sets of data are synchronized. The data is then stored in a real-time database for subsequent calculations.

[0048] The heading at the next moment = (the heading at the current moment + Δα) mod 360°, where Δα is the heading adjustment amount, and Δα mod 360° means taking the remainder after dividing by 360°.

[0049] The ship's speed at the next moment = the ship's speed at the current moment + ΔV, where ΔV is the speed adjustment amount.

[0050] The flip angle at the next moment = the flip angle at the current moment + Δθ, where Δθ is the adjustment amount of the flip angle.

[0051] The flap length at the next moment = the flap length at the current moment + ΔL, where ΔL is the flap length adjustment amount.

[0052] Finally, the ship's operational adjustments are executed. The calculated parameters for the next moment are converted into specific execution commands by the ship's control system, driving the corresponding mechanisms to move. A feedback verification mechanism is also incorporated. For heading adjustments, the heading for the next moment is converted into a 4-20mA current signal to control the rudder, driving it to the target angle. The rudder response time is no more than 0.5s. During the adjustment process, the IMU provides real-time feedback on the actual heading. If the deviation from the target value exceeds 0.5°, a secondary fine-tuning is triggered. For speed adjustments, the ship's speed for the next moment is transmitted to the propulsion system controller. Speed ​​control is achieved by adjusting the main engine throttle opening or the motor speed. The propulsion system response time is no more than 1s. The Doppler log monitors the actual speed in real time. If the deviation exceeds 0.05m / s, the throttle or speed is dynamically corrected. Parameters: For flap angle adjustment, the next flap angle needs to be sent to the hydraulic actuator controller. The hydraulic oil flow is controlled to drive the flap rotation. The actuator response time is no more than 0.8s. The tilt sensor provides real-time feedback of the actual angle. If the deviation exceeds 0.1°, hydraulic pressure compensation adjustment is initiated. For flap length adjustment, the next flap length is transmitted to the telescopic mechanism controller. The stepper motor drives the flap extension and retraction. The stepper motor positioning accuracy is ±0.01m, and the response time is no more than 1s. The laser rangefinder provides real-time feedback of the actual length. If the deviation exceeds 0.03m, the motor is triggered for reverse fine adjustment. After all mechanisms are adjusted, the system collects new status parameters after a 0.2s delay. It confirms that the heading, ship speed, flap angle, and flap length have all reached the target values ​​and the deviations are within the allowable range, thus completing the adjustment for this laying operation. If the target is not met, the adjustment process is repeated until the requirements are met. If the adjustment fails three times consecutively, an alarm will be triggered and the operation will be suspended.

[0053] Reference Figure 2 This invention provides a deep-water laying intelligent control device 20 based on water flow environment perception, used to implement a deep-water laying intelligent control method based on water flow environment perception. The device includes: The initial deep reinforcement learning model building module 21 is used to build an initial deep reinforcement learning model based on the Actor-Critic framework. The initial deep reinforcement learning model includes an Actor network and a Critic network. The final deep reinforcement learning model acquisition module 22 is used to construct the experience pool and reward function, obtain sample data from the experience pool, and iteratively adjust the parameters of the Actor network and Critic network according to the sample data through the reward function to generate the final deep reinforcement learning model. The state vector acquisition module 23 is used to collect the current water flow environment data set through the sensor network on the laying vessel, preprocess the water flow environment data set, and obtain the current state vector. The state vector includes the average water flow velocity, water depth, soft body strain value, flap angle, flap length, and water flow force. The action vector acquisition module 24 is used to input the current state vector into the final deep reinforcement learning model for forward propagation to obtain the action vector at the current moment. The action vector includes heading adjustment, speed adjustment, flap angle adjustment and flap length adjustment. The deployment operation adjustment module 25 is used to adjust the deployment operation of the deployment ship in the next moment according to the motion vector at the current moment.

[0054] This application provides an electronic device, including a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements any of the above-described schemes of a deep-water laying intelligent control method based on water flow environment perception.

[0055] Specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include onboard memory for caching purposes. The processor may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0056] Memory can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0057] This application also provides a computer-readable medium storing a computer program thereon, which, when executed by a processor, implements the deep-water laying intelligent control method based on water flow environment perception, as described above. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method as described in the embodiments of this application.

[0058] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0059] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments, but should be defined not only by the appended claims, but also by their equivalents. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A deep-water laying-out intelligent control method based on water flow environment perception, characterized in that, include: An initial deep reinforcement learning model is constructed based on the Actor-Critic framework. The initial deep reinforcement learning model includes an Actor network and a Critic network. An experience pool and a reward function are constructed. Sample data is obtained from the experience pool. The parameters of the Actor network and the Critic network are iteratively adjusted according to the sample data through the reward function to generate the final deep reinforcement learning model. The current water flow environment data set is collected by the sensor network on the laying vessel. The water flow environment data set is preprocessed to obtain the current state vector. The state vector includes the average water flow velocity, water depth, soft body strain value, flap angle, flap length and water flow force. The current state vector is input into the final deep reinforcement learning model for forward propagation to obtain the current action vector, which includes heading adjustment, speed adjustment, flap angle adjustment and flap length adjustment. Adjust the deployment operation of the deployment ship in the next moment based on the action vector at the current moment.

2. The intelligent control method for deep-water laying based on water flow environment perception according to claim 1, characterized in that, The reward function is a weighted fusion of the software strain value ratio, flap angle deviation, flap length deviation, speed, and strain over-limit indication function.

3. The intelligent control method for deep-water laying based on water flow environment perception according to claim 2, characterized in that, Based on sample data, the parameters of the Actor and Critic networks are iteratively adjusted using a reward function to generate the final deep reinforcement learning model, including: S11: Obtain the sample state vector s at time t from the sample data. t and label action vector a' t ; S12: Transfer the sample state vector s t Input the Actor network and perform forward propagation to obtain the predicted action vector a at time t. t ; S13: Transfer the label action vector a' t and predicted action vector a t Substitute the reward function to calculate the reward value r at time t. t ; S14: Transfer the sample state vector s t and predicted action vector a t Input the Critic network for state evaluation and generate the value Q at time t; adjust the parameters of the Actor network and Critic network according to the value Q to obtain the optimized Actor network and Critic network. S15: Obtain the sample state vector s at time t+1 from the sample data. t+1 According to the sample state vector s t+1 The optimized value Q' at time t is generated using the optimized Actor network and the optimized Critic network, based on the reward value r. t The TD target value y at time t is obtained by calculating the optimized value Q'. t ; S16: Based on value Q and target TD value y t Calculate the loss value at time t; S17: Repeat steps S11-S16 until the loss value converges to the preset value, generating the final deep reinforcement learning model.

4. The intelligent control method for deep-water laying based on water flow environment perception according to claim 3, characterized in that, The calculation process for the reward value includes: The sample speed is obtained from the sample data, the predicted software strain value, the predicted flap angle and the predicted flap length are obtained from the predicted action vector, and the tag software strain value, the tag flap angle and the tag flap length are obtained from the tag action vector. The ratio of the predicted software strain value to the tag software strain value is substituted into the software strain value ratio term. The difference between the predicted flap angle and the tag flap angle is substituted into the flap angle deviation term. The difference between the predicted flap length and the tag flap length is substituted into the flap length deviation term. The sample speed is substituted into the speed term. The predicted software strain value is substituted into the strain over-limit indication function to calculate the reward value.

5. The intelligent control method for deep-water laying based on water flow environment perception according to claim 1, characterized in that, The process of obtaining the state vector includes: The average flow velocity, water depth, soft sprue strain value, flap angle, flap length, water density, soft sprue flow resistance coefficient, average flow velocity within the draft range of the sprue-laying vessel, angle between the flow direction and the longitudinal axis of the sprue-laying vessel, soft sprue width, and water depth at the soft sprue laying location are obtained from the water flow environment data set. The projected area of ​​the soft raft perpendicular to the water flow direction is calculated based on the width of the soft raft, the water depth at the soft raft laying location, and the angle between the water flow direction and the longitudinal axis of the rafting vessel. The hydrodynamic force is calculated based on the density of water, the hydrodynamic drag coefficient of the soft sprue, the average flow velocity within the draft range of the sprue vessel, and the projected area of ​​the soft sprue perpendicular to the water flow direction. The state vector consists of the average flow velocity, water depth, soft body strain value, flap angle, flap length, and flow force.

6. The intelligent control method for deep-water laying based on water flow environment perception according to claim 1, characterized in that, The step of inputting the current state vector into the final deep reinforcement learning model for forward propagation to obtain the current action vector includes: Obtain the weight matrix, bias vector, activation function, and hyperbolic tangent function of the Actor network in the final deep reinforcement learning model; Substitute the current state vector, weight matrix, and bias vector into the activation function to generate the first result; Substitute the first result into the hyperbolic tangent function to generate the action vector at the current moment.

7. The intelligent control method for deep-water laying based on water flow environment perception according to claim 1, characterized in that, The step of adjusting the deployment operation of the deployment ship in the next moment based on the action vector at the current moment includes: Get the current heading, current speed, current flap angle, and current flap length; Calculate the heading for the next moment based on the current heading and heading adjustment amount; The ship's speed at the next moment is calculated based on the current ship speed and the speed adjustment amount. The flip angle for the next moment is calculated based on the current flip angle and the amount of flip angle adjustment. The flap length at the next moment is calculated based on the current flap length and the flap length adjustment amount. The laying operation of the laying vessel is adjusted according to the course, speed, tilting angle, and tilting length at the next moment.

8. A deep-water laying-out intelligent control device based on water flow environment perception, used to implement the deep-water laying-out intelligent control method based on water flow environment perception as described in any one of claims 1 to 7, characterized in that, The device includes: The initial deep reinforcement learning model building module is used to build an initial deep reinforcement learning model based on the Actor-Critic framework. The initial deep reinforcement learning model includes an Actor network and a Critic network. The final deep reinforcement learning model acquisition module is used to construct the experience pool and reward function, obtain sample data from the experience pool, and iteratively adjust the parameters of the Actor network and Critic network based on the sample data through the reward function to generate the final deep reinforcement learning model. The state vector acquisition module is used to collect the current water flow environment data set through the sensor network on the laying vessel, preprocess the water flow environment data set, and obtain the current state vector. The state vector includes the average water flow velocity, water depth, soft body strain value, flap angle, flap length, and water flow force. The action vector acquisition module is used to input the current state vector into the final deep reinforcement learning model for forward propagation to obtain the action vector at the current moment. The action vector includes heading adjustment, speed adjustment, flap angle adjustment and flap length adjustment. The deployment operation adjustment module is used to adjust the deployment operation of the deployment ship in the next moment based on the motion vector at the current moment.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the intelligent control method for deep-water laying based on water flow environment perception as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the intelligent control method for deep-water laying based on water flow environment perception as described in any one of claims 1 to 7.

Images