Desilting robot intelligent control method and system based on deep learning

Abstract

The invention provides a desilting robot intelligent control method and system based on deep learning, and relates to the technical field of intelligent control, and the method comprises the steps: collecting environment information through a multi-source sensor, extracting environment features through a convolutional neural network and a recurrent neural network, and building a three-dimensional dynamic model of a desilting robot operation environment; based on the model and a hierarchical reward function, training a dual neural network by using a depth deterministic strategy gradient algorithm to obtain an intelligent control strategy model; a motion track output by the model is optimized by an improved ant colony algorithm of a double-target pheromone updating strategy based on path length and energy consumption and then is input into a layered model prediction controller, and cooperative control over a dredging robot propeller and a dredging device is achieved. The working efficiency and the energy utilization rate of the dredging robot are effectively improved, and the dredging cost is reduced.

Description

Technical Field

[0001] The present invention relates to intelligent control technology, and in particular to an intelligent control method and system for a dredging robot based on deep learning. Background Art

[0002] Traditional dredging methods mainly rely on manual operation or simple mechanical equipment, which have problems such as low efficiency, high cost, and high safety risks. In recent years, with the rapid development of robotics and artificial intelligence, dredging robots based on deep learning have become a research hotspot, providing a new solution for realizing intelligent and automated dredging. However, the existing dredging robot control methods still have some defects and shortcomings: Limited environmental perception capabilities: Most existing dredging robots use simple sensors for environmental perception, which makes it difficult to obtain complete information about complex underwater environments, resulting in low control accuracy and operating efficiency. Especially in turbid water bodies, traditional visual sensors are difficult to work effectively, limiting the application scope of robots.

[0003] The control strategy is not intelligent enough: Traditional dredging robot control strategies are mostly based on pre-set rules or simple feedback control, which is difficult to adapt to the complex and changeable underwater environment. When environmental factors such as water flow and sediment change, the robot's control performance will be significantly affected, and it may even fail to complete the dredging task.

[0004] It is difficult to balance work efficiency and energy consumption: dredging operations usually consume a lot of energy. How to improve work efficiency and reduce energy consumption is an important challenge for dredging robot control. Most existing control methods only focus on a single indicator of work efficiency or energy consumption, lack consideration of the balance between the two, and it is difficult to achieve the optimal control effect. Summary of the invention

[0005] The embodiments of the present invention provide a deep learning-based intelligent control method and system for a dredging robot, which can solve the problems in the prior art.

[0006] According to a first aspect of the embodiments of the present invention, Provides an intelligent control method for dredging robots based on deep learning, including: The multi-source information of the dredging robot's working environment is collected through underwater cameras, sonar sensors, pressure sensors and gyroscopes, and the multi-source information is input into a convolutional neural network to extract spatial features. At the same time, a recurrent neural network is used to extract temporal features. The extracted spatial features and temporal features are fused to obtain an environmental feature vector, and a three-dimensional dynamic model of the dredging robot's working environment is established based on the environmental feature vector. Establish a state-action space including the state information, motion parameters and dredging operation parameters of the dredging robot, and construct the three-dimensional dynamic model as an environmental interaction module; construct a hierarchical reward function combining the operating efficiency and energy consumption indicators according to the state transfer output of the environmental interaction module, construct a dual neural network architecture based on the deep deterministic policy gradient algorithm, and train the dual neural network architecture based on the hierarchical reward function to obtain an intelligent control strategy model of the dredging robot; The motion trajectory of the dredging robot is output based on the intelligent control strategy model, and the improved ant colony algorithm is used to calculate the motion trajectory to obtain the optimal motion trajectory, wherein the improved ant colony algorithm includes a dual-objective pheromone update strategy based on path length and energy consumption; the optimal motion trajectory is input into a hierarchical model predictive controller, and the propeller and dredging device of the dredging robot are collaboratively controlled through the hierarchical model predictive controller.

[0007] In an optional embodiment, The multi-source information of the working environment of the dredging robot is collected by underwater cameras, sonar sensors, pressure sensors and gyroscopes, the multi-source information is input into a convolutional neural network to extract spatial features, and the time series features are extracted by a recurrent neural network. The extracted spatial features and time series features are fused to obtain an environmental feature vector. The steps of establishing a three-dimensional dynamic model of the working environment of the dredging robot based on the environmental feature vector include: Perform retinal neural network restoration on the image data collected by the underwater camera, perform noise suppression and target enhancement on the sonar sensor data, and perform Kalman filtering denoising on the pressure sensor and gyroscope data to obtain pre-processed multi-source information data; The preprocessed multi-source information data are respectively input into the dual-stream feature extraction network, wherein the spatial feature extraction adopts an improved residual neural network, the improved residual neural network comprises residual blocks, each residual block is sequentially provided with a first convolutional layer for dimensionality reduction, a second convolutional layer for feature extraction and a third convolutional layer for dimensionality increase, to obtain a spatial feature vector; the temporal feature extraction adopts a bidirectional long short-term memory network, the bidirectional long short-term memory network comprises multiple layers of bidirectional long short-term memory layers, to obtain a temporal feature vector; the spatial feature vector and the temporal feature vector are adaptively weighted fused through an attention mechanism to obtain an environmental feature vector; A three-dimensional dynamic model of the working environment of the dredging robot is constructed based on the environmental feature vector, including using an improved OctoMap algorithm to construct a global environmental map, the improved OctoMap algorithm including an adaptive voxel resolution mechanism, a dynamic probability model based on Bayesian updating and a sensor fusion strategy based on distance weighting; a local area terrain model is established by using the environmental feature vector and Gaussian process regression based on anisotropic kernel function, and the kernel function parameters are updated by an online hyperparameter optimization algorithm; based on the local area terrain model, a point cloud registration algorithm based on normal distribution transformation is used to reconstruct the local environment; a multi-layer Kalman filter is designed to dynamically estimate the water velocity field, sediment distribution and obstacle motion state, and a confidence-based inter-layer information transmission mechanism and an environmental change detector are used to update the three-dimensional dynamic model in real time.

[0008] In an optional embodiment, The steps of establishing a state action space including the state information, motion parameters and dredging operation parameters of the dredging robot and constructing the three-dimensional dynamic model as an environmental interaction module include: The state-action space includes a state space and an action space, wherein the state space includes a dredging robot posture vector, a velocity vector and a dredging system working parameter vector, the dredging robot posture vector includes a position parameter and a posture parameter of the dredging robot in a three-dimensional space, the velocity vector includes a linear velocity parameter and an angular velocity parameter, and the dredging system working parameter vector includes a main pump flow parameter, an auxiliary pump flow parameter, a main pump pressure parameter and an auxiliary pump pressure parameter; the action space includes a motion control amount and a dredging parameter adjustment amount, the motion control amount includes a position adjustment parameter and a posture adjustment parameter, and the dredging parameter adjustment amount includes a main pump flow adjustment parameter, an auxiliary pump flow adjustment parameter, a main pump pressure adjustment parameter and an auxiliary pump pressure adjustment parameter; The three-dimensional dynamic model is constructed as an environment interaction module, which includes an environment state mapping unit, an action execution unit and an environment feedback unit, wherein the environment state mapping unit maps the environment feature vector to the state space to obtain a state output, the action execution unit converts the action in the action space into an execution instruction based on the underwater motion characteristics of the dredging robot, and the environment feedback unit generates a state transfer output based on the state output and the execution instruction, and the state transfer output includes a state transfer probability and an immediate reward value.

[0009] In an optional embodiment, The steps of constructing a hierarchical reward function combining work efficiency and energy consumption indicators according to the state transition output of the environmental interaction module, constructing a dual neural network architecture based on a deep deterministic policy gradient algorithm, and training the dual neural network architecture based on the hierarchical reward function to obtain an intelligent control strategy model of the dredging robot include: Constructing a hierarchical reward function according to the state transfer output, wherein the hierarchical reward function includes an operation efficiency reward function, an energy consumption optimization reward function, and a safety constraint reward function; A dual neural network architecture is designed based on a deep deterministic policy gradient algorithm. The dual neural network architecture includes a policy network and an evaluation network. Both the policy network and the evaluation network adopt a multi-layer fully connected structure. The policy network adopts a deterministic policy gradient algorithm to update parameters, and the evaluation network adopts a temporal difference learning algorithm to update parameters. The policy network and the evaluation network respectively set target networks, and the target networks track parameter changes of the corresponding networks through parameter soft update. The dual neural network architecture is trained based on the hierarchical reward function, including collecting the state transition output sequence of the environment interaction module and storing it in the experience replay buffer; randomly sampling training samples from the experience replay buffer, mapping the state transition probability and the immediate reward value in the training samples into the reward value of the work efficiency dimension, the reward value of the energy consumption dimension and the reward value of the safety dimension through the hierarchical reward function, and calculating the total reward based on the adaptive weight method; using the total reward to train the policy network and the evaluation network; when the evaluation value of the total reward converges and is greater than a preset threshold, saving the parameters of the current policy network as the optimal parameters of the intelligent control strategy model.

[0010] In an optional embodiment, The step of constructing a hierarchical reward function according to the state transfer output, wherein the hierarchical reward function includes an operation efficiency reward function, an energy consumption optimization reward function and a safety constraint reward function comprises: Construct a reward function mapping module for state transfer output, wherein the reward function mapping module adopts a multi-layer perceptron structure, wherein the input layer of the multi-layer perceptron structure receives the state transfer probability and the immediate reward value of the state transfer output, the hidden layer of the multi-layer perceptron structure uses a ReLU activation function, and the output layer of the multi-layer perceptron structure uses a hyperbolic tangent function to map the output value of the hidden layer into an efficiency dimension reward value, an energy consumption dimension reward value, and a safety dimension reward value; An operating efficiency reward function is constructed based on the efficiency dimension reward value, and the operating efficiency reward function includes a dredging area per unit time item and a dredging depth compliance rate item, wherein the dredging area per unit time item is obtained by double integration of the dredging velocity field and the dredging depth indication function, and the dredging depth compliance rate item is calculated by the ratio of the number of qualified sampling points to the total number of sampling points; an energy consumption optimization reward function is constructed based on the energy consumption dimension reward value, and the energy consumption optimization reward function includes a motion power consumption item and a pump group power consumption item, wherein the motion power consumption item is calculated based on the input power of the motors of each joint of the dredging robot, and the pump group power consumption item is calculated based on the motor input power of the main pump and the auxiliary pump; a safety constraint reward function is constructed based on the safety dimension reward value, and the safety constraint reward function includes an obstacle distance item and a posture stability item, wherein the obstacle distance item is obtained by calculating the minimum Euclidean distance between the position of the dredging robot and the positions of all obstacles in the environment map, and the posture stability item is obtained by calculating the distance from the centroid projection point to the supporting polygon; The total reward function is constructed by combining the work efficiency reward function, energy consumption optimization reward function and safety constraint reward function through an adaptive weight method. The adaptive weight method calculates the adaptive weight coefficients of the work efficiency reward function, energy consumption optimization reward function and safety constraint reward function based on the normalized gradient values ​​of the work efficiency target, energy consumption optimization target and safety constraint target. The adaptive weight coefficients are normalized by the softmax function.

[0011] In an optional embodiment, Based on the motion trajectory of the dredging robot output by the intelligent control strategy model, the motion trajectory is calculated by using an improved ant colony algorithm to obtain the optimal motion trajectory. The improved ant colony algorithm includes the steps of a dual-target pheromone update strategy based on path length and energy consumption, including: Inputting the state transfer output of the environment interaction module into the intelligent control strategy model to obtain an action sequence, and mapping the action sequence according to the underwater motion characteristics of the dredging robot to obtain an initial motion trajectory of the dredging robot; The nodes of the initial motion trajectory are used as the search space of the improved ant colony algorithm to construct dual-target pheromones, wherein the dual-target pheromones include path length pheromones and energy consumption pheromones, wherein the path length pheromones are calculated based on the Euclidean distance between the trajectory nodes, and the energy consumption pheromones are calculated based on the motion power consumption of the dredging robot and the power consumption of the pump group; Constructing a path selection probability based on the dual-target pheromone and heuristic information, wherein the heuristic information includes obstacle distance information and posture stability information, and the path selection probability is calculated by combining a weighted product of the dual-target pheromone and the heuristic information; Generating candidate motion trajectories in the search space based on the path selection probability, calculating a normalized target value based on the path length and energy consumption of each candidate motion trajectory, calculating the weight coefficients of the path length pheromone and the energy consumption pheromone by using the normalized target value through an exponential function, and updating the dual-target pheromone according to the weight coefficients of the path length pheromone and the energy consumption pheromone; The candidate motion trajectories generated during the search process of the improved ant colony algorithm are evaluated using a total reward function, wherein the total reward function is calculated based on a hierarchical reward function, and the candidate motion trajectory with the maximum total reward value is selected as the optimal motion trajectory of the dredging robot.

[0012] In an optional embodiment, The optimal motion trajectory is input into a hierarchical model prediction controller, and the step of cooperatively controlling the propeller and the dredging device of the dredging robot by the hierarchical model prediction controller includes: Discretize the optimal motion trajectory into a posture sequence including a position vector and an attitude angle vector, construct a posture prediction equation, and construct an objective function of the posture control layer based on the posture error weight matrix and the control amount weight matrix, wherein the posture error weight matrix is ​​used to balance the weights of the position error and the posture error, and the control amount weight matrix is ​​used to balance the weights of the thruster output; based on the output of the posture control layer, establish a pump group system model including the flow and pressure of the main and auxiliary pumps, and construct an objective function of the working condition control layer based on the working condition error weight matrix and the control amount weight matrix, wherein the working condition error weight matrix is ​​used to balance the weights of the flow error and the pressure error, and the control amount weight matrix is ​​used to balance the weights of the pump group parameters; Constructing hierarchical constraint conditions including posture constraints, working condition constraints and coupling constraints, wherein the posture constraints include speed constraints and attitude angle constraints, the working condition constraints include pump group flow constraints and pressure constraints, and the coupling constraints include attitude and flow coupling constraints and depth and pressure coupling constraints; The objective function of the posture control layer and the objective function of the operating condition control layer are weightedly combined to obtain a comprehensive objective function. Under the hierarchical constraints, the sequential quadratic programming method is used to solve the comprehensive objective function to obtain thruster control instructions and dredging device control instructions. The actual posture error and operating condition error are calculated based on the thruster control instructions and the dredging device control instructions, and an error compensation controller is constructed to compensate for the posture error and the operating condition error, and the final thrust of the thruster and the pump group parameters of the dredging device are output.

[0013] According to a second aspect of the embodiments of the present invention, Provides a deep learning-based intelligent control system for dredging robots, including: The first unit is used to collect multi-source information of the working environment of the dredging robot through underwater cameras, sonar sensors, pressure sensors and gyroscopes, input the multi-source information into a convolutional neural network to extract spatial features, and use a recurrent neural network to extract temporal features, and fuse the extracted spatial features and temporal features to obtain an environmental feature vector, and establish a three-dimensional dynamic model of the working environment of the dredging robot based on the environmental feature vector; The second unit is used to establish a state action space including the state information, motion parameters and dredging operation parameters of the dredging robot, and construct the three-dimensional dynamic model as an environmental interaction module; construct a hierarchical reward function combining the operating efficiency and energy consumption indicators according to the state transfer output of the environmental interaction module, construct a dual neural network architecture based on a deep deterministic policy gradient algorithm, and train the dual neural network architecture based on the hierarchical reward function to obtain an intelligent control strategy model of the dredging robot; The third unit is used to output the motion trajectory of the dredging robot based on the intelligent control strategy model, and use the improved ant colony algorithm to calculate the motion trajectory to obtain the optimal motion trajectory, wherein the improved ant colony algorithm includes a dual-objective pheromone update strategy based on path length and energy consumption; the optimal motion trajectory is input into the hierarchical model predictive controller, and the propeller and dredging device of the dredging robot are collaboratively controlled through the hierarchical model predictive controller.

[0014] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0015] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0016] The present invention establishes an accurate three-dimensional dynamic environment model through deep learning, and conducts reinforcement learning training in combination with operating efficiency and energy consumption indicators, so that the dredging robot can autonomously learn the optimal dredging strategy, thereby improving the efficiency of dredging operations.

[0017] The present invention takes energy consumption indicators into consideration in both the reward function and the ant colony algorithm optimization, so that the dredging robot can reduce energy consumption as much as possible while completing the dredging task, thereby achieving the goal of energy saving and environmental protection.

[0018] The control method of the present invention is based on deep learning and reinforcement learning, which enables the dredging robot to have the ability to autonomously perceive the environment, make autonomous decisions and autonomously control, can adapt to complex and changeable underwater environments, and can complete dredging operations without human intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flowchart of an intelligent control method of a dredging robot based on deep learning according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of an intelligent control system for a dredging robot based on deep learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0022] Figure 1 FIG. 1 is a flow chart of an intelligent control method for a dredging robot based on deep learning according to an embodiment of the present invention. Figure 1 As shown, the method includes: S1. Collect multi-source information of the working environment of the dredging robot through underwater cameras, sonar sensors, pressure sensors and gyroscopes, input the multi-source information into a convolutional neural network to extract spatial features, and use a recurrent neural network to extract temporal features, and fuse the extracted spatial features and temporal features to obtain an environmental feature vector, and establish a three-dimensional dynamic model of the working environment of the dredging robot based on the environmental feature vector; S2. Establish a state-action space including the state information, motion parameters and dredging operation parameters of the dredging robot, and construct the three-dimensional dynamic model as an environmental interaction module; construct a hierarchical reward function combining the operating efficiency and energy consumption indicators according to the state transfer output of the environmental interaction module, construct a dual neural network architecture based on the deep deterministic policy gradient algorithm, train the dual neural network architecture based on the hierarchical reward function, and obtain an intelligent control strategy model of the dredging robot; S3. Output the motion trajectory of the dredging robot based on the intelligent control strategy model, and use the improved ant colony algorithm to calculate the motion trajectory to obtain the optimal motion trajectory, wherein the improved ant colony algorithm includes a dual-objective pheromone update strategy based on path length and energy consumption; input the optimal motion trajectory into the hierarchical model predictive controller, and coordinately control the propeller and dredging device of the dredging robot through the hierarchical model predictive controller.

[0023] In an optional embodiment, The multi-source information of the working environment of the dredging robot is collected by underwater cameras, sonar sensors, pressure sensors and gyroscopes, the multi-source information is input into a convolutional neural network to extract spatial features, and the time series features are extracted by a recurrent neural network. The extracted spatial features and time series features are fused to obtain an environmental feature vector. The steps of establishing a three-dimensional dynamic model of the working environment of the dredging robot based on the environmental feature vector include: Perform retinal neural network restoration on the image data collected by the underwater camera, perform noise suppression and target enhancement on the sonar sensor data, and perform Kalman filtering denoising on the pressure sensor and gyroscope data to obtain pre-processed multi-source information data; The preprocessed multi-source information data are respectively input into the dual-stream feature extraction network, wherein the spatial feature extraction adopts an improved residual neural network, the improved residual neural network comprises residual blocks, each residual block is sequentially provided with a first convolutional layer for dimensionality reduction, a second convolutional layer for feature extraction and a third convolutional layer for dimensionality increase, to obtain a spatial feature vector; the temporal feature extraction adopts a bidirectional long short-term memory network, the bidirectional long short-term memory network comprises multiple layers of bidirectional long short-term memory layers, to obtain a temporal feature vector; the spatial feature vector and the temporal feature vector are adaptively weighted fused through an attention mechanism to obtain an environmental feature vector; A three-dimensional dynamic model of the working environment of the dredging robot is constructed based on the environmental feature vector, including using an improved OctoMap algorithm to construct a global environmental map, the improved OctoMap algorithm including an adaptive voxel resolution mechanism, a dynamic probability model based on Bayesian updating and a sensor fusion strategy based on distance weighting; a local area terrain model is established by using the environmental feature vector and Gaussian process regression based on anisotropic kernel function, and the kernel function parameters are updated by an online hyperparameter optimization algorithm; based on the local area terrain model, a point cloud registration algorithm based on normal distribution transformation is used to reconstruct the local environment; a multi-layer Kalman filter is designed to dynamically estimate the water velocity field, sediment distribution and obstacle motion state, and a confidence-based inter-layer information transmission mechanism and an environmental change detector are used to update the three-dimensional dynamic model in real time.

[0024] Exemplarily, first, the collected multi-source information is preprocessed. The image data collected by underwater cameras usually has problems such as blur and distortion, and retinal neural networks are used for restoration. The network simulates the hierarchical structure of the human retina, gradually extracts image features through multi-layer convolution and pooling operations, and repairs the image in combination with local and global information to improve the clarity and contrast of the image. For example, for a turbid underwater image with a resolution of 1920x1080, after being processed by the retinal neural network, the clarity of the image is improved by 20%, and the details are more clearly visible. Sonar sensor data often contains noise interference. Noise suppression and target enhancement techniques, such as wavelet transform or spectral subtraction, are used to remove noise and enhance target signals to improve the signal-to-noise ratio. For example, a sonar signal containing a lot of noise is processed, and the signal-to-noise ratio is increased from 5dB to 15dB. Pressure sensor and gyroscope data may also have noise and drift. Kalman filtering is used for denoising to obtain more accurate attitude and position information. For example, the raw data of the gyroscope has obvious drift, but after Kalman filtering, the drift is reduced by 80%.

[0025] Next, the preprocessed multi-source information data is input into the dual-stream feature extraction network. Spatial feature extraction uses an improved residual neural network, which consists of multiple residual blocks. Each residual block contains three convolutional layers: the first layer is used to reduce the data dimension and reduce the amount of calculation; the second layer is used to extract features and capture the key information in the image; the third layer is used to increase the dimension and restore the data dimension. This structure can effectively avoid the gradient disappearance and gradient explosion problems and improve the training efficiency and performance of the network. For example, after inputting a 256x256 image and processing it with the residual block, the output feature map size remains unchanged, but the features are richer. Temporal feature extraction uses a bidirectional long short-term memory network, which can capture long-term dependencies in time series data. Through bidirectional processing, past and future information can be considered at the same time to improve the accuracy of feature extraction. For example, a section of pressure sensor data containing 100 time steps can extract more representative temporal features after being processed by a bidirectional long short-term memory network. Then, the attention mechanism is used to adaptively weight the spatial feature vector and the temporal feature vector. The attention mechanism can dynamically assign weights according to the importance of different features to improve the efficiency of feature fusion. For example, at a certain moment, image information is more important than sonar information, and the attention mechanism will give higher weight to image features.

[0026] Finally, a three-dimensional dynamic model of the dredging robot's working environment is constructed based on the fused environmental feature vector. The global environment map is constructed using the improved OctoMap algorithm (octree map algorithm). The algorithm uses an adaptive voxel resolution mechanism to dynamically adjust the voxel size according to the complexity of the environment, reducing storage space and calculation while ensuring map accuracy. At the same time, a dynamic probability model based on Bayesian update is used to continuously update the map based on new observation data to improve the real-time and accuracy of the map. In addition, a distance-weighted sensor fusion strategy is used to fuse data from different sensors to improve the reliability of the map. For example, in areas with dense obstacles, a smaller voxel resolution is used, while in open areas, a larger voxel resolution is used. The local area terrain model is established using Gaussian process regression based on anisotropic kernel functions. This method can adaptively adjust the kernel function parameters according to the distribution of the data to improve the fitting accuracy of the model. For example, in areas with large slopes, the use of anisotropic kernel functions can better fit terrain changes. The local environment reconstruction uses a point cloud registration algorithm based on normal distribution transformation. This algorithm can accurately register point cloud data from different perspectives to construct a complete three-dimensional model. For example, by registering sonar point cloud data from multiple perspectives, the complete shape of underwater obstacles can be reconstructed. The dynamic estimation of water velocity field, sediment distribution and obstacle motion state uses a multi-layer Kalman filter. The filter can estimate and predict the state of the system based on the state equation and observation equation of the system. For example, the speed and direction of the water flow can be estimated based on the data of the water velocity sensor. At the same time, the confidence-based inter-layer information transmission mechanism and environmental change detector are used to update the three-dimensional dynamic model in real time. For example, when a new obstacle is detected, the map is immediately updated and the robot's path is replanned.

[0027] The present invention integrates multi-source information and adopts advanced feature extraction and fusion algorithms, which can more accurately perceive various parameters of the underwater environment, including terrain, water flow, obstacles, etc., and avoid the influence of insufficient information or errors of a single sensor; the dynamic probability model, online hyperparameter optimization and multi-layer Kalman filtering and other technologies are adopted to enable the model to be updated in real time to adapt to the dynamic changes of the underwater environment, such as water flow changes, sediment movement, obstacle displacement, etc., to ensure the effectiveness and reliability of the model; the accurate three-dimensional dynamic model provides the dredging robot with more comprehensive environmental information, supports more accurate path planning and operation control, thereby improving the efficiency of dredging operations and reducing operation time and costs.

[0028] In an optional embodiment, The steps of establishing a state action space including the state information, motion parameters and dredging operation parameters of the dredging robot and constructing the three-dimensional dynamic model as an environmental interaction module include: The state-action space includes a state space and an action space, wherein the state space includes a dredging robot posture vector, a velocity vector and a dredging system working parameter vector, the dredging robot posture vector includes a position parameter and a posture parameter of the dredging robot in a three-dimensional space, the velocity vector includes a linear velocity parameter and an angular velocity parameter, and the dredging system working parameter vector includes a main pump flow parameter, an auxiliary pump flow parameter, a main pump pressure parameter and an auxiliary pump pressure parameter; the action space includes a motion control amount and a dredging parameter adjustment amount, the motion control amount includes a position adjustment parameter and a posture adjustment parameter, and the dredging parameter adjustment amount includes a main pump flow adjustment parameter, an auxiliary pump flow adjustment parameter, a main pump pressure adjustment parameter and an auxiliary pump pressure adjustment parameter; The three-dimensional dynamic model is constructed as an environment interaction module, which includes an environment state mapping unit, an action execution unit and an environment feedback unit, wherein the environment state mapping unit maps the environment feature vector to the state space to obtain a state output, the action execution unit converts the action in the action space into an execution instruction based on the underwater motion characteristics of the dredging robot, and the environment feedback unit generates a state transfer output based on the state output and the execution instruction, and the state transfer output includes a state transfer probability and an immediate reward value.

[0029] Exemplarily, the establishment of the state-action space of the dredging robot and the method of constructing the environment interaction module, the core of which is to build a framework that can accurately reflect the state and action of the dredging robot and can interact with the simulated environment. First, the state-action space of the dredging robot is constructed. The state space describes the complete state information of the dredging robot at a specific moment. It contains three main vectors: the posture vector, the velocity vector, and the dredging system working parameter vector. The posture vector describes the position and posture of the robot in three-dimensional space. For example, the Cartesian coordinate system (x, y, z) can be used to represent the position, and the Euler angle (roll, pitch, yaw) can be used to represent the posture. The velocity vector describes the linear velocity and angular velocity of the robot. For example, the linear velocity can be expressed by the velocity components in the three directions (v x , v y , v z ) is expressed as, the angular velocity can be expressed as the rotation speed around the three axes (ω x ,ω y ,ω z ). The dredging system working parameter vector describes the working state of the dredging system, for example, the main pump flow, auxiliary pump flow, main pump pressure and auxiliary pump pressure. Assume that the robot is at position (1, 2, 3), posture (0.1, 0.2, 0.3), linear velocity (0.4, 0.5, 0.6), angular velocity (0.7, 0.8, 0.9), main pump flow is 10, auxiliary pump flow is 5, main pump pressure is 20, and auxiliary pump pressure is 10.

[0030] The action space describes all possible actions that can be applied to the dredging robot. It contains motion control quantities and dredging parameter adjustment quantities. The motion control quantity describes the adjustment of the robot's position and posture, for example, the target position and target posture can be adjusted. The dredging parameter adjustment quantity describes the adjustment of the dredging system's working parameters, for example, the main pump flow, auxiliary pump flow, main pump pressure, and auxiliary pump pressure can be adjusted. Assume that the action at this moment is to adjust the position to (2, 3, 4), the posture to (0.2, 0.3, 0.4), the main pump flow to 12, the auxiliary pump flow to 7, the main pump pressure to 22, and the auxiliary pump pressure to 12.

[0031] Next, we build the environment interaction module, which is a bridge between the state action space of the dredging robot and the simulation environment. It consists of three units: environment state mapping unit, action execution unit, and environment feedback unit.

[0032] The environmental state mapping unit maps the environmental feature vector to the state space. It receives feature vectors from the simulated environment, such as water flow velocity, water depth, sediment concentration, etc., and converts these features into state outputs in the state space. For example, water flow velocity may affect the velocity vector of the robot, and sediment concentration may affect the working parameter vector of the dredging system. Assume that the environmental feature vector is water flow velocity (0.1, 0.2, 0.3), water depth 10, and sediment concentration 0.5. After mapping, the state outputs are position (1.1, 2.2, 3.3), posture (0.11, 0.22, 0.33), linear velocity (0.44, 0.55, 0.66), angular velocity (0.77, 0.88, 0.99), main pump flow 11, auxiliary pump flow 6, main pump pressure 21, and auxiliary pump pressure 11.

[0033] The action execution unit converts the actions in the action space into execution instructions. It receives actions from the action space and converts them into specific execution instructions according to the underwater motion characteristics of the dredging robot, such as propeller speed, rudder angle, etc. For example, the action of adjusting the position may be converted into a change in the propeller speed, and the action of adjusting the posture may be converted into a change in the rudder angle. Suppose the action is to adjust the position to (2, 3, 4) and the posture to (0.2, 0.3, 0.4). The action execution unit converts it into a left propeller speed of 100, a right propeller speed of 120, and a rudder angle of 10.

[0034] The environmental feedback unit generates a state transition output based on the state output and the execution instruction. It inputs the state output and the execution instruction into the simulation environment, receives feedback from the environment, and generates a state transition output, including a state transition probability and an immediate reward value. The state transition probability describes the probability of transitioning from the current state to the next state, and the immediate reward value describes the reward obtained by performing the action in the current state. For example, if a large amount of silt is removed, a higher reward value will be obtained. Assume that the current state is position (1.1, 2.2, 3.3), and the execution instruction is left propeller speed 100, right propeller speed 120, and rudder angle 10. The environmental feedback unit inputs this information into the simulation environment, obtains the next state as position (1.5, 2.5, 3.5), and obtains an immediate reward value of 10.

[0035] By constructing a state-action space and an environment interaction module, the present invention can more accurately control the movement and dredging operation of the dredging robot, thereby improving the dredging efficiency; by training and optimizing in a simulated environment, the trial and error cost in actual dredging operations can be reduced, thereby reducing the dredging cost; by simulating and analyzing various complex situations in a simulated environment, the safety of the dredging robot in actual operations can be improved.

[0036] In an optional embodiment, The steps of constructing a hierarchical reward function combining work efficiency and energy consumption indicators according to the state transition output of the environmental interaction module, constructing a dual neural network architecture based on a deep deterministic policy gradient algorithm, and training the dual neural network architecture based on the hierarchical reward function to obtain an intelligent control strategy model of the dredging robot include: Constructing a hierarchical reward function according to the state transfer output, wherein the hierarchical reward function includes an operation efficiency reward function, an energy consumption optimization reward function, and a safety constraint reward function; A dual neural network architecture is designed based on a deep deterministic policy gradient algorithm. The dual neural network architecture includes a policy network and an evaluation network. Both the policy network and the evaluation network adopt a multi-layer fully connected structure. The policy network adopts a deterministic policy gradient algorithm to update parameters, and the evaluation network adopts a temporal difference learning algorithm to update parameters. The policy network and the evaluation network respectively set target networks, and the target networks track parameter changes of the corresponding networks through parameter soft update. The dual neural network architecture is trained based on the hierarchical reward function, including collecting the state transition output sequence of the environment interaction module and storing it in the experience replay buffer; randomly sampling training samples from the experience replay buffer, mapping the state transition probability and the immediate reward value in the training samples into the reward value of the work efficiency dimension, the reward value of the energy consumption dimension and the reward value of the safety dimension through the hierarchical reward function, and calculating the total reward based on the adaptive weight method; using the total reward to train the policy network and the evaluation network; when the evaluation value of the total reward converges and is greater than a preset threshold, saving the parameters of the current policy network as the optimal parameters of the intelligent control strategy model.

[0037] For example, the method for constructing an intelligent control strategy model for a dredging robot aims to improve the operating efficiency and reduce energy consumption of the dredging robot while ensuring operating safety. The core of this method is to construct a hierarchical reward function based on the output of the environment interaction module, and use a deep deterministic policy gradient algorithm to train a dual neural network architecture, and finally obtain an intelligent control strategy model for the dredging robot.

[0038] First, build an environment interaction module. This module simulates the state transition process of the dredging robot in the actual working environment. For example, the module input can be the current position of the dredging robot, the thickness of the silt, the water flow speed and other information, and the output can be the next position of the robot, the amount of silt removed, the energy consumption and other information. Assume that the initial position of the robot is (0, 0), the silt thickness is 10cm, and the water flow speed is 1m / s. After one action, the position of the robot becomes (1, 0), the silt thickness becomes 5cm, and the energy consumption is 10J. This information will be used as the state transfer output to construct the hierarchical reward function.

[0039] Next, construct a hierarchical reward function. This function consists of three parts: the work efficiency reward function, the energy consumption optimization reward function, and the safety constraint reward function. The work efficiency reward function is positively correlated with the amount of sludge removed. For example, the more sludge removed, the higher the reward value. Assuming that the reward value for removing 5cm of sludge is 5, the reward value for removing 10cm of sludge is 10. The energy consumption optimization reward function is negatively correlated with energy consumption. For example, the lower the energy consumption, the higher the reward value. Assuming that the reward value for energy consumption of 10J is -1, the reward value for energy consumption of 5J is -0.5. The safety constraint reward function is used to constrain the behavior of the robot, for example, to prevent the robot from colliding or tilting too much. Assuming that the reward value for the robot to remain stable is 1, and the reward value for a collision is -10.

[0040] Then, a dual neural network architecture is designed. This architecture includes a policy network and an evaluation network, both of which adopt a multi-layer fully connected structure. The policy network is used to output actions according to the current state, and the evaluation network is used to evaluate the value of the current state. The policy network uses a deterministic policy gradient algorithm to update parameters, and the evaluation network uses a temporal difference learning algorithm to update parameters. In addition, both the policy network and the evaluation network are set with a target network, and the parameter changes of the corresponding network are tracked by soft parameter updates to improve the stability of training. For example, the parameters of the target network are updated every 100 steps, and the update method is to copy part of the current network parameters to the target network.

[0041] Subsequently, the dual neural network architecture is trained based on the hierarchical reward function. First, the state transition output sequence of the environment interaction module is collected and stored in the experience replay buffer. For example, 10,000 state transition output sequences are stored. Then, training samples are randomly sampled from the experience replay buffer, and the state transition probabilities and immediate reward values ​​in the training samples are mapped to the work efficiency dimension reward value, energy consumption dimension reward value, and safety dimension reward value through the hierarchical reward function. For example, assuming that the immediate reward value of a state transition is 1, it is mapped to the work efficiency reward value 0.5, the energy consumption reward value -0.2, and the safety reward value 0.3 through the hierarchical reward function. Then, the total reward is calculated based on the adaptive weight method. For example, according to the current training stage, the reward values ​​of the three dimensions are multiplied by different weights respectively, and then added to obtain the total reward. Finally, the total reward is used to train the policy network and the evaluation network. When the evaluation value of the total reward converges and is greater than the preset threshold, the parameters of the current policy network are saved as the optimal parameters of the intelligent control policy model. For example, when the fluctuation of the total reward for 100 consecutive iterations is less than 0.1 and the total reward is greater than 10, the parameters of the policy network are saved.

[0042] The present invention optimizes the control strategy so that the robot can remove silt more quickly and effectively, thereby improving the overall efficiency of the dredging operation; by optimizing the control strategy, the robot can reduce energy consumption as much as possible while completing the dredging task, thereby reducing operating costs; by introducing safety constraints in the reward function, the robot can effectively avoid collisions or other dangerous situations, thereby ensuring operation safety.

[0043] In an optional embodiment, The step of constructing a hierarchical reward function according to the state transfer output, wherein the hierarchical reward function includes an operation efficiency reward function, an energy consumption optimization reward function and a safety constraint reward function comprises: Construct a reward function mapping module for state transfer output, wherein the reward function mapping module adopts a multi-layer perceptron structure, wherein the input layer of the multi-layer perceptron structure receives the state transfer probability and the immediate reward value of the state transfer output, the hidden layer of the multi-layer perceptron structure uses a ReLU activation function, and the output layer of the multi-layer perceptron structure uses a hyperbolic tangent function to map the output value of the hidden layer into an efficiency dimension reward value, an energy consumption dimension reward value, and a safety dimension reward value; An operating efficiency reward function is constructed based on the efficiency dimension reward value, and the operating efficiency reward function includes a dredging area per unit time item and a dredging depth compliance rate item, wherein the dredging area per unit time item is obtained by double integration of the dredging velocity field and the dredging depth indication function, and the dredging depth compliance rate item is calculated by the ratio of the number of qualified sampling points to the total number of sampling points; an energy consumption optimization reward function is constructed based on the energy consumption dimension reward value, and the energy consumption optimization reward function includes a motion power consumption item and a pump group power consumption item, wherein the motion power consumption item is calculated based on the input power of the motors of each joint of the dredging robot, and the pump group power consumption item is calculated based on the motor input power of the main pump and the auxiliary pump; a safety constraint reward function is constructed based on the safety dimension reward value, and the safety constraint reward function includes an obstacle distance item and a posture stability item, wherein the obstacle distance item is obtained by calculating the minimum Euclidean distance between the position of the dredging robot and the positions of all obstacles in the environment map, and the posture stability item is obtained by calculating the distance from the centroid projection point to the supporting polygon; The total reward function is constructed by combining the work efficiency reward function, energy consumption optimization reward function and safety constraint reward function through an adaptive weight method. The adaptive weight method calculates the adaptive weight coefficients of the work efficiency reward function, energy consumption optimization reward function and safety constraint reward function based on the normalized gradient values ​​of the work efficiency target, energy consumption optimization target and safety constraint target. The adaptive weight coefficients are normalized by the softmax function.

[0044] Exemplarily, first, a reward function mapping module is constructed. This module adopts a multi-layer perceptron structure, and its input layer receives the state transition probability and immediate reward value of the dredging robot. For example, the state transition probability can be expressed as the possibility of the robot transferring from the current state to the next state, and the immediate reward can be the reward for completing a subtask. The hidden layer uses the ReLU activation function to perform a nonlinear transformation on the input. The output layer uses the hyperbolic tangent function to map the output value of the hidden layer to three dimensions: efficiency dimension reward value, energy consumption dimension reward value, and safety dimension reward value. Assuming that the input state transition probability is 0.8 and the immediate reward value is 10, after being processed by the multi-layer perceptron, the output efficiency dimension reward value is 0.7, the energy consumption dimension reward value is -0.5 (negative value indicates low energy consumption), and the safety dimension reward value is 0.9.

[0045] Next, the work efficiency reward function is constructed based on the efficiency dimension reward value. This function contains two parts: the desilting area per unit time and the desilting depth compliance rate. The desilting area per unit time is obtained by calculating the integral of the desilting speed field and the desilting depth indicator function. For example, assuming that the desilting speed is 0.5 square meters per second and the desilting depth indicator function is 1 (indicating that the target depth has been reached), the desilting area per unit time is 0.5 square meters per second. The desilting depth compliance rate is calculated by the ratio of the number of qualified sampling points to the total number of sampling points. For example, if 10 sampling points are set and the desilting depth of 8 of them reaches the target depth, the desilting depth compliance rate is 80%.

[0046] Then, an energy consumption optimization reward function is constructed based on the energy consumption dimension reward value. This function also contains two parts: motion power consumption and pump group power consumption. The motion power consumption is calculated based on the input power of the motors of each joint of the dredging robot. For example, assuming that the robot has three joints and the input power of each joint is 10W, 20W, and 30W respectively, the total motion power consumption is 60W. The pump group power consumption is calculated based on the motor input power of the main pump and the auxiliary pump. For example, assuming that the power of the main pump is 100W and the power of the auxiliary pump is 50W, the power consumption of the pump group is 150W.

[0047] Subsequently, a safety constraint reward function is constructed based on the safety dimension reward value. This function also consists of two parts: obstacle distance and posture stability. The obstacle distance is obtained by calculating the minimum Euclidean distance between the position of the dredging robot and the positions of all obstacles in the environment map. For example, assuming that the distance between the robot and the nearest obstacle is 2 meters, the obstacle distance is 2 meters. The posture stability is obtained by calculating the distance from the center of mass projection point to the supporting polygon. For example, assuming that the distance from the center of mass projection point to the supporting polygon is 0.1 meters, the posture stability index is 0.1 meters.

[0048] Finally, the total reward function is constructed by combining the three reward functions through the adaptive weight method. This method calculates the adaptive weight coefficients of the three reward functions based on the normalized gradient values ​​of the work efficiency target, energy consumption optimization target, and safety constraint target. These weight coefficients are normalized by the softmax function. For example, assuming that the normalized gradient value of the work efficiency target is 0.6, the normalized gradient value of the energy consumption optimization target is 0.3, and the normalized gradient value of the safety constraint target is 0.1, after being processed by the softmax function, the corresponding weight coefficients are 0.55, 0.33, and 0.12 respectively. The total reward function is the sum of the products of these three reward functions and their corresponding weight coefficients.

[0049] The present invention quantitatively evaluates the dredging operation efficiency through indicators such as the dredging area per unit time and the dredging depth compliance rate, and integrates them into the reward function, thereby guiding the dredging robot to optimize the operation path and strategy and improve the dredging efficiency; by considering the motion power consumption and the pump group power consumption, the method can effectively reduce the energy consumption of the dredging robot, extend its operation time, and reduce operating costs; through indicators such as obstacle distance and posture stability, the safety status of the dredging robot is monitored and integrated into the reward function, thereby ensuring that the dredging robot maintains a safe and stable state during the operation and avoiding accidents such as collision or overturning.

[0050] In an optional embodiment, Based on the motion trajectory of the dredging robot output by the intelligent control strategy model, the motion trajectory is calculated by using an improved ant colony algorithm to obtain the optimal motion trajectory. The improved ant colony algorithm includes the steps of a dual-target pheromone update strategy based on path length and energy consumption, including: Inputting the state transfer output of the environment interaction module into the intelligent control strategy model to obtain an action sequence, and mapping the action sequence according to the underwater motion characteristics of the dredging robot to obtain an initial motion trajectory of the dredging robot; The nodes of the initial motion trajectory are used as the search space of the improved ant colony algorithm to construct dual-target pheromones, wherein the dual-target pheromones include path length pheromones and energy consumption pheromones, wherein the path length pheromones are calculated based on the Euclidean distance between the trajectory nodes, and the energy consumption pheromones are calculated based on the motion power consumption of the dredging robot and the power consumption of the pump group; Constructing a path selection probability based on the dual-target pheromone and heuristic information, wherein the heuristic information includes obstacle distance information and posture stability information, and the path selection probability is calculated by combining a weighted product of the dual-target pheromone and the heuristic information; Generating candidate motion trajectories in the search space based on the path selection probability, calculating a normalized target value based on the path length and energy consumption of each candidate motion trajectory, calculating the weight coefficients of the path length pheromone and the energy consumption pheromone by using the normalized target value through an exponential function, and updating the dual-target pheromone according to the weight coefficients of the path length pheromone and the energy consumption pheromone; The candidate motion trajectories generated during the search process of the improved ant colony algorithm are evaluated using a total reward function, wherein the total reward function is calculated based on a hierarchical reward function, and the candidate motion trajectory with the maximum total reward value is selected as the optimal motion trajectory of the dredging robot.

[0051] For example, in order to achieve the optimal trajectory planning of the dredging robot, an intelligent control strategy based on an improved ant colony algorithm is proposed. The strategy first generates an initial motion trajectory using the intelligent control strategy model and environmental interaction information, and then uses the improved ant colony algorithm to search and optimize based on the initial trajectory, and finally obtains the optimal trajectory that takes into account both path length and energy consumption.

[0052] First, the dredging robot senses the surrounding environment information through sensors, such as water depth, obstacle location, water flow speed, etc., and inputs this information into the environmental interaction module. The environmental interaction module converts this information into state information and outputs it to the intelligent control strategy model. The intelligent control strategy model outputs a series of action instructions based on the current state information, such as forward, backward, turn left, turn right, start the pump group, etc. Assume that the state information output by the environmental interaction module is that the water depth is 2 meters, there is an obstacle 5 meters ahead, and the water flow speed is 0.5 meters / second. Based on this information, the intelligent control strategy model outputs an action sequence: forward 3 meters, turn right 30 degrees, forward 2 meters, and start the pump group.

[0053] Next, the action sequence is converted into the initial motion trajectory of the dredging robot. According to the underwater motion characteristics of the dredging robot, such as maximum speed, turning radius, pump start time, etc., each action instruction is mapped into a trajectory. For example, "forward 3 meters" can be mapped into a straight line trajectory with a length of 3 meters, and "turn right 30 degrees" can be mapped into a circular arc trajectory with a radius of 1 meter. Assuming that the maximum speed of the dredging robot is 1 meter / second and the turning radius is 1 meter, the above action sequence can be mapped into an initial motion trajectory consisting of two straight lines and a circular arc.

[0054] The nodes of the initial motion trajectory are used as the search space of the improved ant colony algorithm. At each node, path length pheromone and energy consumption pheromone are constructed. Path length pheromone is calculated based on the distance between nodes. The shorter the distance, the higher the pheromone concentration. Assuming that the distance between node A and node B is 1 meter, and the distance between node A and node C is 2 meters, the path length pheromone concentration between node A and node B is higher than the path length pheromone concentration between node A and node C. Energy consumption pheromone is calculated based on the motion power consumption of the dredging robot and the power consumption of the pump group. The lower the energy consumption, the higher the pheromone concentration. Assuming that the energy consumed from node A to node B is 10 joules, and the energy consumed from node A to node C is 20 joules, the energy consumption pheromone concentration between node A and node B is higher than the energy consumption pheromone concentration between node A and node C.

[0055] During the search process, the ant selects the next node based on the path selection probability. The path selection probability is determined by the dual-target pheromone and the heuristic information. The heuristic information includes obstacle distance information and posture stability information. The obstacle distance information indicates the distance between the node and the nearest obstacle. The farther the distance, the higher the selection probability. The posture stability information indicates the posture stability of the dredging robot at the node. The higher the stability, the higher the selection probability. Assuming that node A is 2 meters away from the nearest obstacle and the posture stability is 0.9, and node B is 1 meter away from the nearest obstacle and the posture stability is 0.8, the probability of the ant selecting node A is higher. The path selection probability is calculated by combining the weighted product of the dual-target pheromone and the heuristic information.

[0056] Ants generate candidate motion trajectories in the search space based on the path selection probability. For each candidate trajectory, its path length and energy consumption are calculated and normalized. The weight coefficients of the path length pheromone and the energy consumption pheromone are calculated using the normalized target value through an exponential function. The dual-target pheromone is updated according to the weight coefficient. Assuming that the path length of one candidate trajectory is 10 meters and the energy consumption is 100 joules, and the path length of another candidate trajectory is 20 meters and the energy consumption is 50 joules, the path length pheromone weight coefficient of the first candidate trajectory is higher, and the energy consumption pheromone weight coefficient of the second candidate trajectory is higher.

[0057] All candidate trajectories are evaluated using the total reward function, which is calculated based on the hierarchical reward function and takes into account factors such as path length, energy consumption, obstacle distance, and posture stability. The candidate trajectory with the highest total reward value is selected as the optimal motion trajectory of the dredging robot. Assuming that the total reward value of one candidate trajectory is 0.9 and the total reward value of another candidate trajectory is 0.8, the first candidate trajectory is selected as the optimal trajectory.

[0058] The present invention shortens the dredging time and improves the dredging efficiency by optimizing the motion trajectory; reduces energy consumption and reduces the dredging cost by optimizing the motion trajectory; and improves the safety of the dredging robot by considering factors such as obstacle distance and posture stability.

[0059] In an optional embodiment, The optimal motion trajectory is input into a hierarchical model prediction controller, and the step of cooperatively controlling the propeller and the dredging device of the dredging robot by the hierarchical model prediction controller includes: Discretize the optimal motion trajectory into a posture sequence including a position vector and an attitude angle vector, construct a posture prediction equation, and construct an objective function of the posture control layer based on the posture error weight matrix and the control amount weight matrix, wherein the posture error weight matrix is ​​used to balance the weights of the position error and the posture error, and the control amount weight matrix is ​​used to balance the weights of the thruster output; based on the output of the posture control layer, establish a pump group system model including the flow and pressure of the main and auxiliary pumps, and construct an objective function of the working condition control layer based on the working condition error weight matrix and the control amount weight matrix, wherein the working condition error weight matrix is ​​used to balance the weights of the flow error and the pressure error, and the control amount weight matrix is ​​used to balance the weights of the pump group parameters; Constructing hierarchical constraint conditions including posture constraints, working condition constraints and coupling constraints, wherein the posture constraints include speed constraints and attitude angle constraints, the working condition constraints include pump group flow constraints and pressure constraints, and the coupling constraints include attitude and flow coupling constraints and depth and pressure coupling constraints; The objective function of the posture control layer and the objective function of the operating condition control layer are weightedly combined to obtain a comprehensive objective function. Under the hierarchical constraints, the sequential quadratic programming method is used to solve the comprehensive objective function to obtain thruster control instructions and dredging device control instructions. The actual posture error and operating condition error are calculated based on the thruster control instructions and the dredging device control instructions, and an error compensation controller is constructed to compensate for the posture error and the operating condition error, and the final thrust of the thruster and the pump group parameters of the dredging device are output.

[0060] Exemplarily, first, the optimal motion trajectory of the dredging robot is obtained. The trajectory can be generated by a path planning algorithm, including various positions and postures that the robot expects to reach. For example, the trajectory can be represented as a series of discrete posture points, each of which contains three-dimensional coordinates (x, y, z) and attitude angles (roll angle, pitch angle, yaw angle). For example, the first posture point is (1, 2, 3, 0, 0, 0), the second posture point is (1.5, 2.5, 3.2, 0.1, 0.2, 0.3), and so on.

[0061] Next, the optimal motion trajectory is input into the hierarchical model predictive controller. The controller is divided into a posture control layer and a working condition control layer. The posture control layer is responsible for controlling the position and posture of the robot, and the working condition control layer is responsible for controlling the working state of the dredging device.

[0062] In the posture control layer, the posture error is first calculated based on the difference between the current posture and the desired posture. For example, if the current position is (1.2, 2.3, 3.1) and the desired position is (1.5, 2.5, 3.2), the position error is (0.3, 0.2, 0.1). Then, according to the pre-set position error weight and attitude error weight, for example, the position weight is 1 and the attitude weight is 0.5, the position error and attitude error are weighted and combined into a comprehensive posture error. At the same time, the control quantity output by the thruster, such as the thrust of the thruster, is considered, and according to the pre-set control quantity weight, for example, the thrust weight is 0.1, an objective function including the posture error and the control quantity cost is constructed. The goal is to minimize this objective function.

[0063] The output of the attitude control layer is the desired thrust of the thruster. In the operating condition control layer, the operating condition error is first calculated based on the difference between the current operating condition and the desired operating condition. For example, if the current flow rate is 5m³ / h and the desired flow rate is 6m³ / h, the current pressure is 2MPa and the desired pressure is 2.5MPa, then the flow error is 1m³ / h and the pressure error is 0.5MPa. Then, according to the pre-set flow error weight and pressure error weight, for example, the flow weight is 1 and the pressure weight is 2, the flow error and the pressure error are weighted and combined into a comprehensive operating condition error. At the same time, the control quantity of the pump group parameters, such as the main pump flow, the auxiliary pump flow and the pressure, are considered, and according to the pre-set control quantity weights, for example, the main pump flow weight is 0.2, the auxiliary pump flow weight is 0.3, and the pressure weight is 0.5, an objective function including the operating condition error and the control quantity cost is constructed. The goal is to minimize this objective function.

[0064] The output of the working condition control layer is the main pump flow, auxiliary pump flow and pressure. In order to ensure the stability and safety of the controller, a series of constraints need to be set. These constraints include posture constraints, working condition constraints and coupling constraints. For example, the speed cannot exceed 1m / s, and the attitude angle change rate cannot exceed 10° / s. Working condition constraints include flow rate cannot exceed 10m³ / h, and pressure cannot exceed 3MPa. Coupling constraints include changes in attitude angle that affect flow rate, and depth that affects pressure.

[0065] The objective function of the posture control layer and the objective function of the working condition control layer are weighted to obtain a comprehensive objective function. For example, the weight of the objective function of the posture control layer is 0.8, and the weight of the objective function of the working condition control layer is 0.2. When all constraints are met, the sequential quadratic programming method is used to solve the comprehensive objective function to obtain the thruster control instructions and the dredging device control instructions.

[0066] According to the thruster control instructions and the dredging device control instructions, the actual posture error and working condition error are calculated. For example, the difference between the actual position and the expected position, the difference between the actual flow and the expected flow. These errors are compensated using the error compensation controller, and finally the thrust of the thruster and the pump group parameters of the dredging device are output.

[0067] By collaboratively controlling the thruster and the dredging device, the present invention can make the dredging robot move according to the optimal trajectory and maintain the optimal dredging working state, thereby improving the dredging efficiency; by optimizing the control strategy, the energy consumption of the thruster and the dredging device can be reduced, thereby reducing the cost of the dredging operation; the hierarchical control structure and the error compensation mechanism can improve the robustness of the control system, enabling it to adapt to complex underwater environments and changes in working conditions.

[0068] Figure 2 FIG. 1 is a schematic diagram of the structure of an intelligent control system of a dredging robot based on deep learning according to an embodiment of the present invention. Figure 2 As shown, the system comprises: The first unit is used to collect multi-source information of the working environment of the dredging robot through underwater cameras, sonar sensors, pressure sensors and gyroscopes, input the multi-source information into a convolutional neural network to extract spatial features, and use a recurrent neural network to extract temporal features, and fuse the extracted spatial features and temporal features to obtain an environmental feature vector, and establish a three-dimensional dynamic model of the working environment of the dredging robot based on the environmental feature vector; The second unit is used to establish a state action space including the state information, motion parameters and dredging operation parameters of the dredging robot, and construct the three-dimensional dynamic model as an environmental interaction module; construct a hierarchical reward function combining the operating efficiency and energy consumption indicators according to the state transfer output of the environmental interaction module, construct a dual neural network architecture based on a deep deterministic policy gradient algorithm, and train the dual neural network architecture based on the hierarchical reward function to obtain an intelligent control strategy model of the dredging robot; The third unit is used to output the motion trajectory of the dredging robot based on the intelligent control strategy model, and use the improved ant colony algorithm to calculate the motion trajectory to obtain the optimal motion trajectory, wherein the improved ant colony algorithm includes a dual-objective pheromone update strategy based on path length and energy consumption; the optimal motion trajectory is input into the hierarchical model predictive controller, and the propeller and dredging device of the dredging robot are collaboratively controlled through the hierarchical model predictive controller.

[0069] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0070] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0071] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. The intelligent control method of dredging robot based on deep learning is characterized by: include: The multi-source information of the dredging robot's working environment is collected through underwater cameras, sonar sensors, pressure sensors and gyroscopes, and the multi-source information is input into a convolutional neural network to extract spatial features. At the same time, a recurrent neural network is used to extract temporal features. The extracted spatial features and temporal features are fused to obtain an environmental feature vector, and a three-dimensional dynamic model of the dredging robot's working environment is established based on the environmental feature vector. Establish a state-action space including the state information, motion parameters and dredging operation parameters of the dredging robot, and construct the three-dimensional dynamic model as an environmental interaction module; construct a hierarchical reward function combining the operating efficiency and energy consumption indicators according to the state transfer output of the environmental interaction module, construct a dual neural network architecture based on the deep deterministic policy gradient algorithm, and train the dual neural network architecture based on the hierarchical reward function to obtain an intelligent control strategy model of the dredging robot; The motion trajectory of the dredging robot is output based on the intelligent control strategy model, and the improved ant colony algorithm is used to calculate the motion trajectory to obtain the optimal motion trajectory, wherein the improved ant colony algorithm includes a dual-objective pheromone update strategy based on path length and energy consumption; the optimal motion trajectory is input into a hierarchical model predictive controller, and the propeller and dredging device of the dredging robot are collaboratively controlled through the hierarchical model predictive controller.

2. The method according to claim 1, characterized in that The multi-source information of the working environment of the dredging robot is collected by underwater cameras, sonar sensors, pressure sensors and gyroscopes, the multi-source information is input into a convolutional neural network to extract spatial features, and the time series features are extracted by a recurrent neural network. The extracted spatial features and time series features are fused to obtain an environmental feature vector. The steps of establishing a three-dimensional dynamic model of the working environment of the dredging robot based on the environmental feature vector include: Perform retinal neural network restoration on the image data collected by the underwater camera, perform noise suppression and target enhancement on the sonar sensor data, and perform Kalman filtering denoising on the pressure sensor and gyroscope data to obtain pre-processed multi-source information data; The preprocessed multi-source information data are respectively input into the dual-stream feature extraction network, wherein the spatial feature extraction adopts an improved residual neural network, the improved residual neural network comprises residual blocks, each residual block is sequentially provided with a first convolutional layer for dimensionality reduction, a second convolutional layer for feature extraction and a third convolutional layer for dimensionality increase, to obtain a spatial feature vector; the temporal feature extraction adopts a bidirectional long short-term memory network, the bidirectional long short-term memory network comprises multiple layers of bidirectional long short-term memory layers, to obtain a temporal feature vector; the spatial feature vector and the temporal feature vector are adaptively weighted fused through an attention mechanism to obtain an environmental feature vector; A three-dimensional dynamic model of the working environment of the dredging robot is constructed based on the environmental feature vector, including using an improved OctoMap algorithm to construct a global environmental map, the improved OctoMap algorithm including an adaptive voxel resolution mechanism, a dynamic probability model based on Bayesian updating and a sensor fusion strategy based on distance weighting; a local area terrain model is established by using the environmental feature vector and Gaussian process regression based on anisotropic kernel function, and the kernel function parameters are updated by an online hyperparameter optimization algorithm; based on the local area terrain model, a point cloud registration algorithm based on normal distribution transformation is used to reconstruct the local environment; a multi-layer Kalman filter is designed to dynamically estimate the water velocity field, sediment distribution and obstacle motion state, and a confidence-based inter-layer information transmission mechanism and an environmental change detector are used to update the three-dimensional dynamic model in real time.

3. The method according to claim 1, characterized in that The steps of establishing a state action space including the state information, motion parameters and dredging operation parameters of the dredging robot and constructing the three-dimensional dynamic model as an environmental interaction module include: The state-action space includes a state space and an action space, wherein the state space includes a dredging robot posture vector, a velocity vector and a dredging system working parameter vector, the dredging robot posture vector includes a position parameter and a posture parameter of the dredging robot in a three-dimensional space, the velocity vector includes a linear velocity parameter and an angular velocity parameter, and the dredging system working parameter vector includes a main pump flow parameter, an auxiliary pump flow parameter, a main pump pressure parameter and an auxiliary pump pressure parameter; the action space includes a motion control amount and a dredging parameter adjustment amount, the motion control amount includes a position adjustment parameter and a posture adjustment parameter, and the dredging parameter adjustment amount includes a main pump flow adjustment parameter, an auxiliary pump flow adjustment parameter, a main pump pressure adjustment parameter and an auxiliary pump pressure adjustment parameter; The three-dimensional dynamic model is constructed as an environment interaction module, which includes an environment state mapping unit, an action execution unit and an environment feedback unit, wherein the environment state mapping unit maps the environment feature vector to the state space to obtain a state output, the action execution unit converts the action in the action space into an execution instruction based on the underwater motion characteristics of the dredging robot, and the environment feedback unit generates a state transfer output based on the state output and the execution instruction, and the state transfer output includes a state transfer probability and an immediate reward value.

4. The method according to claim 1, characterized in that: The steps of constructing a hierarchical reward function combining work efficiency and energy consumption indicators according to the state transition output of the environmental interaction module, constructing a dual neural network architecture based on a deep deterministic policy gradient algorithm, and training the dual neural network architecture based on the hierarchical reward function to obtain an intelligent control strategy model of the dredging robot include: Constructing a hierarchical reward function according to the state transfer output, wherein the hierarchical reward function includes an operation efficiency reward function, an energy consumption optimization reward function, and a safety constraint reward function; A dual neural network architecture is designed based on a deep deterministic policy gradient algorithm. The dual neural network architecture includes a policy network and an evaluation network. Both the policy network and the evaluation network adopt a multi-layer fully connected structure. The policy network adopts a deterministic policy gradient algorithm to update parameters, and the evaluation network adopts a temporal difference learning algorithm to update parameters. The policy network and the evaluation network respectively set target networks, and the target networks track parameter changes of the corresponding networks through parameter soft update. The dual neural network architecture is trained based on the hierarchical reward function, including collecting the state transition output sequence of the environment interaction module and storing it in the experience replay buffer; randomly sampling training samples from the experience replay buffer, mapping the state transition probability and the immediate reward value in the training samples into the reward value of the work efficiency dimension, the reward value of the energy consumption dimension and the reward value of the safety dimension through the hierarchical reward function, and calculating the total reward based on the adaptive weight method; using the total reward to train the policy network and the evaluation network; when the evaluation value of the total reward converges and is greater than a preset threshold, saving the parameters of the current policy network as the optimal parameters of the intelligent control strategy model.

5. The method according to claim 4, characterized in that The step of constructing a hierarchical reward function according to the state transfer output, wherein the hierarchical reward function includes an operation efficiency reward function, an energy consumption optimization reward function and a safety constraint reward function comprises: Construct a reward function mapping module for state transfer output, wherein the reward function mapping module adopts a multi-layer perceptron structure, wherein the input layer of the multi-layer perceptron structure receives the state transfer probability and the immediate reward value of the state transfer output, the hidden layer of the multi-layer perceptron structure uses a ReLU activation function, and the output layer of the multi-layer perceptron structure uses a hyperbolic tangent function to map the output value of the hidden layer into an efficiency dimension reward value, an energy consumption dimension reward value, and a safety dimension reward value; An operating efficiency reward function is constructed based on the efficiency dimension reward value, and the operating efficiency reward function includes a dredging area per unit time item and a dredging depth compliance rate item, wherein the dredging area per unit time item is obtained by double integration of the dredging velocity field and the dredging depth indication function, and the dredging depth compliance rate item is calculated by the ratio of the number of qualified sampling points to the total number of sampling points; an energy consumption optimization reward function is constructed based on the energy consumption dimension reward value, and the energy consumption optimization reward function includes a motion power consumption item and a pump group power consumption item, wherein the motion power consumption item is calculated based on the input power of the motors of each joint of the dredging robot, and the pump group power consumption item is calculated based on the motor input power of the main pump and the auxiliary pump; a safety constraint reward function is constructed based on the safety dimension reward value, and the safety constraint reward function includes an obstacle distance item and a posture stability item, wherein the obstacle distance item is obtained by calculating the minimum Euclidean distance between the position of the dredging robot and the positions of all obstacles in the environment map, and the posture stability item is obtained by calculating the distance from the centroid projection point to the supporting polygon; The total reward function is constructed by combining the work efficiency reward function, energy consumption optimization reward function and safety constraint reward function through an adaptive weight method. The adaptive weight method calculates the adaptive weight coefficients of the work efficiency reward function, energy consumption optimization reward function and safety constraint reward function based on the normalized gradient values ​​of the work efficiency target, energy consumption optimization target and safety constraint target. The adaptive weight coefficients are normalized by the softmax function.

6. The method according to claim 1, characterized in that Based on the motion trajectory of the dredging robot output by the intelligent control strategy model, the motion trajectory is calculated by using an improved ant colony algorithm to obtain the optimal motion trajectory. The improved ant colony algorithm includes the steps of a dual-target pheromone update strategy based on path length and energy consumption, including: Inputting the state transfer output of the environment interaction module into the intelligent control strategy model to obtain an action sequence, and mapping the action sequence according to the underwater motion characteristics of the dredging robot to obtain an initial motion trajectory of the dredging robot; The nodes of the initial motion trajectory are used as the search space of the improved ant colony algorithm to construct dual-target pheromones, wherein the dual-target pheromones include path length pheromones and energy consumption pheromones, wherein the path length pheromones are calculated based on the Euclidean distance between the trajectory nodes, and the energy consumption pheromones are calculated based on the motion power consumption of the dredging robot and the power consumption of the pump group; Constructing a path selection probability based on the dual-target pheromone and heuristic information, wherein the heuristic information includes obstacle distance information and posture stability information, and the path selection probability is calculated by combining a weighted product of the dual-target pheromone and the heuristic information; Generating candidate motion trajectories in the search space based on the path selection probability, calculating a normalized target value based on the path length and energy consumption of each candidate motion trajectory, calculating the weight coefficients of the path length pheromone and the energy consumption pheromone by using the normalized target value through an exponential function, and updating the dual-target pheromone according to the weight coefficients of the path length pheromone and the energy consumption pheromone; The candidate motion trajectories generated during the search process of the improved ant colony algorithm are evaluated using a total reward function, wherein the total reward function is calculated based on a hierarchical reward function, and the candidate motion trajectory with the maximum total reward value is selected as the optimal motion trajectory of the dredging robot.

7. The method according to claim 1, characterized in that The optimal motion trajectory is input into a hierarchical model prediction controller, and the step of cooperatively controlling the propeller and the dredging device of the dredging robot by the hierarchical model prediction controller includes: Discretize the optimal motion trajectory into a posture sequence including a position vector and an attitude angle vector, construct a posture prediction equation, and construct an objective function of the posture control layer based on the posture error weight matrix and the control amount weight matrix, wherein the posture error weight matrix is ​​used to balance the weights of the position error and the posture error, and the control amount weight matrix is ​​used to balance the weights of the thruster output; based on the output of the posture control layer, establish a pump group system model including the flow and pressure of the main and auxiliary pumps, and construct an objective function of the working condition control layer based on the working condition error weight matrix and the control amount weight matrix, wherein the working condition error weight matrix is ​​used to balance the weights of the flow error and the pressure error, and the control amount weight matrix is ​​used to balance the weights of the pump group parameters; Constructing hierarchical constraint conditions including posture constraints, working condition constraints and coupling constraints, wherein the posture constraints include speed constraints and attitude angle constraints, the working condition constraints include pump group flow constraints and pressure constraints, and the coupling constraints include attitude and flow coupling constraints and depth and pressure coupling constraints; The objective function of the posture control layer and the objective function of the operating condition control layer are weightedly combined to obtain a comprehensive objective function. Under the hierarchical constraints, the sequential quadratic programming method is used to solve the comprehensive objective function to obtain thruster control instructions and dredging device control instructions. The actual posture error and operating condition error are calculated based on the thruster control instructions and the dredging device control instructions, and an error compensation controller is constructed to compensate for the posture error and the operating condition error, and the final thrust of the thruster and the pump group parameters of the dredging device are output.

8. A dredging robot intelligent control system based on deep learning, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to collect multi-source information of the working environment of the dredging robot through underwater cameras, sonar sensors, pressure sensors and gyroscopes, input the multi-source information into a convolutional neural network to extract spatial features, and use a recurrent neural network to extract temporal features, and fuse the extracted spatial features and temporal features to obtain an environmental feature vector, and establish a three-dimensional dynamic model of the working environment of the dredging robot based on the environmental feature vector; The second unit is used to establish a state action space including the state information, motion parameters and dredging operation parameters of the dredging robot, and construct the three-dimensional dynamic model as an environmental interaction module; A hierarchical reward function combining work efficiency and energy consumption indicators is constructed according to the state transition output of the environmental interaction module, a dual neural network architecture is constructed based on a deep deterministic policy gradient algorithm, and the dual neural network architecture is trained based on the hierarchical reward function to obtain an intelligent control strategy model of the dredging robot; The third unit is used to output the motion trajectory of the dredging robot based on the intelligent control strategy model, and use the improved ant colony algorithm to calculate the motion trajectory to obtain the optimal motion trajectory, wherein the improved ant colony algorithm includes a dual-objective pheromone update strategy based on path length and energy consumption; the optimal motion trajectory is input into the hierarchical model predictive controller, and the propeller and dredging device of the dredging robot are collaboratively controlled through the hierarchical model predictive controller.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images