Underwater robot control method, device and electronic equipment

Abstract

The application provides an underwater robot control method and device and electronic equipment, and belongs to the technical field of intelligent control of underwater equipment. The method comprises the following steps: acquiring operation characteristic information of an underwater robot and environment perception information of a water area environment in which the underwater robot is located; determining an operation instruction speed, an intention confidence and a fatigue degree of an operation object based on the operation characteristic information; performing fluid disturbance compensation and obstacle avoidance repulsion processing on the environment perception information to obtain a robot agent speed; determining a control weight factor based on the intention confidence, the fatigue degree, and fluid disturbance intensity and obstacle risk in the environment perception information; performing fusion processing on the operation instruction speed, the robot agent speed and the control weight factor to obtain an end desired speed of the underwater robot; and performing speed control on the underwater robot based on the end desired speed. The above scheme can effectively improve the control accuracy and safety of the underwater robot in a complex working environment.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology for underwater equipment, specifically to underwater robot control methods, devices, and electronic equipment. Background Technology

[0002] With the increasing demand for deep-sea resource development, seabed engineering maintenance, cage inspection, ecological monitoring, and underwater emergency response, underwater robots are gradually becoming an important component of marine engineering equipment. When underwater robots perform operations in scenarios such as deep-sea resource development, seabed inspection, and underwater facility maintenance, they are often controlled remotely. Current technologies mainly rely on graphical interfaces or joysticks, where the operator directly issues single operational commands to the underwater robot based on video feedback information to control its movements.

[0003] However, in practical applications, operators are prone to fatigue and blurred intentions during long hours of work, while underwater environments present fluid disturbances and obstacles. Existing technologies rely solely on direct control with a single command, failing to detect operator fatigue and the level of confidence in their intentions, and also unable to compensate for or avoid risks from fluid disturbances and obstacles. This results in low control accuracy and a high risk of collisions in complex hydrological conditions.

[0004] Therefore, there is an urgent need for a control method that can comprehensively assess operator status and environmental risks and dynamically allocate control weights to improve control accuracy and safety in complex environments. Summary of the Invention

[0005] To address the aforementioned problems, this application provides an underwater robot control method to improve the control accuracy and safety of underwater robots in complex working environments. The technical solution is as follows: In a first aspect, the present invention provides an underwater robot control method, comprising: Acquire operational characteristic information for the underwater robot and environmental perception information of the water environment in which the underwater robot is located. The operational characteristic information is a set of multimodal operational information issued by the underwater robot's operating object to the underwater robot. Based on operational characteristic information, determine the speed of operational instructions, confidence level of intent, and fatigue level of the operation object; The environmental perception information is processed including fluid disturbance compensation and obstacle avoidance to obtain the robot agent speed, which is the motion guidance speed that guides the underwater robot to approach the target and avoid obstacles. Based on intent confidence, fatigue, and fluid disturbance intensity and obstacle risk in environmental perception information, control weighting factors are determined. The operating command speed, robot agent speed, and control weight factor are fused together to obtain the desired end-effector speed of the underwater robot, and the underwater robot speed is controlled based on the desired end-effector speed.

[0006] Combining the first aspect and the above implementation methods, in some possible implementation methods, based on operational characteristic information, the speed of the operation command, the confidence level of the intent, and the fatigue level of the operation object are determined, including: The operational feature information is subjected to anti-interference filtering and time alignment processing to obtain alignment features; Based on the pose and kinematic information in the operation feature information, determine the speed of the operation command for the operation object; The alignment features are input into the intent recognition model for processing to determine the intent confidence of the target object; The root mean square value and median frequency of the spectrum are calculated based on the electromyographic signal characteristics in the operation feature information, and the fatigue level of the operation object is determined by combining the continuous operation time.

[0007] Combining the first aspect and the above implementation methods, in some possible implementation methods, the environmental perception information is processed including fluid disturbance compensation and obstacle avoidance to obtain the robot agent speed, including: Based on the target object in the environmental perception information, the target pose of the underwater robot is determined, and the target attraction term is determined based on the target pose and the current pose of the end effector. Analyze the obstacle location information in the environmental perception information to determine the obstacle rejection terms; Based on the water flow velocity and end effector velocity in the environmental sensing information, determine the fluid disturbance compensation term; The robot agent speed is obtained by fusing and limiting the target attraction term, obstacle repulsion term, and fluid disturbance compensation term.

[0008] Combining the first aspect and the above implementation methods, in some possible implementation methods, control weighting factors are determined based on intent confidence, fatigue level, and the intensity of fluid disturbance and obstacle risk in environmental perception information, including: By comparing the speed of the operation command with the speed of the robot agent, the master-slave motion consistency index is obtained. The fluid disturbance intensity, obstacle risk, communication delay, fatigue, intention confidence and master-slave motion consistency index are weighted and calculated to obtain the weighted result. The communication delay is the round-trip time difference between the operation equipment of the object and the underwater robot for command transmission and / or status feedback. The weighted results are subjected to interval limiting to obtain the control weight factor.

[0009] Combining the first aspect and the above implementation methods, in some possible implementation methods, the operation command speed, robot agent speed, and control weight factor are fused to obtain the desired end-effector speed of the underwater robot, including: The control weight factor is used as the fusion weight for the robot agent speed, and the difference between the preset total weight value and the control weight factor is used as the fusion weight for the operation command speed. The desired end-effector speed of the underwater robot is obtained by weighted summation of the robot agent speed with its corresponding fusion weight and the operation command speed with its corresponding fusion weight.

[0010] Combining the first aspect and the above implementation methods, in some possible implementation methods, after controlling the speed of the underwater robot based on the desired end-effector velocity, the following additional steps are included: The joint velocity command is obtained by performing inverse kinematics solution based on the desired end-effector velocity and the Jacobian matrix of the robotic arm. The joint velocity command is combined with the zero-space velocity to generate joint control command. The underwater robot's robotic arm is controlled and adjusted based on the joint control command. The zero-space velocity is the speed that meets the joint limit and singularity avoidance requirements.

[0011] Combining the first aspect and the above implementation methods, in some possible implementation methods, after speed control of the underwater robot based on the desired end-effector velocity, the following are included: Obtain the state feature sequence of the operation objects within the history window; The state feature sequence is processed based on the time delay compensation prediction model to obtain the predicted motion trajectory of the operation object; The predicted motion trajectory of the manipulated object is used as a feedforward term, and combined with environmental perception information to generate compensation control commands. The desired speed at the terminal is compensated for time delay based on the compensation control command.

[0012] In combination with the first aspect and the above implementation methods, some possible implementation methods also include: Establish a skill set for underwater robots. Based on fluid disturbance compensation, the motion trajectories in the operation skill library are enhanced and processed by strategy learning to obtain underwater operation skill strategies. Autonomous operation control of underwater robots based on underwater operation skill strategies.

[0013] Secondly, the present invention also provides an underwater robot control device, comprising: The data acquisition unit is used to acquire operational feature information of the underwater robot and environmental perception information of the water environment in which the underwater robot is located. The operational feature information is a set of multimodal operational information issued by the underwater robot's operating object to the underwater robot. The data processing unit is used to determine the operation command speed, intent confidence, and fatigue level of the operation object based on operation feature information. The data generation unit is used to process environmental perception information, including fluid disturbance compensation and obstacle avoidance, to obtain the robot agent speed. The robot agent speed is the motion guidance speed that guides the underwater robot to approach the target and avoid obstacles. The weighting factor acquisition unit is used to determine the control weighting factors based on the intent confidence, fatigue level, and fluid disturbance intensity and obstacle risk in the environmental perception information. The speed control unit is used to fuse the operation command speed, robot agent speed, and control weight factor to obtain the desired end-effector speed of the underwater robot, and to control the speed of the underwater robot based on the desired end-effector speed.

[0014] Thirdly, the present invention also provides an electronic device, including a data acquisition unit, a memory, and a processor, wherein, The data acquisition device is used to acquire operational characteristic information of the underwater robot and environmental perception information of the water environment in which the underwater robot is located. Memory, used to store executable program code; The processor is used to determine the operation command speed, intent confidence, and fatigue level of the target object based on operation feature information; to obtain the robot agent speed by performing environmental perception information processing including fluid disturbance compensation and obstacle avoidance; to determine the control weight factor based on intent confidence, fatigue level, and fluid disturbance intensity and obstacle risk in the environmental perception information; to fuse the operation command speed, robot agent speed, and control weight factor to obtain the desired end-effector speed of the underwater robot; and to perform speed control on the underwater robot based on the desired end-effector speed.

[0015] The beneficial effects of this invention are as follows: This application obtains operational characteristic information and environmental perception information for underwater robots, and determines the speed of operational commands, intent confidence, and fatigue level based on the operational characteristic information, thus accurately overcoming the inability to perceive these factors. By performing fluid disturbance compensation and obstacle avoidance processing on the environmental perception information, the robot's proxy speed is obtained, thereby achieving compensation and avoidance of fluid disturbance and obstacle risks. By determining control weight factors based on intent confidence, fatigue level, and the intensity of fluid disturbance and obstacle risks in the environmental perception information, the operational command speed, robot proxy speed, and control weight factors are fused to obtain the desired end-effector speed, and the underwater robot's speed is controlled based on this desired end-effector speed. By determining and fusing control weight factors, the defects of single command control are overcome, effectively improving the control accuracy and safety of underwater robots. Attached Figure Description To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating the underwater robot control method provided by the present invention. Figure 2 Provided by the present invention Figure 1 Schematic diagram of the execution flow scenario in the embodiment; Figure 3 For the present invention Figure 1 A schematic diagram of an embodiment of S102; Figure 4 For the present invention Figure 1 A schematic diagram of an embodiment of S103; Figure 5 For the present invention Figure 1 A schematic diagram of an embodiment of S104; Figure 6 For the present invention Figure 1 A schematic diagram of an embodiment of S105; Figure 7 A schematic diagram of the execution process for controlling the underwater robot arm provided by the present invention; Figure 8 For the present invention Figure 1 A schematic diagram of an embodiment of S105; Figure 9 This is a schematic diagram of the process for autonomous operation control of an underwater robot provided by the present invention; Figure 10 A schematic diagram of the underwater robot control device provided by the present invention; Figure 11 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0018] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0019] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0020] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0021] This application provides an underwater robot control method, wherein the execution subject of the underwater robot control method is an underwater robot control method device or an electronic device equipped with an underwater robot control method device. A detailed description follows; it should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0022] It should be noted that the overall architecture of this invention consists of a master end, a slave end, and a shared control decision core connecting the master and slave ends. The master end, located on the object being operated on, includes a master-machine interface module. This module collects multimodal operational characteristic information of the object being operated on through force feedback devices, electromyography sensors, etc., and receives force feedback from the slave end to form a closed-loop perception of the object. The slave end, located on the underwater robot side, includes an environmental perception module and an inverse kinematics and execution control module. The environmental perception module is responsible for collecting environmental perception information such as target pose, obstacle information, water flow information, and end-effector contact force. The inverse kinematics and execution control module is responsible for converting the desired end-effector velocity issued by the decision core into joint control commands and driving the robotic arm to execute them.

[0023] The shared control decision core connects the master and slave ends and includes a multimodal intent recognition module, a robot agent command generation module, a weight factor calculation module, a speed fusion module, and a time delay compensation module. This decision core receives operation feature information output by the master end human-machine interaction module and environmental perception information output by the slave end environmental perception module. The multimodal intent recognition module analyzes the intent confidence and fatigue level of the operation object. The robot agent command generation module combines the environmental perception information to generate a robot agent speed with obstacle avoidance and current resistance capabilities. The weight factor calculation module and the speed fusion module achieve continuous dynamic fusion of operation command speed and robot agent speed based on human-machine status and environmental risks. The time delay compensation module compensates for underwater acoustic communication delay. Finally, the fused end-effector desired speed is sent to the slave end inverse kinematics and execution control module. At the same time, the slave end underwater force feedback module transmits the end-effector contact force and environmental impedance back to the master end human-machine interaction module.

[0024] Please see Figure 1 , Figure 1 This is a flowchart illustrating the underwater robot control method provided in an embodiment of this application. Figure 1 As shown, the method in this application embodiment may include the following steps S101-S105: S101, acquire operational characteristic information of the underwater robot and environmental perception information of the water environment in which the underwater robot is located.

[0025] In this embodiment, an underwater robot refers to a remotely operated device equipped with an underwater robotic arm, such as a remotely operated vehicle (ROV) or an autonomous underwater vehicle (AUV). Operational characteristic information refers to the set of multimodal operational information issued by the host device to the underwater robot, including force feedback device pose, velocity, electromyographic signals, operational forces, or interactive forces. The aquatic environment refers to the marine or pool environment in which the underwater robot performs grasping, collection, detection, and maintenance tasks, where there are flow field disturbances and visual limitations. Environmental perception information refers to the set of information collected from the end device, including target pose, obstacle location, end-effector contact force, robot state, and water flow velocity. The operator refers to the personnel located at the host control console who send control commands to the underwater robot through force feedback devices and other interactive devices.

[0026] In one feasible implementation, the input information of the object being operated on is collected through a force feedback device, and the output end pose and master end velocity are output. At the same time, the electromyographic signal of the forearm and the kinematic signal of the arm of the object being operated on are collected. Based on the visual sensor, force sensor and acoustic sensor of the slave end environment perception module carried by the underwater robot, the pose of the target object, the environmental point cloud, the current pose of the slave end, the water flow velocity and the end contact force are collected, thereby generating environmental perception information.

[0027] For example, the specific form of the multimodal fusion feature vector in step S101 above is as follows:

[0028] Among them, formula parameters represent Multimodal fusion feature vector at time step and These represent the pose and velocity of the master force feedback device, respectively. Represents electromyographic signal characteristics, Represents the main terminal's operational or interactive capabilities. This represents the current pose of the end effector of the marine robot. Represents end contact force. This represents water flow velocity or equivalent fluid disturbance information.

[0029] S102, based on the operation feature information, determine the operation command speed, intention confidence and fatigue level of the operation object.

[0030] In this embodiment, the operation command speed refers to the velocity vector of the desired underwater robot end effector motion input by the object through the manipulator force feedback device. Intent confidence refers to the reliability assessment value of the recognition result of the object's current operational intent. Fatigue level refers to a quantitative indicator of the physiological and psychological fatigue level experienced by the object during continuous teleoperation.

[0031] In one feasible implementation, the operational feature information is subjected to anti-interference filtering and normalization to remove impulse noise and high-frequency interference, and signals of different dimensions are mapped to a unified feature scale. The speed of the operational command is calculated based on the processed pose and kinematic information. The spatiotemporally aligned features are input into a dynamic Bayesian network or a long short-term memory network, and the operational intent category and its probability are output as the intent confidence level. Simultaneously, the root mean square value and median frequency of the electromyographic signal are extracted, and the fatigue level of the operated object is calculated in conjunction with the continuous operation time.

[0032] S103 performs environmental perception information processing, including fluid disturbance compensation and obstacle avoidance, to obtain the robot agent speed.

[0033] In this embodiment, fluid disturbance compensation refers to the process of estimating water resistance based on the water flow velocity and the end effector velocity, and then superimposing a reverse compensation force in the motion command to counteract the fluid disturbance. Obstacle avoidance and repulsion processing refers to the process of constructing a repulsive potential field based on the distance between the robotic arm and the obstacle, and generating a velocity in the direction away from the obstacle. Robot agent speed refers to the motion guidance speed generated by the robot autonomous planning module to guide the underwater robot to approach the target and avoid obstacles.

[0034] In one feasible implementation, a target attraction potential field is constructed based on the target pose and the current pose of the end effector, and the target attraction term is obtained by differentiation; an obstacle repulsion potential field is constructed based on the distances from the robotic arm links and the end effector to the obstacle, and the obstacle repulsion term is obtained by differentiation; a water flow disturbance term is estimated based on the water flow velocity, the end effector velocity, the water density, and the drag coefficient. The target attraction term, the obstacle repulsion term, and the water flow disturbance compensation term are superimposed and limited to generate the robot proxy velocity. When the input from the manipulated object is insufficient or there is a risk of collision, this robot proxy velocity can guide the robotic arm to move in a safe direction.

[0035] S104. Based on the confidence level of intent, fatigue level, and the intensity of fluid disturbance and obstacle risk in the environmental perception information, determine the control weight factor.

[0036] In this embodiment, fluid disturbance intensity refers to a quantitative assessment value of the water flow velocity or water resistance amplitude. Obstacle risk refers to the degree of collision danger when the robotic arm or end effector approaches a safety threshold with respect to an obstacle. Control weighting factor refers to a continuously changing weighting parameter used to allocate the proportion of manual teleoperation commands and robot autonomous agent commands in the final fusion speed.

[0037] In one feasible implementation, a weighting factor calculation module calculates the master-slave motion consistency index between the command speed of the operating object and the robot agent speed, and calculates the fluid impedance matching degree of the human-machine-water environment. A weighted sum is then calculated, taking into account obstacle risk, fluid disturbance intensity, communication latency, fatigue, intent confidence, master-slave motion consistency index, and fluid impedance matching degree, and processed by an interval limiting function to obtain the control weighting factor. When obstacle risk, fluid disturbance intensity, communication latency, or fatigue increases, the control weighting factor increases, increasing the proportion of robot agent control; when intent confidence is high and master-slave motion consistency is high, the control weighting factor decreases, increasing the proportion of manual command control.

[0038] S105, the operation command speed, robot agent speed and control weight factor are fused to obtain the desired end-effector speed of the underwater robot, and the underwater robot speed is controlled based on the desired end-effector speed.

[0039] In this embodiment, the expected end-effector velocity refers to the expected composite velocity that is finally sent to the underwater robot end effector after integrating the manual teleoperation command and the robot's autonomous agent command.

[0040] In one feasible implementation, the speed fusion module uses a control weight factor as the weight of the robot agent speed and the difference between the preset total weight value and the control weight factor as the weight of the operation command speed. The two are then linearly weighted and summed to obtain the desired end-effector speed.

[0041] The inverse kinematics module maps the desired end-effector velocity to joint velocity commands based on the generalized inverse of the robotic arm's Jacobian matrix, and then superimposes zero-space velocities for joint limiting and singularity avoidance, generating the final joint control commands for the actuator. Through continuous weight allocation and velocity fusion, robotic arm jitter caused by abrupt changes in manual / autonomous modes is avoided.

[0042] Please refer to the following: Figure 2 , Figure 2 This invention provides Figure 1 An example of the execution flow scenario is shown in the diagram. Figure 2 As shown, the system acquires the intention confidence and fatigue level of the target object, perceives obstacle risks and water flow disturbances in the underwater environment, and combines these with communication latency to obtain a control weighting factor. The system acquires the speed of the operation commands issued by the target object and the speed of the robot agent autonomously planned based on environmental obstacle avoidance and current resistance constraints. Based on the control weighting factor, the speed of the manual operation commands and the speed of the autonomous robot agent are continuously weighted and fused to output a smooth and safe desired end-effector speed.

[0043] In summary, this application overcomes the inability to perceive certain aspects by acquiring operational characteristic information and environmental perception information for the underwater robot, and determining the speed of operational commands, intent confidence, and fatigue level based on the operational characteristic information. By processing the environmental perception information, including fluid disturbance compensation and obstacle avoidance, the robot's proxy speed is obtained, thereby compensating for and avoiding fluid disturbances and obstacle risks. Control weight factors are determined based on intent confidence, fatigue level, and the intensity of fluid disturbances and obstacle risks in the environmental perception information. The operational command speed, robot proxy speed, and control weight factors are then fused to obtain the desired end-effector speed, which is used to control the underwater robot's speed. By determining and fusing control weight factors, the limitations of single-command control are overcome, effectively improving the control accuracy and safety of the underwater robot.

[0044] Please see Figure 3 , Figure 3 This is a flowchart illustrating the underwater robot control method provided in an embodiment of this application. Figure 3As shown, the method in this application embodiment may include the following steps S301-S304: S301, perform anti-interference filtering and time alignment processing on the operation feature information to obtain alignment features.

[0045] In this embodiment, anti-interference filtering refers to the signal processing procedure used to eliminate impulse noise and high-frequency noise contained in the sensor-acquired signals and to perform state estimation. Time alignment processing refers to the interpolation or buffering process that aligns asynchronous multi-source signals to a unified time reference based on the differences in sampling frequency and communication delay of each sensor. Alignment features refer to the multimodal feature vector sequence with a unified timestamp and suppressed noise after anti-interference filtering and time alignment.

[0046] In one feasible implementation, electromyography, force sensing, visual sensing, and hydrodynamic sensing information are sampled simultaneously, and normalization processing is used to map different dimensions to a unified feature scale, such as using max-min normalization or Z-score normalization. Disturbance rejection filtering employs a serial cascade or parallel branching approach. In serial cascade, median filtering first removes impulse noise, wavelet transform suppresses high-frequency noise, and then time-varying Kalman filtering or particle filtering is used for state estimation. In parallel branching, each filter processes its corresponding mode and is weighted and fused based on sensor signal-to-noise ratio, sampling delay, and confidence level. Time alignment processing uses the timestamp of the master force feedback device's pose as a reference, aligning other sensor signals through linear interpolation or discarding buffers to finally obtain the alignment features.

[0047] S302, determine the speed of the operation command for the object being operated based on the pose information and kinematic information in the operation feature information.

[0048] In this embodiment, pose information refers to the position and attitude data of the end effector of the master force feedback device in the base coordinate system. Kinematic information refers to the linear velocity and angular velocity data of the end effector of the master force feedback device.

[0049] In one feasible implementation, the pose information output by the master force feedback device is read, and the master velocity is calculated using a first-order difference or low-pass filter differential algorithm, which serves as the operation command velocity. Alternatively, the master velocity can be obtained by directly reading the kinematic information driven by the force feedback device, which is also used as the operation command velocity. This operation command velocity characterizes the instantaneous direction and speed of the underwater robot's end effector motion that the operating object currently desires.

[0050] S303, The alignment feature input is processed by the intent recognition model to determine the intent confidence of the operation object.

[0051] In the embodiments of this application, the intent recognition model refers to a model built based on machine learning algorithms, used to classify the multimodal alignment features of the input, identify the current operation intent category of the operation object and output the corresponding probability, such as a dynamic Bayesian network or a long short-term memory network.

[0052] In one feasible implementation, alignment features are input into the intent recognition model, and the model outputs the probability distribution of the current operation intent belonging to a preset category.

[0053] For example, the specific form of the intent confidence formula in step S303 above is as follows:

[0054] Among them, formula parameters represent Confidence of the operation object's intent at any given moment. represent The operational intent category at any given time, wherein the operational intent category includes at least one of approach, grasp, hold, evacuate, detect, or maintain. Represents from time 1 to The sequence of multimodal fused feature vectors at time step.

[0055] S304. Calculate the root mean square value and median frequency of the spectrum based on the electromyographic signal characteristics in the operation feature information, and determine the fatigue level of the operation object by combining the continuous operation time.

[0056] In this embodiment, electromyographic signal characteristics refer to the signal sequence reflecting the electrical activity of muscle contraction collected from the surface of the forearm of the object being manipulated. The root mean square (RMS) value is the root mean square statistic of the electromyographic signal amplitude over a time window, reflecting the intensity of muscle contraction. The median frequency of the spectrum refers to the frequency value in the power spectrum of the electromyographic signal that divides the total power into two equal halves, reflecting the decrease in the discharge frequency of the motor units when the muscle is fatigued. The duration of continuous operation refers to the cumulative length of time the object continuously performs a teleoperation task.

[0057] In one feasible implementation, electromyography (EMG) signal features are extracted, and the root mean square (RMS) value and median frequency of the EMG signal after fast Fourier transform within a sliding window are calculated. These values ​​are then combined with the recorded duration of operation to calculate fatigue level using a weighted average. When the RMS value exceeds a calibration reference value or the median frequency drops below a reference value, the fatigue level of the operator is determined to be elevated.

[0058] For example, the fatigue formula in step S304 above is in the following specific form:

[0059] Among them, formula parameters This represents the fatigue level of the object being operated on. Represents the root mean square value of electromyography. Represents the median frequency of electromyography. and Represents the calibration reference value. Represents continuous operation time. Represents a preset time scale. , , This represents the weighting coefficient.

[0060] In summary, this application achieves accurate perception of the intent and physiological state of the target object by performing anti-interference filtering and time alignment on multimodal information, outputting intent confidence using an intent recognition model, and quantifying fatigue by combining electromyographic signal characteristics and continuous operation time. This provides a reliable state basis for subsequent shared control weight allocation.

[0061] Please see Figure 4 , Figure 4 This is a flowchart illustrating the underwater robot control method provided in an embodiment of this application. Figure 4 As shown, the method in this application embodiment may include the following steps S401-S404: S401, based on the target object in the environmental perception information, determine the target pose of the underwater robot, and determine the target attraction term based on the target pose and the current pose of the end effector.

[0062] In this embodiment, the target object refers to the underwater object that the underwater robot is currently required to approach, grasp, or perform operations on. The target pose refers to the position and orientation of the target object in the slave-endpoint coordinate system. The current pose of the end effector refers to the current position and orientation of the underwater robot's end effector in the slave-endpoint coordinate system. The target attraction term refers to the attraction vector in the artificial potential field that guides the end effector toward the target pose.

[0063] In one feasible implementation, the target pose of the target object is obtained by visual recognition or point cloud matching through the environmental perception module, the current pose of the end effector is read, the target attraction potential field is constructed, and the negative gradient of the potential field with respect to the current pose of the end effector is obtained to obtain the target attraction term.

[0064] For example, the specific form of the formulas for the target attraction potential field and the target attraction term in the above steps is as follows:

[0065] Among them, formula parameters Represents the target attraction field, Represents the target attraction. Represents the target pose. Represents the current pose of the end effector. This represents the attraction gain.

[0066] S402, Analyze the obstacle location information in the environmental perception information to determine the obstacle rejection term.

[0067] In this embodiment, obstacle location information refers to the position and contour information of static or dynamic obstacles existing in the underwater working environment in the end-base coordinate system. The obstacle repulsion term refers to the repulsive force vector generated in the artificial potential field near the obstacle, guiding the end effector or robotic arm link away from the obstacle.

[0068] In one feasible implementation, obstacle location information is obtained based on environmental point cloud or visual recognition, and the shortest distance from the end effector or robotic arm link to the obstacle is calculated. When the distance is less than a preset safety distance, a repulsive potential field is constructed based on the reciprocal of the distance and the gradient is calculated to obtain the obstacle repulsion term. When the distance is greater than or equal to the safety distance, the repulsion term is zero.

[0069] For example, the specific form of the obstacle rejection term formula in step S402 above is as follows:

[0070] Among them, formula parameters Represents the obstacle rejection term. This represents the distance from the end effector or robotic arm link to the obstacle. Represents a safe distance. This represents the unit direction vector from the obstacle to the robotic arm. This represents repulsion gain.

[0071] S403, determine the fluid disturbance compensation term based on the water flow velocity and end effector velocity in the environmental perception information.

[0072] In this embodiment, the water flow velocity refers to the instantaneous velocity vector of the flow field in the underwater robot's operating environment, which can be measured by a Doppler velocimeter or an acoustic Doppler current profiler. The end effector velocity refers to the absolute velocity vector of the underwater robot's end effector moving in the flow field. The fluid disturbance compensation term refers to the active compensation force term used to counteract the water resistance effect of the water flow on the end effector or the clamped target.

[0073] In one feasible implementation, the relative incoming flow velocity is calculated based on the difference between the water flow velocity and the end effector velocity. Combined with the water density, drag coefficient, and equivalent frontal area, a water flow disturbance estimate is calculated as a fluid disturbance compensation term. The equivalent frontal area is determined based on the end effector shape, gripper opening and closing state, and relative incoming flow direction. That is, the projected profile of the end effector on the normal plane of the relative incoming flow direction is obtained first, and then the projected area is calculated. For regular grippers, a bounding box approximation of length, width, and attitude angle is used. For complex end effectors, a three-dimensional model offline projection table is used and interpolated according to attitude.

[0074] For example, the specific form of the fluid disturbance compensation term formula in step S403 above is as follows:

[0075] Among them, formula parameters This represents the water flow disturbance estimation term, also known as the fluid disturbance compensation term. Represents water density. Represents the drag coefficient. Represents the equivalent frontal area. Represents water flow velocity. This represents the speed of the end effector.

[0076] S404 fuses and limits the target attraction term, obstacle repulsion term, and fluid disturbance compensation term to obtain the robot agent speed.

[0077] In the embodiments of this application, the fusion limiting processing refers to the linear superposition of the target attraction term, obstacle repulsion term and fluid disturbance compensation term after multiplying each by the corresponding gain, and limiting the amplitude of the superimposed velocity vector to meet the mechanical speed limiting requirements.

[0078] In one feasible implementation, the target approach gain is multiplied by the target attraction term, the obstacle avoidance gain is multiplied by the obstacle repulsion term, and the fluid compensation gain is multiplied by the fluid disturbance compensation term. The resultant velocity is then saturated and truncated using a velocity limiting function to obtain the robot agent velocity.

[0079] For example, the specific form of the robot agent speed formula in step S404 above is as follows:

[0080] Among them, formula parameters Represents the speed of the robot agent. Represents the velocity limiting function. , , These represent target approach, obstacle avoidance, and fluid compensation gains, respectively.

[0081] In summary, this application combines artificial potential field with fluid disturbance compensation to generate a robot agent speed with autonomous target approach, obstacle avoidance, and resistance to fluid disturbance, ensuring the robot's safety and operational continuity when the input from the operating object is insufficient or the environment is hazardous.

[0082] Please see Figure 5 , Figure 5 This is a flowchart illustrating the underwater robot control method provided in an embodiment of this application. Figure 5As shown, the method in this application embodiment may include the following steps S501-S504: S501 compares the speed of the operation command with the speed of the robot agent to obtain the master-slave motion consistency index.

[0083] In this embodiment of the application, the master-slave motion consistency index refers to a normalized evaluation index that measures the similarity in direction and amplitude between the command speed of the master-end operation object and the speed of the slave robot agent.

[0084] In one feasible implementation, the cosine of the angle between the speed of the command from the manipulated object and the speed of the robot agent is calculated to characterize directional consistency, and the negative exponent of the difference between the two is calculated to characterize amplitude consistency. The product of the two is used as the master-slave motion consistency index. The larger the master-slave motion consistency index, the more consistent the master's intention and the slave's agent motion are; when the index is small, it indicates that there is a conflict between the two, and the robot agent control weight needs to be increased to suppress conflicting commands.

[0085] For example, the specific form of the master-slave motion consistency index formula in step S501 above is as follows:

[0086] Among them, formula parameters This represents the consistency index of master-slave motion. Represents the speed of instructions for the operand. Speed ​​of robot agents The included angle, This represents the normalized scale of the velocity difference.

[0087] For example, the specific form of the master-slave motion consistency index formula in step S501 above is as follows: S502 calculates the weighted result by weighting the fluid disturbance intensity, obstacle risk, communication delay, fatigue, intention confidence and master-slave motion consistency index.

[0088] In this embodiment of the application, communication latency refers to the round-trip time difference between the transmission of the master instruction to the slave and the feedback of the slave status to the master.

[0089] In one feasible implementation, the fluid impedance matching degree between the equivalent impedance of the operating object at the master end, the equivalent impedance of the water environment, and the equivalent impedance of the underwater robot at the slave end is calculated. Obstacle risk, fluid disturbance intensity, communication delay, and fatigue are used as positive factors to increase the robot's autonomous weight, while intention confidence, master-slave motion consistency index, and fluid impedance matching degree are used as negative factors to increase the weight of manual control. A linear weighted summation is performed to obtain the weighted result.

[0090] For example, the specific form of the formula involved in the fluid impedance matching degree and weighted calculation in step S502 above is as follows:

[0091]

[0092] Among them, formula parameters Represents fluid impedance matching degree. Represents the equivalent impedance of the main-side operating object. Represents the equivalent impedance of the aquatic environment. This represents the equivalent impedance of the marine robot at the end. This represents the weighted result (i.e., the weighting factor before the limit is applied). Represents the basic weight. Represents obstacle risk, Represents the intensity of fluid disturbance. Represents communication latency. This represents the fatigue level of the object being operated on. Represents the confidence level of the intention. This represents the consistency index of master-slave motion. to This represents the weighting coefficient.

[0093] S503 performs interval limiting on the weighted results to obtain the control weight factor.

[0094] In this embodiment of the application, the interval limiting process refers to the truncation process that constrains the weighted result within the closed interval [0,1] to ensure that the control weight factor has a probabilistic meaning.

[0095] In one feasible implementation, the weighted result in step S502 is truncated by an interval limiting function. If the weighted result is less than 0, 0 is output; if it is greater than 1, 1 is output; otherwise, the original value is output, thus obtaining the control weight factor.

[0096] For example, the specific form of the control weight factor formula in step S503 above is as follows:

[0097] Among them, formula parameters Represents the control weighting factor. Represents an interval-limited amplitude function. This represents the weighted result in step S502. The larger the control weight factor, the more it is biased towards the robot agent's speed; the smaller the control weight factor, the more it is biased towards the speed of the operation object's instructions.

[0098] In summary, this application achieves a continuous and smooth allocation of control weights between manual instructions and autonomous robot commands by comprehensively considering fluid disturbance, obstacle risk, time delay, fatigue, intention confidence, and master-slave consistency index for weighting range, thus avoiding control abrupt changes and mechanical jitter caused by traditional mode switching.

[0099] Please see Figure 6 , Figure 6 This is a flowchart illustrating the underwater robot control method provided in an embodiment of this application. Figure 6 As shown, the method in this application embodiment may include the following steps S601-S604: S601 uses the control weight factor as the fusion weight of the robot agent speed and the difference between the preset total weight value and the control weight factor as the fusion weight of the operation command speed.

[0100] In this embodiment, the fusion weight refers to the weighting coefficient corresponding to each velocity component participating in the velocity fusion calculation. The preset total weight value is a constant of the preset sum of fusion weights, usually set to 1, to ensure that the velocity dimension after fusion is consistent with the original velocity.

[0101] In one feasible implementation, the control weight factor is directly used as the fusion weight of the robot agent speed, and the difference between 1 and the control weight factor is used as the fusion weight of the operation command speed. Thus, when the control weight factor tends to 1, the fusion weight of the robot agent speed tends to 1, and the fusion weight of the operation command speed tends to 0; conversely, when the control weight factor tends to 0, the fusion weight is the opposite.

[0102] S602, the robot agent speed and its corresponding fusion weight, and the operation command speed and its corresponding fusion weight are weighted and summed to obtain the expected end-effector speed of the underwater robot.

[0103] In the embodiments of this application, the weighted summation process refers to a linear fusion operation process in which each velocity component is multiplied by its corresponding fusion weight and then summed.

[0104] In one feasible implementation, the robot agent speed is multiplied by a control weight factor, the operation command speed is multiplied by the difference between 1 and the control weight factor, and the products of the two are added together to obtain the desired end-effector speed of the underwater robot.

[0105] For example, the specific form of the formula for the desired terminal velocity in step S602 above is as follows:

[0106] Among them, formula parameters Represents the expected velocity of the marine robot's end effector. Represents the control weighting factor. Represents the speed of the robot agent. This represents the speed of the instructions used to manipulate the object.

[0107] S603 performs inverse kinematics calculations based on the desired end-effector velocity and the Jacobian matrix of the robotic arm to obtain joint velocity commands.

[0108] In this embodiment, the Jacobian matrix of the robotic arm refers to the partial derivative matrix describing the linear mapping relationship between joint velocities and end effector velocities. Inverse kinematics solving refers to the process of solving for the corresponding joint velocities using the inverse or generalized inverse of the Jacobian matrix based on the desired end effector velocities. Joint velocity commands refer to the desired joint angular velocities or joint velocity commands issued to the joint controllers of each joint of the robotic arm.

[0109] In one feasible implementation, the pseudo-inverse or damped least squares inverse of the Jacobian matrix of the robotic arm is calculated, and the desired end-effector velocity is multiplied by the generalized inverse matrix to obtain the particular solution part of the joint velocity command.

[0110] S604 combines joint velocity commands with zero-space velocity commands to generate joint control commands, and controls and adjusts the robotic arm of the underwater robot based on these joint control commands.

[0111] In this embodiment, zero-space velocity refers to the joint velocity component within the zero space of the robotic arm's Jacobian matrix that does not change the desired end-effector velocity but can adjust the joint configuration. Joint control commands refer to the control quantities, including position, velocity, or torque, ultimately sent to the joint actuator.

[0112] In one feasible implementation, a complete joint velocity command is generated by superimposing null space velocities that satisfy joint constraints, singularity avoidance, or energy consumption optimization on the inverse kinematics solution, and then sent to the actuator for execution.

[0113] For example, the specific form of the joint velocity command formula in the above steps is as follows:

[0114] Among them, formula parameters Represents joint speed command, The Jacobian matrix represents the robotic arm. Represents the generalized inverse of the Jacobian matrix. Represents the expected terminal velocity. Represents the identity matrix. This represents zero-space velocity that satisfies joint constraints, singularity avoidance, or energy optimization.

[0115] Please refer to the following: Figure 7 , Figure 7 This is a schematic diagram illustrating the execution process of controlling the underwater robot arm provided by the present invention. (See diagram below.) Figure 7 As shown, based on the desired end-effector velocity obtained by fusion, the inverse kinematics mapping is solved by combining the Jacobian matrix of the robotic arm to obtain the minimum norm key velocity command that drives the end-effector to generate the desired motion; the key velocity command is superimposed with the zero-space velocity in the zero space of the Jacobian matrix to achieve joint limit avoidance and configuration optimization without changing the motion state of the end-effector, thereby generating a complete joint control command; the joint control command is sent to the actuator to control each joint of the robotic arm to perform the corresponding motion to complete the underwater operation task.

[0116] In summary, this application achieves a smooth transition between manual commands and autonomous agent speeds and joint-level control execution through continuous weight fusion and inverse kinematics solution, thereby improving the stability and operational flexibility of closed-loop control.

[0117] In one feasible implementation, after performing the step of speed control of the underwater robot based on the desired end-effector velocity, the following is further performed: Obtain the state feature sequence of the operation objects within the history window.

[0118] In this embodiment, the historical window refers to a time period from the current moment back to a preset step length in the past. The state feature sequence refers to a multimodal feature vector sequence composed of master pose, velocity, electromyographic features, force sensation, environmental risk, and slave feedback state arranged in chronological order within the historical window.

[0119] In one feasible implementation, a historical window feature sequence with a buffer length of N is buffered. The feature sequence includes master end pose, master end velocity, root mean square value and median frequency of electromyography, master end interaction force, target relative pose, obstacle risk, water flow velocity, communication delay and slave end feedback status; the input is normalized and time series concatenated to form a state feature sequence.

[0120] The state feature sequence is processed based on a time delay compensation prediction model to obtain the predicted motion trajectory of the operation object.

[0121] In this embodiment, the time-delay compensation prediction model refers to a time-series prediction model built on a long short-term memory network, used to predict the future multi-step motion trajectory of an object based on a sequence of historical state features. The predicted motion trajectory refers to the sequence of the expected end pose or velocity trajectory of the object in the future prediction time domain output by the model.

[0122] In one feasible implementation, the state feature sequence is input into the delay compensation prediction model, the temporal features are extracted through the LSTM hidden layer, and the output of the hidden layer is passed through the fully connected regression layer to obtain the future H-step motion trajectory of the object, so as to offset the lag caused by the underwater acoustic communication delay.

[0123] The predicted motion trajectory of the manipulated object is used as a feedforward term, and compensation control commands are generated by combining environmental perception information.

[0124] In this embodiment, the feedforward term refers to the advanced control quantity calculated based on the predicted motion trajectory, used to respond in advance to the intention of the manipulated object. The compensation control command refers to the additional control command generated by fusing the feedforward term with the current environmental safety constraints to offset communication delays.

[0125] In one feasible implementation, the predicted motion trajectory is converted into the expected future velocity or position increment as a feedforward term, and a compensation control command is generated by combining the feedback state from the slave end with environmental safety constraints.

[0126] Based on the compensation control command, the desired speed at the terminal is subjected to time delay compensation processing.

[0127] In the embodiments of this application, the delay lag compensation process refers to the process of using compensation control commands to correct the current desired speed of the terminal to eliminate the control lag and state error caused by the round-trip delay of underwater acoustic communication.

[0128] In one feasible implementation, the compensation control command is superimposed onto the current end-point desired speed or position loop, and the predicted feedforward amount is used to compensate for the time delay caused by network transmission and data processing, thereby improving the stability of the teleoperation closed loop.

[0129] In summary, this application uses a time delay compensation prediction model to predict the future trajectory of the manipulated object based on the historical state feature sequence, and uses this as a feedforward term to generate compensation control commands, effectively compensating for the impact of long time delay in underwater acoustic communication on communication stability.

[0130] Underwater robots struggle to reuse single-trajectory skills in complex operational scenarios and are ill-equipped to handle long-term, complex tasks. Furthermore, semantic issues exist between high-level task commands and low-level motion control. Therefore, please refer to [link to relevant documentation]. Figure 8 , Figure 8 This is a flowchart illustrating the underwater robot control method provided in an embodiment of this application. Figure 8 As shown, the method in this application embodiment may include the following steps S801-S803: S801, establish a skill library for underwater robot operations.

[0131] In this embodiment of the application, the job skill base refers to a hierarchical skill database that includes basic actions, combined actions, and expert tasks.

[0132] In one feasible implementation, a multi-level granularity skill library is constructed: basic actions include approaching, aligning, gripping, releasing, and withdrawing; combined actions include grasping, sampling, detecting, and repairing; and expert tasks include cage inspection, seabed component maintenance, and target recovery. Each skill is represented by dynamic motion primitives.

[0133] S802 enhances and learns the motion trajectories in the operation skill library based on fluid disturbance compensation to obtain underwater operation skill strategies.

[0134] In this embodiment, the motion trajectory refers to the end effector motion trajectory encoded by dynamic motion primitives in the job skill library. Enhancement and strategy learning processing refers to the enhancement learning process that adds fluid compensation terms to the dynamic equations and adapts the skill trajectory to the underwater flow field environment through simulation and online correction. Underwater operation skill strategy refers to a skill execution strategy that, after flow field adaptability enhancement, can be stably executed in a disturbed underwater environment.

[0135] In one feasible implementation, for skills represented by dynamic motion primitives in the operational skill library, a fluid compensation term is added to the dynamic equation to enhance disturbance; the enhanced disturbance trajectory under different flow velocities, directions and vortex conditions is generated through computational fluid dynamics simulation, and the skill strategy is pre-trained using a virtual simulation platform and corrected online in a real tank or sea trial environment to achieve virtual-real collaborative transfer from land teaching to underwater operations.

[0136] For example, the specific form of the perturbation-enhanced dynamic motion primitive formula in the above steps is as follows:

[0137] Among them, formula parameters Represents the skill trajectory state. Represents internal state variables. Represents the target state. Represents the phase variable. The nonlinear forcing term representing the dynamic motion primitive, Represents the time scale parameter. and Represents stability parameters. This represents the fluid compensation term.

[0138] S803 enables autonomous operation control of underwater robots based on underwater operation skill strategies.

[0139] In this embodiment, autonomous operation control refers to the process by which an underwater robot, upon receiving a high-level task instruction or authorization from an operating object, invokes the matching underwater operation skill strategy from the skill library to autonomously complete the underlying motion trajectory planning and execution.

[0140] In one feasible implementation, when the target object triggers a specific operational skill through voice command or intent confidence, the system calls the enhanced underwater operational skill strategy, which generates the desired pose and velocity in the current phase using dynamic motion primitives and sends them to the inverse kinematics module for execution, thereby achieving autonomous underwater operation.

[0141] Please refer to the following: Figure 9 , Figure 9 This is a schematic diagram illustrating the process of autonomous operation control for an underwater robot provided by the present invention. Figure 9 As shown, firstly, teaching data from land or a calm environment is acquired. Then, by introducing fluid disturbance compensation, the motion trajectory represented by the teaching data is enhanced and processed through strategy learning to adapt it to complex flow field environments. This generates robust underwater operation skill strategies and stores them in the operation skill library. In actual operation, the underwater robot can apply the matching underwater operation skill strategies in the operation skill library for autonomous operation control, thereby achieving effective transfer and autonomous execution from land-based teaching skills to the underwater dynamic environment.

[0142] In summary, this application enhances the transferability and current resistance of underwater operation skills from land-based teaching to the underwater environment by establishing a hierarchical skill library and introducing fluid compensation terms into dynamic motion primitives for reinforcement and strategy learning.

[0143] The following will combine Figure 10 This application provides a detailed description of the underwater robot control device provided in its embodiments. It should be noted that... Figure 10 The underwater robot control device in the present application is used to perform the functions described herein. Figures 1-9 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts relevant to the embodiments of this application. For specific technical details not disclosed, please refer to this application. Figures 1-9 In the embodiment shown, the underwater robot control device 10 may include a data acquisition unit 11, a data processing unit 12, a data generation unit 13, a weight factor acquisition unit 14, and a speed control unit 15, as detailed below: Data acquisition unit 11 is used to acquire operational feature information of the underwater robot and environmental perception information of the water environment in which the underwater robot is located. The operational feature information is a set of multimodal operational information issued by the underwater robot's operating object to the underwater robot. Data processing unit 12 is used to determine the operation instruction speed, intent confidence and fatigue level of the operation object based on operation feature information; Data generation unit 13 is used to process environmental perception information, including fluid disturbance compensation and obstacle avoidance, to obtain robot agent speed. Robot agent speed is the motion guidance speed that guides the underwater robot to approach the target and avoid obstacles. The weighting factor acquisition unit 14 is used to determine the control weighting factor based on the intent confidence, fatigue level, and fluid disturbance intensity and obstacle risk in the environmental perception information. The speed control unit 15 is used to fuse the operation command speed, the robot agent speed and the control weight factor to obtain the desired end speed of the underwater robot, and to control the speed of the underwater robot based on the desired end speed.

[0144] The underwater robot control device 10 provided in the above embodiments can realize the technical solutions described in the above underwater robot control method embodiments. The specific implementation principles of each module or unit can be found in the corresponding content in the above underwater robot control method embodiments, and will not be repeated here.

[0145] like Figure 11 As shown, the present invention also provides an electronic device 20. The electronic device 20 includes a processor 21, a memory 22, a display 23, and a data acquisition device 24. Figure 11 Only some of the components of the electronic device 20 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0146] In some embodiments, processor 21 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 22 or process data, such as the underwater robot control method of the present invention.

[0147] In some embodiments, processor 21 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 21 may be local or remote. In some embodiments, processor 21 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, inter-cloud, multi-cloud, or any combination thereof.

[0148] In some embodiments, memory 22 may be an internal storage unit of electronic device 20, such as a hard disk or memory of electronic device 20. In other embodiments, memory 22 may also be an external storage device of electronic device 20, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 20.

[0149] Furthermore, the memory 22 may include both internal storage units of the electronic device 20 and external storage devices. The memory 22 is used to store application software and various types of data installed on the electronic device 20.

[0150] In some embodiments, display 23 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 23 is used to display information from electronic device 20 and to display a visual user interface. Data acquisition unit 24 is used to collect operational characteristic information of the underwater robot and environmental perception information of the aquatic environment in which the underwater robot is located. Components 21-24 of electronic device 20 communicate with each other via a system bus.

[0151] In one embodiment, when processor 21 executes the underwater robot control program in memory 22, the following steps can be performed: Acquire operational characteristic information for the underwater robot and environmental perception information of the water environment in which the underwater robot is located. The operational characteristic information is a set of multimodal operational information issued by the underwater robot's operating object to the underwater robot. Based on operational characteristic information, determine the speed of operational instructions, confidence level of intent, and fatigue level of the operation object; The environmental perception information is processed including fluid disturbance compensation and obstacle avoidance to obtain the robot agent speed, which is the motion guidance speed that guides the underwater robot to approach the target and avoid obstacles. Based on intent confidence, fatigue, and fluid disturbance intensity and obstacle risk in environmental perception information, control weighting factors are determined. The operating command speed, robot agent speed, and control weight factor are fused together to obtain the desired end-effector speed of the underwater robot, and the underwater robot speed is controlled based on the desired end-effector speed.

[0152] It should be understood that when the processor 21 executes the underwater robot control program in the memory 22, in addition to the functions mentioned above, it can also perform other functions, as detailed in the description of the corresponding method embodiments above.

[0153] Furthermore, this embodiment of the invention does not specifically limit the type of electronic device 20 mentioned. Electronic device 20 can be a tablet computer, a personal digital assistant (PDA), a wearable device, a laptop computer, or other portable electronic devices. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, electronic device 20 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0154] Accordingly, this application also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the underwater robot control methods provided in the above-described method embodiments.

[0155] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0156] The underwater robot control method, device, and electronic equipment provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A control method for an underwater robot, characterized in that, include: The operation feature information of the underwater robot and the environmental perception information of the water environment in which the underwater robot is located are obtained. The operation feature information is a set of multimodal operation information issued by the operation object of the underwater robot to the underwater robot. Based on the operational feature information, the operation command speed, intent confidence, and fatigue level of the operation object are determined; The environmental perception information is processed including fluid disturbance compensation and obstacle avoidance to obtain the robot agent speed, which is the motion guidance speed that guides the underwater robot to approach the target and avoid obstacles. Based on the intent confidence level, the fatigue level, and the fluid disturbance intensity and obstacle risk in the environmental perception information, a control weighting factor is determined. The operation command speed, the robot agent speed, and the control weight factor are fused together to obtain the desired end-effector speed of the underwater robot, and the underwater robot speed is controlled based on the desired end-effector speed.

2. The method according to claim 1, characterized in that, The step of determining the operation command speed, intent confidence, and fatigue level of the operation object based on the operation feature information includes: The operational feature information is subjected to anti-interference filtering and time alignment processing to obtain alignment features; Based on the pose information and kinematic information in the operation feature information, the operation command speed of the operation object is determined; The alignment features are input into the intent recognition model for processing to determine the intent confidence of the operation object; The root mean square value and median frequency of the spectrum are calculated based on the electromyographic signal characteristics in the operation feature information, and the fatigue level of the operation object is determined in combination with the continuous operation time.

3. The method according to claim 1, characterized in that, The process of processing the environmental perception information, including fluid disturbance compensation and obstacle avoidance, to obtain the robot agent speed includes: Based on the target object in the environmental perception information, the target pose of the underwater robot is determined, and the target attraction item is determined based on the target pose and the current pose of the end effector. Analyze the obstacle location information in the environmental perception information to determine the obstacle rejection terms; Based on the water flow velocity and end effector velocity in the environmental perception information, a fluid disturbance compensation term is determined; The target attraction term, the obstacle repulsion term, and the fluid disturbance compensation term are fused and limited to obtain the robot agent speed.

4. The method according to claim 1, characterized in that, The determination of control weighting factors based on the intent confidence level, the fatigue level, and the fluid disturbance intensity and obstacle risk in the environmental perception information includes: The speed of the operation command is compared with the speed of the robot agent to obtain the master-slave motion consistency index; The fluid disturbance intensity, obstacle risk, communication delay, fatigue level, intention confidence level and master-slave motion consistency index are weighted and calculated to obtain a weighted result. The communication delay is the round-trip time difference between the command transmission and / or status feedback between the operation device of the operation object and the underwater robot. The weighted result is subjected to interval limiting processing to obtain the control weight factor.

5. The method according to claim 1, characterized in that, The step of fusing the operation command speed, the robot agent speed, and the control weight factor to obtain the desired end-effector speed of the underwater robot includes: The control weight factor is used as the fusion weight of the robot agent speed, and the difference between the preset total weight value and the control weight factor is used as the fusion weight of the operation command speed. The desired end-effector speed of the underwater robot is obtained by weighted summation of the robot agent speed and its corresponding fusion weight, and the operation command speed and its corresponding fusion weight.

6. The method according to claim 1, characterized in that, After controlling the speed of the underwater robot based on the desired end-effector velocity, the method further includes: Based on the desired end-effector velocity and the Jacobian matrix of the robotic arm, inverse kinematics is performed to obtain the joint velocity command. The joint velocity command is combined with the zero-space velocity to generate a joint control command. The robotic arm of the underwater robot is controlled and adjusted based on the joint control command. The zero-space velocity is the velocity that satisfies joint limit and singularity avoidance.

7. The method according to claim 1, characterized in that, After controlling the speed of the underwater robot based on the desired end-effector velocity, the method further includes: Obtain the state feature sequence of the operation object within the history window; The state feature sequence is processed based on the time delay compensation prediction model to obtain the predicted motion trajectory of the operation object; The predicted motion trajectory of the manipulated object is used as a feedforward term, and a compensation control command is generated by combining the environmental perception information. Based on the compensation control command, the desired speed at the terminal is subjected to time delay compensation processing.

8. The method according to claim 1, characterized in that, The method further includes: Establish a skill set for underwater robots. Based on fluid disturbance compensation, the motion trajectories in the operation skill library are enhanced and processed with strategy learning to obtain underwater operation skill strategies. The underwater robot is autonomously controlled based on the underwater operation skill strategy.

9. An underwater robot control device, characterized in that, The device includes: The data acquisition unit is used to acquire operational feature information of the underwater robot and environmental perception information of the water environment in which the underwater robot is located. The operational feature information is a set of multimodal operational information issued by the operation object of the underwater robot to the underwater robot. The data processing unit is used to determine the operation instruction speed, intent confidence, and fatigue level of the operation object based on the operation feature information. The data generation unit is used to perform fluid disturbance compensation and obstacle avoidance processing on the environmental perception information to obtain the robot agent speed, which is the motion guidance speed that guides the underwater robot to approach the target and avoid obstacles. The weighting factor acquisition unit is used to determine the control weighting factor based on the intent confidence, the fatigue level, and the fluid disturbance intensity and obstacle risk in the environmental perception information. The speed control unit is used to fuse the operation command speed, the robot agent speed, and the control weight factor to obtain the desired end-effector speed of the underwater robot, and to perform speed control on the underwater robot based on the desired end-effector speed.

10. An electronic device, characterized in that, The electronic device includes a data acquisition unit, a memory, and a processor, wherein, The data acquisition device is used to acquire operational characteristic information of the underwater robot and environmental perception information of the water environment in which the underwater robot is located; The memory is used to store executable program code; The processor is configured to: determine the operation command speed, intent confidence, and fatigue level of the operation object based on the operation feature information; perform fluid disturbance compensation and obstacle avoidance processing on the environmental perception information to obtain the robot agent speed; determine a control weight factor based on the intent confidence, the fatigue level, and the fluid disturbance intensity and obstacle risk in the environmental perception information; fuse the operation command speed, the robot agent speed, and the control weight factor to obtain the desired end-effector speed of the underwater robot; and perform speed control on the underwater robot based on the desired end-effector speed.

Images