An underwater human-robot shared teleoperation control method based on bilateral trust mechanism

Abstract

The application discloses a kind of underwater man-machine shared teleoperation control methods based on bilateral trust mechanism.Method includes: master end force feedback device teleoperation control underwater mechanical arm moves, so that underwater mechanical arm end underwater hand-eye camera identifies underwater target, first, the master end evaluation model of master end force feedback device is established to obtain master end trust factor;Establish the visual evaluation model of underwater hand-eye camera, and finally obtain visual trust factor by density-based visual measurement outlier rejection method;Master end trust factor, visual trust factor and each control instruction are collectively processed by man-machine shared teleoperation method based on fuzzy logic, and the desired control instruction of underwater mechanical arm is obtained to carry out man-machine shared teleoperation control.The application can effectively eliminate the outlier of visual identification, ensure the reliability of visual servo system, introduce visual auxiliary system, can further reduce the burden of operator, improve the task success rate and operation efficiency of teleoperation system.

Description

Technical Field

[0001] This invention relates to an underwater teleoperation control method, specifically an underwater human-machine shared teleoperation control method based on a bilateral trust mechanism. Background Technology

[0002] Teleoperated robotics technology enables humans to enter challenging environments such as the deep sea to carry out underwater resource development and marine equipment maintenance tasks. Underwater robotic arms serve as effective tools to replace human hands. However, due to the latency of remote communication and limitations in environmental feedback (force / visual feedback, etc.), traditional manual operation methods exhibit significant limitations, especially under prolonged and delicate working conditions, where operator fatigue easily occurs, leading to decreased efficiency and accuracy. Furthermore, traditional manual operation methods heavily rely on the operator's control of the underwater robotic arm, thus requiring a high level of professional background and technical proficiency.

[0003] However, underwater robotic arms currently struggle to achieve full autonomy in exploring complex deep-sea environments. Therefore, to improve the intelligence level of traditional teleoperation, current research has clearly shifted towards developing different shared control strategies. The core objective is to integrate the robot's decision-making capabilities into human operations by detecting human intentions, allocating control permissions, and providing robot feedback, thereby emphasizing effective "human-robot collaboration." However, most current shared control strategies rely on unilateral states (operator or robot) for control permission transfer and are rarely applied in underwater environments. There is a lack of a unified framework to simultaneously address control permission transfer and assisted teleoperation, significantly increasing the operator's cognitive load and decision-making error rate. Summary of the Invention

[0004] To address the problems existing in the background art, this invention provides an underwater human-machine shared teleoperation control method based on a bilateral trust mechanism. This invention addresses the issues of high operator dependence and low efficiency in traditional manual operation modes by introducing a visual assistance system, which can further reduce the operator's burden and improve the task success rate and operational efficiency of the teleoperation system.

[0005] The technical solution adopted in this invention is:

[0006] The underwater human-machine shared remote operation control method based on a bilateral trust mechanism of the present invention includes:

[0007] Step 1: When the underwater robotic arm moves under the remote control of the master force feedback device, and the underwater hand-eye camera at the end of the underwater robotic arm identifies the underwater target, the master evaluation model of the master force feedback device is first established. The operator's end pose and speed of the master force feedback device and the preset task difficulty parameters when the underwater robotic arm performs the underwater operation are input into the master evaluation model. After processing, the master trust factor is output.

[0008] Step 2: Establish a visual evaluation model for the underwater hand-eye camera. Based on the target pose identified by the end-user underwater hand-eye camera, obtain the visual measurement anomaly probability through a density-based visual measurement outlier removal method. Input the visual measurement anomaly probability into the visual evaluation model, process it, and output the visual trust factor.

[0009] Step 3: Based on the end pose velocity of the master force feedback device and the target pose identified by the slave underwater hand-eye camera, control commands for several operating modes of the underwater robotic arm are obtained. The master trust factor, visual trust factor and various control commands are processed together through a human-machine shared teleoperation method based on fuzzy logic to obtain the desired control commands for the underwater robotic arm, and then the underwater robotic arm is controlled by human-machine shared teleoperation.

[0010] In the first step, the main evaluation model of the main force feedback device is as follows:

[0011]

[0012]

[0013]

[0014] in, and These are the master-end trust factor and its gain coefficient, respectively. , and These are the execution parameters of the main end and their upper and lower limits; Set the parameter limits for the main execution end; To control the cycle The number of; The end-effector pose and velocity of the main force feedback device; These are the preset task difficulty parameters for the underwater robotic arm.

[0015] In the second step, the visual evaluation model for the underwater hand-eye camera is as follows:

[0016]

[0017] in, and These are the visual trust factor and its gain coefficient, respectively. and These represent the visual measurement anomaly probability and its bias value, respectively.

[0018] In the second step, the density-based visual measurement outlier removal method specifically involves acquiring images of underwater targets using an underwater hand-eye camera, and then obtaining real-time... i Visual measurement information of underwater targets at all times At the initial sampling time, sampling begins when the visual measurement information is not null, and visual measurement information of N underwater targets is collected to form a network with a width of [missing information]. Measurement sliding window Measurement sliding window Inside N The average value of each visual measurement is recorded as the core point. Measuring sliding window After sliding to the next moment, obtain the visual measurement information for that moment. If the visual measurement information for the next moment is null or exceeds the previously obtained core point, then proceed. Preset area radius If the visual measurement information at the next time step is considered an outlier, a zero-order preservation method is used to assign the visual measurement information at the next time step to the visual measurement information at the previous time step to remove the outlier. Then, the non-outlier value or the assigned visual measurement information at the next time step is entered into the measurement sliding window. Continue with the next outlier removal process. Record the number of outliers found at the current moment. Obtain the probability of visual measurement anomalies at the current moment. , .

[0019] In the third step, the underwater robotic arm's operating modes include direct teleoperation, visual servoing, collaborative operation, and emergency stop mode. The specific control commands for each operating mode are as follows:

[0020] a) Control commands in direct teleoperation mode :

[0021]

[0022]

[0023] in, For variable gain nonlinear coefficients; The end-effector pose and velocity of the main force feedback device; To control the cycle; Gain with constant coefficients; and These represent the position of the end effector of the underwater robotic arm and the boundary of its workspace. This represents the L2 norm.

[0024] b) Control commands in visual servo mode :

[0025]

[0026]

[0027]

[0028] in, This represents the maximum speed of the underwater robotic arm's end effector. Let the target object pose function be used. and The poses of the target object and the underwater robotic arm end effector relative to the underwater robotic arm base coordinate system are respectively. The target object pose identified by the underwater hand-eye camera is the optimized pose. Let be the unit pose vector from the end effector of the underwater robotic arm to the target object.

[0029] c) Control commands in collaborative operation mode :

[0030]

[0031]

[0032]

[0033]

[0034]

[0035] in, Control commands in direct teleoperation mode Correction value; Control commands in visual servo mode The correction value, ; The unit pose vector from the end effector of the underwater robotic arm to the target object. Correction value; The pose of the underwater robotic arm's end effector Correction value; This is the pose transformation matrix of the underwater robotic arm's end effector relative to the coordinate system of the underwater robotic arm's base; For the pose transformation matrix of the underwater hand-eye camera relative to the end effector of the underwater robotic arm; Representing vectors exist The component in the direction is 0.

[0036] d) Control commands in emergency stop mode The value is 0. This mode is defined to deal with unexpected or dangerous situations. In emergency stop mode, the underwater robotic arm will remain stationary.

[0037] In the third step, the fuzzy logic-based human-machine shared teleoperation method specifically involves inputting a master-end trust factor and a visual trust factor, establishing a fuzzy set of master-end trust factors containing classes S (small S), M (medium M), and L (large L), and a fuzzy set of visual trust factors containing classes S (small S), M (medium M), and L (large L). A standard Gaussian membership function is used, and the centroid method is employed for defuzzification, along with a priority principle (master-end trust factor). Priority is higher than visual trust factor Specifically, when the visual trust factor... and master-slave trust factor When all belong to the small S class, the control command in emergency stop mode is used. Desired control commands for underwater robotic arms When visual trust factor and master-slave trust factor When all belong to the large L category, the control instructions in the cooperative operation mode are used. Desired control commands for underwater robotic arms When visual trust factor and master-slave trust factor When all commands belong to Class M, priority selection is used, i.e., control commands in direct teleoperation mode are used. Desired control commands for underwater robotic arms When visual trust factor Belongs to the Little S category, main-end trust factor When it belongs to the medium M category or the large L category, or the visual trust factor Belongs to the M category, main-end trust factor When it belongs to the large L category, the control commands in the direct teleoperation mode are used. Desired control commands for underwater robotic arms When the master-side trust factor Belongs to the Little S category, visual trust factor When it belongs to the M or L category, or the main trust factor Belongs to the M category, visual trust factor When it belongs to the large L category, the control commands in visual servo mode are used. Desired control commands for underwater robotic arms Ultimately, this controls the end effector position of the underwater robotic arm.

[0038] The beneficial effects of this invention are:

[0039] 1. The density-based outlier removal method for visual measurement designed in this invention can effectively remove outliers in visual recognition and ensure the reliability of the visual servo system.

[0040] 2. The underwater human-machine shared teleoperation control method based on a bilateral trust mechanism designed in this invention introduces a visual assistance system, which can help human operators reduce physical / mental workload and improve the success rate and operational efficiency of the teleoperation system. Attached Figure Description

[0041] Figure 1 This is a structural diagram of the underwater robotic arm teleoperation system of the present invention;

[0042] Figure 2 This is a block diagram of the underwater human-machine shared remote operation control system based on a bilateral trust mechanism designed in this invention;

[0043] Figure 3 This invention uses a standard Gaussian membership function graph for a human-machine shared remote operation control method based on fuzzy logic.

[0044] Figure 4 This is a diagram illustrating the effect of the density-based visual measurement outlier removal method designed in this invention.

[0045] Figure 5 This is a diagram illustrating the effect of the underwater human-machine shared remote operation control system based on a bilateral trust mechanism designed in this invention. Figure 5 (a) is a graph showing the changes in the primary trust factor and the visual trust factor. Figure 5 (b) is a diagram showing the changes in the underwater robotic arm's operating modes. Figure 5 (c) is a graph showing the variation of the desired position trajectory of the underwater robotic arm's end effector in the X-axis direction under four operating modes. Figure 5 (d) is a graph showing the variation of the desired position trajectory of the underwater robotic arm's end effector in the Y-axis direction under four operating modes. Figure 5 (e) is a graph showing the variation of the desired position trajectory of the underwater robotic arm end effector in the Z-axis direction under four operating modes;

[0046] Figure 6 This diagram illustrates the improvement in operational performance achieved by the method of this invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0048] like Figure 1 As shown, this is the structure of the underwater robotic arm teleoperation system of the present invention, wherein, The origin of the coordinate system of the base of the main force feedback device. The origin of the coordinate system at the end of the main force feedback device. and These are the X-axis and Z-axis coordinates of the end of the main force feedback device relative to the base coordinate system, respectively. Let the origin of the underwater robotic arm base coordinate system be the origin. Let O be the origin of the coordinate system at the end effector of the underwater robotic arm. and These are the X-axis and Z-axis coordinates of the underwater robotic arm's end effector relative to the base coordinate system, respectively. Let the origin of the underwater hand-eye camera coordinate system be... and These are the X-axis and Z-axis coordinates of the underwater hand-eye camera, respectively. The origin of the target object's coordinate system.

[0049] like Figure 2 As shown, the underwater human-machine shared remote operation control method based on a bilateral trust mechanism of the present invention is as follows:

[0050] Step 1: When the underwater robotic arm moves under the remote control of the master force feedback device, and the underwater hand-eye camera at the end of the underwater robotic arm identifies the underwater target, the master evaluation model of the master force feedback device is first established. The operator's end pose and speed of the master force feedback device and the preset task difficulty parameters when the underwater robotic arm performs the underwater operation are input into the master evaluation model. After processing, the master trust factor is output.

[0051] The specific evaluation model for the master-end force feedback device is as follows:

[0052]

[0053]

[0054]

[0055] in, and These are the master-end trust factor and its gain coefficient, respectively. , and These are the execution parameters of the main end and their upper and lower limits. Defined as in System control cycle The percentage of the task execution speed integral within the entire time interval; the larger the value, the stronger the task execution capability. Set the parameter limits for the master end. Defined as the preset task difficulty parameter Regarding the main client's task execution capability limits and preset task difficulty parameters A larger value indicates a greater task difficulty, and the host will select the value between 0 and 1 for each specific task. To control the cycle The number of; The end-effector pose and velocity of the main force feedback device; These are the preset task difficulty parameters for the underwater robotic arm.

[0056] The performance of the main force feedback device is related to factors such as the difficulty of the mission and its ability to perform underwater tasks, ensuring... Between [0,1], and the ability of the master end to execute tasks. Greater than a certain limit Master-side trust factor Keep it at 1, otherwise it's 0, and when the task difficulty... When increased, the limit value of the main terminal's ability to execute tasks. The number of tasks will increase accordingly, meaning that the more difficult the task, the smaller the fault tolerance margin for the master force feedback device, and thus the higher the requirements for the master.

[0057] Step 2: Establish a visual evaluation model for the underwater hand-eye camera. Based on the target pose identified by the end-user underwater hand-eye camera, obtain the visual measurement anomaly probability through a density-based visual measurement outlier removal method. Input the visual measurement anomaly probability into the visual evaluation model, process it, and output the visual trust factor.

[0058] The visual evaluation model for underwater hand-eye cameras is as follows:

[0059]

[0060] in, and These are the visual trust factor and its gain coefficient, respectively. and These represent the visual measurement anomaly probability and its bias value, respectively.

[0061] The performance of visual servoing systems largely depends on the reliability of visual measurement information. In underwater environments, when the target is occluded or high-contrast interference occurs, vision-based feedback systems exhibit time-varying uncertainties in measurement, resulting in outliers. To address this, a density-based visual measurement outlier removal method is proposed. Specifically, this method involves acquiring images of underwater targets using an underwater hand-eye camera, and then obtaining... i Visual measurement information of underwater targets at all times At the initial sampling time, sampling begins when the visual measurement information is not null, and visual measurement information of N underwater targets is collected to form a network with a width of [missing information]. Measurement sliding window Measurement sliding window Inside N The average value of each visual measurement is recorded as the core point. Measuring sliding window After sliding to the next moment, obtain the visual measurement information for that moment. If the visual measurement information for the next moment is null or exceeds the previously obtained core point, then proceed. Preset area radius If the visual measurement information at the next time step is considered an outlier, a zero-order preservation method is used to assign the visual measurement information at the next time step to the visual measurement information at the previous time step to remove the outlier. Then, the non-outlier value or the assigned visual measurement information at the next time step is entered into the measurement sliding window. Continue with the next outlier removal process. Record the number of outliers found at the current moment. Obtain the probability of visual measurement anomalies at the current moment. , .

[0062] Step 3: Based on the end pose velocity of the master force feedback device and the target pose identified by the slave underwater hand-eye camera, control commands for several operating modes of the underwater robotic arm are obtained. The master trust factor, visual trust factor and various control commands are processed together through a human-machine shared teleoperation method based on fuzzy logic to obtain the desired control commands for the underwater robotic arm, and then the underwater robotic arm is controlled by human-machine shared teleoperation.

[0063] The underwater robotic arm has four operating modes: direct teleoperation, vision servoing, collaborative operation, and emergency stop. The specific control commands for each operating mode are as follows:

[0064] a) Control commands in direct teleoperation mode :

[0065]

[0066]

[0067] in, For variable gain nonlinear coefficients; The end-effector pose and velocity of the main force feedback device; To control the cycle; Gain with constant coefficients; and These represent the position of the end effector of the underwater robotic arm and the boundary of its workspace. This represents the L2 norm.

[0068] In direct teleoperation mode, a variable gain nonlinear master-slave incremental mapping method is designed to fully utilize the workspace of the slave-end underwater manipulator. Variable gain nonlinear coefficients. The design ensures that the mapping coefficient decreases as the end effector of the underwater robotic arm approaches the workspace boundary.

[0069] b) Control commands in visual servo mode :

[0070]

[0071]

[0072]

[0073] in, This represents the maximum speed of the underwater robotic arm's end effector. Let the target object pose function be used. and The poses of the target object and the underwater robotic arm end effector relative to the underwater robotic arm base coordinate system are respectively. The target object pose identified by the underwater hand-eye camera is the optimized pose. Let be the unit pose vector from the end effector of the underwater robotic arm to the target object.

[0074] In visual servo mode, the movement of the underwater robotic arm is entirely controlled by the visual servo system, which gradually guides the end effector of the underwater robotic arm toward the target. To achieve precise control, a speed-limited path interpolation method is used during target recognition to generate control commands in visual servo mode. Functions can be designed to be any... The relevant functions satisfy the condition that the speed of the underwater robotic arm's end effector decreases as it gets closer to the target.

[0075] c) Control commands in collaborative operation mode :

[0076]

[0077]

[0078]

[0079]

[0080]

[0081] in, Control commands in direct teleoperation mode Correction value; Control commands in visual servo mode The correction value, ; The unit pose vector from the end effector of the underwater robotic arm to the target object. Correction value; The pose of the underwater robotic arm's end effector Correction value; This is the pose transformation matrix of the underwater robotic arm's end effector relative to the coordinate system of the underwater robotic arm's base; For the pose transformation matrix of the underwater hand-eye camera relative to the end effector of the underwater robotic arm; Representing vectors exist The component in the direction is 0.

[0082] In some underwater alignment tasks, such as during wet plug-and-socket assembly, slight vibrations from the operator during direct teleoperation can affect the precise alignment between the plug and socket, thus reducing operational efficiency. To address this challenge, real-time target data identified by an underwater hand-eye camera can assist operators in completing alignment tasks, thereby enabling collaborative work.

[0083] In collaborative operation mode, the underwater hand-eye camera at the slave end ensures that the target object remains in the center of the field of view of the underwater robotic arm's end effector, while the operator controls the underwater robotic arm's end effector to approach or move away from the target object by manipulating the force feedback device at the master end, without changing the attitude of the underwater robotic arm's end effector.

[0084] Specifically, the unit pose vector from the underwater robotic arm's end effector to the target object in the visual servoing command acquired in visual servoing mode. Make corrections to ensure the target object's position relative to the underwater robotic arm's end effector pose. The component in the direction is 0, thus obtaining the corrected unit pose vector. This ensures the target remains centered in the field of view of the underwater robotic arm's end effector, without approaching it, thus obtaining corrected visual commands. Further refine the operator commands in direct teleoperation mode. , making exist The directional projection is 0, allowing the operator to control the underwater robotic arm's end effector to approach or move away from the target object solely by manipulating the master-end force feedback device without altering the underwater robotic arm's attitude. This is the corrected operator command. Ultimately, the fused commands from the operator and the slave underwater hand-eye camera are obtained, which are the underwater robotic arm control commands in the collaborative operation mode. To enable visually assisted teleoperation.

[0085] d) Control commands in emergency stop mode The value is 0. This mode is defined to deal with unexpected or dangerous situations. In emergency stop mode, the underwater robotic arm will remain stationary.

[0086] The fuzzy logic-based human-computer shared teleoperation method specifically involves inputting a master-end trust factor and a visual trust factor, establishing a fuzzy set of master-end trust factors containing classes S (small S), M (medium M), and L (large L), and a fuzzy set of visual trust factors containing classes S (small S), M (medium M), and L (large L). A standard Gaussian membership function is used, along with centroid defuzzification and a priority principle (master-end trust factor). Priority is higher than visual trust factor Specifically, when the visual trust factor... and master-slave trust factor When all belong to the small S class, the control command in emergency stop mode is used. Desired control commands for underwater robotic arms When visual trust factor and master-slave trust factor When all belong to the large L category, the control instructions in the cooperative operation mode are used. Desired control commands for underwater robotic arms When visual trust factor and master-slave trust factor When all commands belong to Class M, priority selection is used, i.e., control commands in direct teleoperation mode are used. Desired control commands for underwater robotic arms When visual trust factor Belongs to the Little S category, main-end trust factor When it belongs to the medium M category or the large L category, or the visual trust factor Belongs to the M category, main-end trust factor When it belongs to the large L category, the control commands in the direct teleoperation mode are used. Desired control commands for underwater robotic arms When the master-side trust factor Belongs to the Little S category, visual trust factor When it belongs to the M or L category, or the main trust factor Belongs to the M category, visual trust factor When it belongs to the large L category, the control commands in visual servo mode are used. Desired control commands for underwater robotic arms Ultimately, this controls the end effector position of the underwater robotic arm.

[0087] This invention establishes a human-machine shared remote operation control strategy based on fuzzy logic. It inputs control commands for four operating modes of the underwater robotic arm, establishes a nonlinear relationship between the trust factor and the operating mode based on the master end trust factor and the visual trust factor, and the fuzzy rules are shown in Table 1. This outputs the desired control command for the underwater robotic arm, and finally realizes the human-machine shared remote operation control of the underwater robotic arm.

[0088] Table 1 Fuzzy Rules

[0089]

[0090] The two inputs to fuzzy logic are the master-end trust factor and the master-end trust factor. and visual trust factor The output is the desired control command for the underwater robotic arm, and the established fuzzy set contains small... ,middle Heda Using the standard Gaussian membership function, such as Figure 3 As shown. The fuzzy rules are shown in Table 1, which is based on the priority principle (master-side trust factor). Priority is higher than visual trust factor That is, when and When the membership function outputs are equally high, the master-end trust factor takes precedence. The centroid method is then used for defuzzification, ultimately producing the desired control command for the underwater robotic arm based on fuzzy logic. .

[0091] This invention inputs the speed of the force feedback device at the master end based on the master end evaluation model and the visual evaluation model. and the difficulty of underwater operations Output master-side trust factor Input the original pose information of the target object identified by the underwater hand-eye camera. Output visual trust factor This establishes a bilateral trust mechanism for the underwater robotic arm.

[0092] The method of this invention effectively solves the problems of high dependence on operators and low operation efficiency in traditional manual operation mode, and can further reduce the burden on operators and improve the task success rate and operation efficiency of remote operation system.

[0093] In its specific implementation, this invention's underwater human-machine shared teleoperation control method based on a bilateral trust mechanism was experimentally tested on a five-functional underwater electric manipulator platform consisting of a master force feedback hand and a hand-eye camera. The results were compared with direct teleoperation methods and shared teleoperation control methods based on unilateral trust to verify the performance improvements of the proposed underwater manipulator human-machine shared teleoperation control method. During verification, the system control cycle was 0.002s; in the direct teleoperation mode, the constant coefficient gain... Select as: The maximum permissible speed of the underwater robotic arm's end effector movement in visual servoing and collaborative operation modes. Select as: In the master-side trust assessment, the number of samples... Master-end trust factor gain coefficient Select as: Task difficulty Select as: , upper limit and lower limit Choose as follows: In visual trust assessment, the width of the sliding window is measured. Select as: Visual trust factor gain coefficient Select as: Bias value for visual measurement of anomaly probability Select as: Measure the radius of the region at the core point of the sliding window. Select as: .

[0094] The structure of the density-based visual measurement outlier removal method designed in this invention is as follows: Figure 4 As shown, Figure 4The solid line in the diagram represents the original signal of the underwater hand-eye camera recognizing the target object, while the dashed line represents the visual signal processed by the density-based visual measurement outlier removal method. It can be seen that when the target object recognized by the underwater hand-eye camera is obstructed or there is high-contrast interference in the underwater environment, i.e., the visual system experiences uncertain external interference, this leads to measurement outliers. The density-based visual measurement outlier removal method designed in this invention effectively eliminates these outliers, thereby ensuring the reliability of the visual servo system. The results of the underwater robotic arm's operating modes and the expected position trajectory of the end effector under each operating mode as a function of the trust factor are shown below. Figure 5 As shown, Figure 5 In (a), the solid line represents the primary trust factor, and the dashed line represents the visual trust factor. Figure 5 In (b), the solid lines represent the operating modes: 1 represents direct remote operation mode, 2 represents visual servo mode, 3 represents collaborative operation mode, and 4 represents emergency stop mode. Figure 5 (c) Figure 5 (d) and Figure 5 In (e), the solid line represents the desired position trajectory of the underwater manipulator's end effector in direct teleoperation mode, the light dashed line represents the desired position trajectory in visual servo mode, the black dashed line represents the desired position trajectory in collaborative operation mode, and the dotted line represents the final desired position trajectory of the underwater manipulator's end effector obtained through the human-machine shared teleoperation control strategy. It can be seen that the underwater manipulator can adaptively and smoothly switch between four operation modes (direct teleoperation, visual servo, collaborative operation, and emergency stop mode) based on the master-end trust factor and the visual trust factor, enabling the vision system to effectively assist the operator in completing underwater tasks. Finally, the improvement in operational performance of the underwater manipulator's human-machine shared teleoperation control method proposed in this invention is as follows: Figure 6 As shown, C1 represents the direct teleoperation method, C2 represents the shared teleoperation control method based on unilateral trust, and C3 represents the underwater manipulator shared teleoperation control method proposed in this invention. The bar chart represents the task completion time, and the line chart represents the task success rate. It can be seen that compared with direct teleoperation (C1), the shared teleoperation control method based on unilateral trust (C2) can shorten the completion time by about 30% and increase the success rate by about 30%, while the underwater manipulator shared teleoperation control method (C3) designed in this invention can shorten the completion time by about 35% and increase the success rate by about 40%.

[0095] Compared to traditional underwater robotic arm teleoperation methods, this invention employs visual servoing to assist the operator. Through fuzzy logic and a two-way trust mechanism, it achieves intelligent scheduling and smooth switching of various operating modes for the underwater robotic arm, resulting in significant improvements in task completion time and success rate. This demonstrates that the human-machine shared teleoperation control method for underwater robotic arms designed in this invention can help the master operator reduce physical and mental workload and improve the task success rate and operational efficiency of the underwater teleoperation system.

[0096] The above content is merely a technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A method for underwater human-machine shared remote operation control based on a bilateral trust mechanism, characterized in that, include: Step 1: The master force feedback device remotely controls the movement of the underwater manipulator, enabling the underwater hand-eye camera at the end of the underwater manipulator to identify underwater targets. At the same time, a master evaluation model of the master force feedback device is established. The end pose and velocity of the master force feedback device and the preset task difficulty parameters of the underwater manipulator are input into the master evaluation model. After processing, the master trust factor is output. Step 2: Establish a visual evaluation model for the underwater hand-eye camera. Based on the target pose identified by the end-user underwater hand-eye camera, obtain the visual measurement anomaly probability through a density-based visual measurement outlier removal method. Input the visual measurement anomaly probability into the visual evaluation model, process it, and output the visual trust factor. Step 3: Based on the end pose velocity of the master force feedback device and the target pose identified by the slave underwater hand-eye camera, control commands for several operating modes of the underwater robotic arm are obtained. The master trust factor, visual trust factor and various control commands are processed together through a human-machine shared teleoperation method based on fuzzy logic to obtain the desired control commands for the underwater robotic arm, and then the underwater robotic arm is controlled by human-machine shared teleoperation. In the first step, the main evaluation model of the main force feedback device is as follows: in, and These are the master-end trust factor and its gain coefficient, respectively. , and These are the execution parameters of the main end and their upper and lower limits; Set the parameter limits for the main execution end; To control the cycle The number of; The end-effector pose and velocity of the main force feedback device; Preset task difficulty parameters for the underwater robotic arm; Represents the L2 norm; In the second step, the visual evaluation model for the underwater hand-eye camera is as follows: in, and These are the visual trust factor and its gain coefficient, respectively. and These are the visual measurement anomaly probabilities and their bias values, respectively. In the second step, the density-based visual measurement outlier removal method specifically involves acquiring images of underwater targets using an underwater hand-eye camera, and then obtaining the visual measurement information of the underwater targets at time i in real time. At the initial sampling time, sampling begins when the visual measurement information is not null, and visual measurement information of N underwater targets is collected to form a network with a width of [missing information]. Measurement sliding window Measurement sliding window The average value of N visual measurement data points is denoted as the core point. Measuring sliding window After sliding to the next moment, obtain the visual measurement information for that moment. If the visual measurement information for the next moment is null or exceeds the previously obtained core point, then proceed. Preset area radius If the visual measurement information at the next time step is considered an outlier, a zero-order preservation method is used to assign the visual measurement information at the next time step to the visual measurement information at the previous time step to remove the outlier. Then, the non-outlier value or the assigned visual measurement information at the next time step is entered into the measurement sliding window. Continue the outlier removal process for the next time; record the number of outliers obtained at the current moment. Obtain the probability of visual measurement anomalies at the current moment. , .

2. The underwater human-machine shared remote operation control method based on a bilateral trust mechanism according to claim 1, characterized in that: In the third step, the underwater robotic arm's operating modes include direct teleoperation, visual servoing, collaborative operation, and emergency stop mode. The specific control commands for each operating mode are as follows: a) Control commands in direct teleoperation mode : in, For variable gain nonlinear coefficients; The end-effector pose and velocity of the main force feedback device; To control the cycle; Gain with constant coefficients; and These represent the position of the end effector of the underwater robotic arm and the boundary of its workspace. b) Control commands in visual servo mode : in, This represents the maximum speed of the underwater robotic arm's end effector. Let the target object pose function be used. and The poses of the target object and the underwater robotic arm end effector relative to the underwater robotic arm base coordinate system are respectively. Let be the unit pose vector from the end effector of the underwater robotic arm to the target object; c) Control commands in collaborative operation mode : in, Control commands in direct teleoperation mode Correction value; Control commands in visual servo mode Correction value; The unit pose vector from the end effector of the underwater robotic arm to the target object. Correction value; The pose of the underwater robotic arm's end effector Correction value; This is the pose transformation matrix of the underwater robotic arm's end effector relative to the coordinate system of the underwater robotic arm's base; For the pose transformation matrix of the underwater hand-eye camera relative to the end effector of the underwater robotic arm; d) Control commands in emergency stop mode It is 0.

3. The underwater human-machine shared remote operation control method based on a bilateral trust mechanism according to claim 1, characterized in that: In the third step, the fuzzy logic-based human-machine shared teleoperation method specifically involves inputting a master-end trust factor and a visual trust factor, establishing a fuzzy set of master-end trust factors containing classes S (small S), M (medium M), and L (large L), and a fuzzy set of visual trust factors containing classes S (small S), M (medium M), and L (large L). A standard Gaussian membership function is used, along with centroid defuzzification and priority principles. Priority is higher than visual trust factor ; Specifically, when the visual trust factor and master-slave trust factor When all belong to the small S class, the control command in emergency stop mode is used. Desired control commands for underwater robotic arms ; When visual trust factor and master-slave trust factor When all belong to the large L category, the control instructions in the cooperative operation mode are used. Desired control commands for underwater robotic arms When visual trust factor and master-slave trust factor When all commands belong to Class M, priority selection is used, i.e., control commands in direct teleoperation mode are used. Desired control commands for underwater robotic arms When visual trust factor Belongs to the Little S category, main-end trust factor When it belongs to the medium M category or the large L category, or the visual trust factor Belongs to the M category, main-end trust factor When it belongs to the large L category, the control commands in the direct teleoperation mode are used. Desired control commands for underwater robotic arms When the master-side trust factor Belongs to the Little S category, visual trust factor When it belongs to the M or L category, or the main trust factor Belongs to the M category, visual trust factor When it belongs to the large L category, the control commands in visual servo mode are used. Desired control commands for underwater robotic arms Ultimately, this controls the end effector position of the underwater robotic arm.

Images