Underwater man-machine sharing type teleoperation control method based on bilateral trust mechanism

Abstract

The invention discloses an underwater man-machine sharing type teleoperation control method based on a bilateral trust mechanism. The method comprises the steps that main end force feedback equipment remotely operates and controls the underwater mechanical arm to move, so that when an underwater hand-eye camera at the tail end of the underwater mechanical arm recognizes an underwater target object, a main end evaluation model of the main end force feedback equipment is firstly established, and a main end trust factor is obtained; establishing a visual evaluation model of the underwater hand-eye camera, and finally obtaining a visual trust factor through a density-based visual measurement abnormal value elimination method; and the main end trust factor, the visual trust factor and all the control instructions are jointly processed through a man-machine sharing type teleoperation method based on fuzzy logic, and then an expected control instruction of the underwater mechanical arm is obtained so that man-machine sharing type teleoperation control can be carried out. Abnormal values of visual identification can be effectively eliminated, the reliability of a visual servo system is ensured, a visual auxiliary system is introduced, the burden of an operator can be further relieved, and the task success rate and the operation efficiency of a teleoperation system are improved.

Description

TECHNICAL FIELD

[0001] The application relates to an underwater teleoperation control method, in particular to an underwater human-robot shared teleoperation control method based on a bilateral trust mechanism. BACKGROUND

[0002] Teleoperation robot technology enables humans to enter challenging environments such as the deep sea to carry out underwater resource development, marine equipment maintenance and other work tasks, in which underwater manipulators are effective work tools for human replacement hands. However, due to the remote communication time delay and limited environmental feedback (force / tactile / visual feedback, etc.), traditional manual operation mode shows obvious limitations, especially under long-time and fine-scene working conditions, operators are prone to fatigue, resulting in a decrease in operation efficiency and accuracy. In addition, the traditional manual operation mode is highly dependent on the operator's control of the underwater manipulator, so the operator's professional background and technical proficiency are required.

[0003] However, it is still difficult for underwater manipulators to achieve complete autonomy in exploring tasks in complex deep-sea operating environments. Therefore, to improve the intelligent level of traditional teleoperation, current research has shifted to developing different shared control strategies, the core goal of which is to integrate the decision-making ability of the robot into human operation by detecting human operation intentions, allocating control rights, and providing robot feedback information, thereby emphasizing effective "human-robot collaboration". However, most of the current shared control strategies rely on unilateral state (operator or robot side) for control right transfer, and almost none have been applied in underwater environments, and there is rarely a unified framework to solve the problems of control right transfer and auxiliary teleoperation at the same time, greatly increasing the operator's cognitive load and decision-making error rate. SUMMARY

[0004] In order to solve the problems in the background art, the application provides an underwater human-robot shared teleoperation control method based on a bilateral trust mechanism. The application is used to solve the problems of high dependence on operators and low operation efficiency in traditional manual operation mode, and introduces a visual assistance system, which can further reduce the burden on operators and improve the task success rate and operation efficiency of the teleoperation system.

[0005] The technical scheme adopted by the application is: The underwater human-robot shared teleoperation control method based on the bilateral trust mechanism of the application comprises: The first step is to establish a master end evaluation model of the master end force feedback device when the underwater hand-eye camera at the end of the underwater manipulator identifies the underwater target object. The master end trust factor is outputted after the master end trust factor is inputted into the master end evaluation model, the end pose velocity of the master end force feedback device and the preset task difficulty parameter of the underwater manipulator are inputted into the master end evaluation model, and the master end trust factor is processed.

[0006] The second step is to establish a visual evaluation model of the underwater hand-eye camera. The visual trust factor is outputted after the visual trust factor is inputted into the visual evaluation model, the visual measurement abnormal probability is obtained by the density-based visual measurement abnormal value elimination method, and the visual measurement abnormal probability is inputted into the visual evaluation model and processed.

[0007] The third step is to obtain the control instruction of the underwater manipulator in several operation modes according to the end pose velocity of the master end force feedback device and the target object pose identified by the underwater hand-eye camera. The desired control instruction of the underwater manipulator is obtained by processing the master end trust factor, the visual trust factor and each control instruction through the fuzzy logic-based human-machine shared teleoperation method, and the underwater manipulator is controlled by the human-machine shared teleoperation.

[0008] In the first step, the master end evaluation model of the master end force feedback device is as follows: wherein, and are the master end trust factor and the gain coefficient thereof, respectively; , and are the master end execution parameter and the upper and lower limits thereof, respectively; is the master end execution parameter limit; is the number of control cycles ; is the end pose velocity of the master end force feedback device; is the preset task difficulty parameter of the underwater manipulator.

[0009] In the second step, the visual evaluation model of the underwater hand-eye camera is as follows: wherein, and are the visual trust factor and the gain coefficient thereof, respectively; and are the visual measurement abnormal probability and the bias value thereof, respectively.

[0010] The second step is that the density-based visual measurement outlier elimination method is specifically that the underwater hand-eye camera collects the image of the underwater target object, and then the visual measurement information of the underwater target object is obtained in real time i at the moment . When the visual measurement information is not empty at the initial sampling moment, the sampling is started, the visual measurement information of N underwater target objects is collected to form a measurement sliding window with a width of . The average value of the visual measurement information in the measurement sliding window is recorded as the core point N . When the measurement sliding window slides to the next moment to obtain the visual measurement information of the next moment, if the visual measurement information of the next moment is empty or exceeds the preset area radius of the core point obtained last time , the visual measurement information of the next moment is an outlier, and the visual measurement information of the next moment is assigned as the visual measurement information of the last moment by using the zero-order holding method to eliminate the outlier, and then the non-outlier or the assigned visual measurement information of the next moment is entered into the measurement sliding window to continue the next outlier elimination. The number of outliers obtained at the current moment is recorded , and the visual measurement abnormality probability at the current moment is obtained . .

[0011] In the third step, the operation modes of the underwater manipulator include direct teleoperation, visual servo, cooperative operation and emergency stop mode, and the control instructions in each operation mode are as follows: a) Control instruction in direct teleoperation mode : wherein, is a variable gain nonlinear coefficient; is the end position and pose speed of the master force feedback device; is the control period; is a constant coefficient gain; and are the end effector position and workspace boundary of the underwater manipulator, respectively; represents the two-norm.

[0012] b) Control instruction in visual servo mode : wherein, is the maximum velocity of the underwater manipulator end-effector motion; is the target object pose function; and are the poses of the target object and the underwater manipulator end-effector with respect to the underwater manipulator base frame respectively, the target object pose identified from the underwater end-effector camera is the optimized pose; is the unit pose vector from the underwater manipulator end-effector to the target object.

[0013] c) Control command in cooperative mode : wherein, is the correction value of the control command in direct teleoperation mode; is the correction value of the control command in visual servoing mode, ; is the correction value of the unit pose vector from the underwater manipulator end-effector to the target object; is the correction value of the underwater manipulator end-effector pose ; is the pose transformation matrix of the underwater manipulator end-effector with respect to the underwater manipulator base frame; is the pose transformation matrix of the underwater end-effector camera with respect to the underwater manipulator end-effector; denotes the vector has a component of 0 in direction.

[0014] d) Control command in emergency mode is 0. This mode is defined to cope with unexpected or dangerous situations, in emergency mode, the underwater manipulator is guaranteed to be stationary.

[0015] The third step is based on fuzzy logic human-machine shared remote operation method, specifically, inputting master terminal trust factor and visual trust factor, establishing master terminal trust factor fuzzy set containing small S, medium M and large L, and visual trust factor fuzzy set containing small S, medium M and large L, adopting standard Gaussian membership function, and adopting barycentric method to defuzzification and priority principle (master terminal trust factor priority is higher than visual trust factor ). Specifically, when the visual trust factor and the master terminal trust factor all belong to small S, the control instruction in the emergency stop mode is used as the expected control instruction of the underwater manipulator ; when the visual trust factor and the master terminal trust factor all belong to large L, the control instruction in the cooperative operation mode is used as the expected control instruction of the underwater manipulator ; when the visual trust factor and the master terminal trust factor all belong to medium M, priority selection is adopted, that is, the control instruction in the direct remote operation mode is used as the expected control instruction of the underwater manipulator ; when the visual trust factor belongs to small S, the master terminal trust factor belongs to medium M or large L, or the visual trust factor belongs to medium M, and the master terminal trust factor belongs to large L, the control instruction in the direct remote operation mode is used as the expected control instruction of the underwater manipulator ; when the master terminal trust factor belongs to small S, the visual trust factor belongs to medium M or large L, or the master terminal trust factor belongs to medium M, and the visual trust factor belongs to large L, the control instruction in the visual servo mode is used as the expected control instruction of the underwater manipulator , and finally the end pose of the underwater manipulator is controlled.

[0016] The beneficial effects of the present application are: 1. The density-based visual measurement outlier rejection method designed in the present application can effectively reject the outliers of visual recognition, and ensure the reliability of the visual servo system.

[0017] 2. The underwater human-machine shared teleoperation control method based on the bilateral trust mechanism has the visual auxiliary system, can help the human operator to reduce the physical / mental workload, and improves the task success rate and operation efficiency of the teleoperation system. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a structure diagram of the underwater manipulator teleoperation system of the present application; Figure 2 is a block diagram of the underwater human-machine shared teleoperation control system based on the bilateral trust mechanism designed by the present application; Figure 3 is a standard Gaussian membership function diagram used by the human-machine shared teleoperation control method based on fuzzy logic of the present application; Figure 4 is an effect diagram of the density-based visual measurement outlier rejection method designed by the present application; Figure 5 is an effect diagram of the underwater human-machine shared teleoperation control based on the bilateral trust mechanism designed by the present application, wherein, Figure 5 (a) of is a change diagram of the master trust factor and the visual trust factor, Figure 5 (b) of is a change diagram of the underwater manipulator operation mode, Figure 5 (c) of is a change diagram of the expected position trajectory of the underwater manipulator end effector in the X-axis direction under four operation modes, Figure 5 (d) of is a change diagram of the expected position trajectory of the underwater manipulator end effector in the Y-axis direction under four operation modes, Figure 5 (e) of is a change diagram of the expected position trajectory of the underwater manipulator end effector in the Z-axis direction under four operation modes; Figure 6 is an improvement effect diagram of the method of the present application in the operation performance. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical scheme and advantages of the present application more clear and understandable, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0020] As shown in Figure 1 , it is a structure of the underwater manipulator teleoperation system of the present application, wherein, is the origin of the base coordinate system of the master force feedback device, is the origin of the end coordinate system of the master force feedback device, and X and Z axis coordinates of the master end force feedback device relative to the base coordinate system respectively; is the origin of the underwater manipulator base coordinate system, is the origin of the underwater manipulator end coordinate system, and X and Z axis coordinates of the underwater manipulator end relative to the base coordinate system respectively; is the origin of the underwater hand-eye camera coordinate system, and X and Z axis coordinates of the underwater hand-eye camera respectively; is the origin of the target object coordinate system.

[0021] As shown in Figure 2 , the underwater human-machine shared teleoperation control method based on the bilateral trust mechanism of the application is as follows: Step 1: When the underwater hand-eye camera of the underwater manipulator end recognizes the underwater target object under the teleoperation control of the master end force feedback device, first, a master end evaluation model of the master end force feedback device is established, the end pose speed of the operator operating the master end force feedback device and the preset task difficulty parameter of the underwater manipulator executing the underwater operation task are input into the master end evaluation model, and the master end trust factor is output after processing.

[0022] The master end evaluation model of the master end force feedback device is as follows: wherein, and are the master end trust factor and its gain coefficient respectively; , and are the master end execution parameters and their upper and lower limits respectively, is defined as the proportion of the operation speed integral of executing the task in system control period to the entire time interval, and the larger the value is, the stronger the execution task ability is; is the master end execution parameter limit, is defined as the master end execution task ability limit related to the preset task difficulty parameter , and the larger the preset task difficulty parameter value is, the greater the task difficulty is, which is selected by the master end between 0 and 1 according to the specific task; is the number of control periods ; is the end pose speed of the master end force feedback device; is the preset task difficulty parameter of the underwater manipulator.

[0023] The performance of the master force feedback device is related to the task difficulty, the ability to perform underwater tasks, and other factors, and is guaranteed to be between [0, 1], and the ability of the master to perform the task is greater than a certain limit , the master trust factor is kept at 1, otherwise at 0, and when the task difficulty increases, the limit value of the ability of the master to perform the task will increase accordingly, which means the more difficult the task, the smaller the fault tolerance space of the master force feedback device, that is, the higher the requirement for the master.

[0024] Step 2: Establish a visual evaluation model for the underwater hand-eye camera. According to the pose of the target object recognized by the slave underwater hand-eye camera, obtain the visual measurement abnormal probability by the density-based visual measurement abnormal value elimination method, input the visual measurement abnormal probability into the visual evaluation model, and output the visual trust factor after processing; The visual evaluation model of the underwater hand-eye camera is as follows: wherein, and are the visual trust factor and its gain coefficient, respectively; and are the visual measurement abnormal probability and its bias value, respectively.

[0025] The performance of the visual servo system depends largely on the reliability of the visual measurement information. In the underwater environment, when the measurement target is blocked or the underwater operation environment appears high-contrast interference, the visual-based feedback system has time-varying uncertainty in measurement, that is, there are measurement outliers. To this end, a density-based visual measurement outlier elimination method is proposed. The density-based visual measurement outlier elimination method is as follows: the underwater hand-eye camera collects the image of the underwater target object, and then obtains the visual measurement information of the underwater target object at i time , at the initial sampling time, when the visual measurement information is not empty, the sampling starts, N visual measurement information of the underwater target object is collected to form a measurement sliding window with a width of ; the average value of the visual measurement information in the measurement sliding window is recorded as the core point N , and the measurement sliding window slides to the next time to obtain the visual measurement information at the next time, when the visual measurement information at the next time is empty or exceeds the preset area radius of the core point obtained last time If the visual measurement information at the next time step is considered an outlier, a zero-order preservation method is used to assign the visual measurement information at the next time step to the visual measurement information at the previous time step to remove the outlier. Then, the non-outlier value or the assigned visual measurement information at the next time step is entered into the measurement sliding window. Continue with the next outlier removal process. Record the number of outliers found at the current moment. Obtain the probability of visual measurement anomalies at the current moment. , .

[0026] Step 3: Based on the end pose velocity of the master force feedback device and the target pose identified by the slave underwater hand-eye camera, control commands for several operating modes of the underwater robotic arm are obtained. The master trust factor, visual trust factor and various control commands are processed together through a human-machine shared teleoperation method based on fuzzy logic to obtain the desired control commands for the underwater robotic arm, and then the underwater robotic arm is controlled by human-machine shared teleoperation.

[0027] The underwater robotic arm has four operating modes: direct teleoperation, vision servoing, collaborative operation, and emergency stop. The specific control commands for each operating mode are as follows: a) Control commands in direct teleoperation mode : in, For variable gain nonlinear coefficients; The end-effector position and velocity of the main force feedback device; To control the cycle; Gain with constant coefficients; and These represent the position of the end effector of the underwater robotic arm and the boundary of its workspace. This represents the L2 norm.

[0028] In direct teleoperation mode, a variable gain nonlinear master-slave incremental mapping method is designed to fully utilize the workspace of the slave-end underwater manipulator. Variable gain nonlinear coefficients. The design ensures that the mapping coefficient decreases as the end effector of the underwater robotic arm approaches the workspace boundary.

[0029] b) Control commands in visual servo mode : in, maximal velocity of the underwater manipulator end-effector motion; target object pose function; and respectively are the pose of the target object and the pose of the underwater manipulator end-effector relative to the underwater manipulator base frame, the target object pose identified from the underwater end-effector camera is the optimized pose; is the unit pose vector from the underwater manipulator end-effector to the target object.

[0030] In the visual servoing mode, the underwater manipulator motion is fully controlled by the visual servoing system, which gradually guides the underwater manipulator end-effector towards the target. In order to achieve precise control, the velocity limited path interpolation method is used when the target is identified, thus generating the control command in the visual servoing mode. The function can be designed as any function related to and satisfying the condition that the velocity of the underwater manipulator end-effector is smaller as it gets closer to the target object.

[0031] c) Control command in the cooperative work mode : wherein, is the correction value of the control command in the direct teleoperation mode; is the correction value of the control command in the visual servoing mode, ; is the correction value of the unit pose vector from the underwater manipulator end-effector to the target object; is the correction value of the pose of the underwater manipulator end-effector; is the pose transformation matrix of the underwater manipulator end-effector relative to the underwater manipulator base frame; is the pose transformation matrix of the underwater end-effector camera relative to the underwater manipulator end-effector; denotes that the component of the vector in the direction is 0.

[0032] In some underwater alignment tasks, such as during wet plug-and-socket assembly, slight vibrations from the operator during direct teleoperation can affect the precise alignment between the plug and socket, thus reducing operational efficiency. To address this challenge, real-time target data identified by an underwater hand-eye camera can assist operators in completing alignment tasks, thereby enabling collaborative work.

[0033] In collaborative operation mode, the underwater hand-eye camera at the slave end ensures that the target object remains in the center of the field of view of the underwater robotic arm's end effector, while the operator controls the underwater robotic arm's end effector to approach or move away from the target object by manipulating the force feedback device at the master end, without changing the attitude of the underwater robotic arm's end effector.

[0034] Specifically, the unit pose vector from the underwater robotic arm's end effector to the target object in the visual servoing command acquired in visual servoing mode. Make corrections to ensure the target object's position relative to the underwater robotic arm's end effector pose. The component in the direction is 0, thus obtaining the corrected unit pose vector. This ensures the target remains centered in the field of view of the underwater robotic arm's end effector, without approaching it, thus obtaining corrected visual commands. Further refine the operator instructions in direct teleoperation mode. , making exist The directional projection is 0, allowing the operator to control the underwater robotic arm's end effector to approach or move away from the target object solely by manipulating the master-end force feedback device without altering the underwater robotic arm's attitude. This is the corrected operator command. Ultimately, the fused commands from the operator and the slave underwater hand-eye camera are obtained, which are the underwater robotic arm control commands in the collaborative operation mode. To enable visually assisted teleoperation.

[0035] d) Control commands in emergency stop mode The value is 0. This mode is defined to deal with unexpected or dangerous situations. In emergency stop mode, the underwater robotic arm will remain stationary.

[0036] The fuzzy logic-based human-computer shared teleoperation method specifically involves inputting a master-end trust factor and a visual trust factor, establishing a fuzzy set of master-end trust factors containing classes S (small S), M (medium M), and L (large L), and a fuzzy set of visual trust factors containing classes S (small S), M (medium M), and L (large L). A standard Gaussian membership function is used, along with centroid defuzzification and a priority principle (master-end trust factor). Priority is higher than visual trust factor Specifically, when the visual trust factor... and master-slave trust factor When all belong to the small S class, the control command in emergency stop mode is used. Desired control commands for underwater robotic arms When visual trust factor and master-slave trust factor When all belong to the large L category, the control instructions in the cooperative operation mode are used. Desired control commands for underwater robotic arms When visual trust factor and master-slave trust factor When all commands belong to Class M, priority selection is used, i.e., control commands in direct teleoperation mode are used. Desired control commands for underwater robotic arms When visual trust factor Belongs to the Little S category, main-end trust factor When it belongs to the medium M category or the large L category, or the visual trust factor Belongs to the M category, main-end trust factor When it belongs to the large L category, the control commands in the direct teleoperation mode are used. Desired control commands for underwater robotic arms When the master-side trust factor Belongs to the Little S category, visual trust factor When it belongs to the M or L category, or the main trust factor Belongs to the M category, visual trust factor When it belongs to the large L category, the control commands in visual servo mode are used. Desired control commands for underwater robotic arms Ultimately, this controls the end effector position of the underwater robotic arm.

[0037] This invention establishes a human-machine shared remote operation control strategy based on fuzzy logic. It inputs control commands for four operating modes of the underwater robotic arm, establishes a nonlinear relationship between the trust factor and the operating mode based on the master end trust factor and the visual trust factor, and the fuzzy rules are shown in Table 1. This outputs the desired control command for the underwater robotic arm, and finally realizes the human-machine shared remote operation control of the underwater robotic arm.

[0038] Table 1 Fuzzy Rules The two inputs to fuzzy logic are the master-end trust factor and the master-end trust factor. and visual trust factor The output is the desired control command for the underwater robotic arm, and the established fuzzy set contains small... ,middle Heda Using the standard Gaussian membership function, such asFigure 3 As shown. The fuzzy rules are shown in Table 1, which is based on the priority principle (master-side trust factor). Priority is higher than visual trust factor That is, when and When the membership function outputs are equally high, the master-end trust factor takes precedence. The centroid method is then used for defuzzification, ultimately producing the desired control command for the underwater robotic arm based on fuzzy logic. .

[0039] This invention inputs the speed of the force feedback device at the master end based on the master end evaluation model and the visual evaluation model. and the difficulty of underwater operations Output master-side trust factor Input the original pose information of the target object identified by the underwater hand-eye camera. Output visual trust factor This establishes a bilateral trust mechanism for the underwater robotic arm.

[0040] The method of this invention effectively solves the problems of high dependence on operators and low operation efficiency in traditional manual operation mode, and can further reduce the burden on operators and improve the task success rate and operation efficiency of remote operation system.

[0041] In its specific implementation, this invention's underwater human-machine shared teleoperation control method based on a bilateral trust mechanism was experimentally tested on a five-function underwater electric manipulator platform consisting of a master force feedback hand and a hand-eye camera. The results were compared with direct teleoperation methods and shared teleoperation control methods based on unilateral trust to verify the performance improvement of the proposed underwater manipulator human-machine shared teleoperation control method. During verification, the system control cycle was 0.002s; in the direct teleoperation mode, the constant coefficient gain... Select as: The maximum permissible speed of the underwater robotic arm's end effector movement in visual servoing and collaborative operation modes. Select as: In the master-side trust assessment, the number of samples... Master-end trust factor gain coefficient Select as: Task difficulty Select as: , upper limit and lower limit Choose as follows: In visual trust assessment, the width of the sliding window is measured. Select as: Visual trust factor gain coefficient is selected as: , visual measurement abnormal probability bias value is selected as: , the area radius of the measurement sliding window core point is selected as: .

[0042] The structure of the density-based visual measurement outlier rejection method designed in the application is shown in Figure 4 , The solid line in Figure 4 represents the original signal of the underwater hand-eye camera recognizing the target object, and the dashed line represents the visual signal after the density-based visual measurement outlier rejection method processing. It can be seen that when the target object recognized by the underwater hand-eye camera is blocked or there is high-contrast interference in the underwater environment, that is, there is uncertain external interference in the visual system, resulting in measurement outliers. The density-based visual measurement outlier rejection method designed in the application effectively eliminates these outliers, thereby ensuring the reliability of the visual servo system. The results of the underwater manipulator operation mode and the end effector desired position trajectory under each operation mode changing with the trust factor of the application are shown in Figure 5 , The solid line in Figure 5 (a) represents the master trust factor, and the dashed line represents the visual trust factor, The solid line in Figure 5 (b) represents the operation mode, 1 represents the direct teleoperation mode, 2 represents the visual servo mode, 3 represents the collaborative operation mode, and 4 represents the emergency stop mode, The solid line in Figure 5 (c), The solid line in Figure 5 (d), and The solid line in Figure 5 (e) represents the underwater manipulator end effector desired position trajectory in the direct teleoperation mode, the light dashed line represents the underwater manipulator end effector desired position trajectory in the visual servo mode, the black dashed line represents the underwater manipulator end effector desired position trajectory in the collaborative operation mode, and the dotted line represents the underwater manipulator end effector final desired position trajectory obtained by the human-machine shared teleoperation control strategy. It can be seen that the underwater manipulator can adaptively and smoothly switch between the four operation modes (direct teleoperation, visual servo, collaborative operation, and emergency stop mode) according to the master trust factor and the visual trust factor, so that the visual system effectively assists the operator to complete the underwater task. Finally, the improvement results of the underwater manipulator human-machine shared teleoperation control method proposed in the application in terms of operation performance are shown in Figure 6As shown, C1 represents a direct teleoperation method, C2 represents a shared teleoperation control method based on unilateral trust, C3 represents the underwater manipulator human-machine shared teleoperation control method proposed by the application, the column chart represents the task completion time, and the line chart represents the task success rate, and it can be seen that compared with the direct teleoperation (C1), the shared teleoperation control method based on unilateral trust (C2) can shorten the completion time by about 30%, and the success rate is increased by about 30%, and the underwater manipulator human-machine shared teleoperation control method (C3) designed by the application can shorten the completion time by about 35%, and the success rate is increased by about 40%.

[0043] Compared with the traditional underwater manipulator teleoperation method, the application adopts visual servo to assist the operator, realizes intelligent scheduling and smooth switching of various operation modes of the underwater manipulator through fuzzy logic and design of a bidirectional trust mechanism, and has great improvement in task completion time and success rate, which shows that the underwater manipulator human-machine shared teleoperation control method designed by the application can help the master operator to reduce the physical / mental workload, and improve the task success rate and operation efficiency of the underwater teleoperation system.

[0044] The above content is only the technical idea of the application, and cannot limit the protection scope of the application, and any modification made according to the technical idea of the application on the basis of the technical scheme falls within the protection scope of the claims of the application.

Claims

1. An underwater human-in-the-loop shared teleoperation control method based on a bilateral trust mechanism, characterized by, The application relates to a method for human-robot shared teleoperation control of an underwater manipulator. The first step is: when a master force feedback device teleoperationally controls the underwater manipulator to move so that an underwater hand-eye camera at the end of the underwater manipulator identifies an underwater target object, a master evaluation model of the master force feedback device is first established; the end pose speed of the master force feedback device and preset task difficulty parameters of the underwater manipulator are input into the master evaluation model, and a master trust factor is output after processing; The second step is: a visual evaluation model of the underwater hand-eye camera is established; according to the target object pose identified by the underwater hand-eye camera, a visual measurement abnormal value elimination method based on density is used to obtain a visual measurement abnormal probability; the visual measurement abnormal probability is input into the visual evaluation model, and a visual trust factor is output after processing; The third step is: according to the end pose speed of the master force feedback device and the target object pose identified by the underwater hand-eye camera, control instructions of the underwater manipulator in several operation modes are obtained; the master trust factor, the visual trust factor and the control instructions are processed by a human-robot shared teleoperation method based on fuzzy logic to obtain expected control instructions of the underwater manipulator, and the underwater manipulator is teleoperated by human-robot sharing.

2. The method of claim 1, wherein the method is based on a bilateral trust mechanism. In the first step, the master evaluation model of the master force feedback device is as follows: wherein, and are the master trust factor and its gain coefficient, respectively; , and are the master execution parameter and its upper and lower limits, respectively; is the master execution parameter limit; is the control period number; is the end pose velocity of the master force feedback device; is the preset task difficulty parameter of the underwater manipulator.

3. The method of claim 1, wherein the method is based on a bilateral trust mechanism. In the second step, the visual evaluation model of the underwater hand-eye camera is as follows: wherein, and are the visual trust factor and its gain coefficient, respectively; and are the visual measurement anomaly probability and its bias value, respectively.

4. The method of claim 1, wherein the method is based on a bilateral trust mechanism. The second step is a density-based visual measurement outlier elimination method, which specifically comprises the following steps: collecting images of underwater targets by an underwater hand-eye camera, and then obtaining visual measurement information of underwater targets in real time i at each moment . When the visual measurement information is not a null value at the initial sampling moment, sampling is started, N visual measurement information of underwater targets is collected to form a measurement sliding window with a width of . The average of the visual measurement information in the measurement sliding window is recorded as a core point N . The measurement sliding window slides to the next moment to obtain visual measurement information of the next moment. When the visual measurement information of the next moment is a null value or exceeds a preset area radius of the core point obtained last time, the visual measurement information of the next moment is an outlier, and a zero-order hold method is used to assign the visual measurement information of the next moment to the visual measurement information of the last moment to eliminate the outlier. Then, the non-outlier or assigned visual measurement information of the next moment is entered into the measurement sliding window for the next outlier elimination. The number of outliers obtained at the current moment is recorded , and the visual measurement abnormality probability at the current moment is obtained . ​​ 5. The method of claim 1, wherein the method is based on a bilateral trust mechanism. In the third step, the operation modes of the underwater manipulator include direct teleoperation, visual servo, cooperative operation and emergency stop mode, and the control instructions in the operation modes are as follows: a) control instructions in direct teleoperation mode : wherein, is a variable gain nonlinearity coefficient; is the end pose velocity of the master force feedback device; is the control period; is a constant coefficient gain; and are the end effector position and workspace boundary of the underwater manipulator, respectively; denotes the two-norm; b) Control instructions in visual servoing mode : wherein, is the maximum velocity of the underwater manipulator end-effector motion; is the target object pose function; and are the poses of the target object and the underwater manipulator end-effector, respectively, relative to the underwater manipulator base frame; is the unit pose vector from the underwater manipulator end-effector to the target object; c) Control instructions in a cooperative mode of operation : wherein, is a correction value for the control command in direct teleoperation mode ; is a correction value for the control command in visual servoing mode ; is a correction value for the unit pose vector of the underwater manipulator end-effector to the target object ; is a correction value for the pose of the underwater manipulator end-effector ; is a pose transformation matrix of the underwater manipulator end-effector relative to the underwater manipulator base coordinate system; is a pose transformation matrix of the underwater hand-eye camera relative to the underwater manipulator end-effector; d) Control instructions in emergency stop mode is 0.

6. The method of claim 1, wherein the method is based on a bilateral trust mechanism. The man-machine shared remote operation method based on fuzzy logic in the third step specifically comprises: inputting a master terminal trust factor and a visual trust factor, establishing a master terminal trust factor fuzzy set comprising small S, medium M and large L classes and a visual trust factor fuzzy set comprising small S, medium M and large L classes, adopting a standard Gaussian membership function, and adopting a barycenter method to defuzzify and a priority principle (the priority of the master terminal trust factor is higher than that of the visual trust factor ). Specifically, when the visual trust factor and the master terminal trust factor both belong to the small S class, a control instruction in an emergency stop mode is adopted as a desired control instruction of the underwater mechanical arm. When the visual trust factor and the master trust factor both belong to the large L category, the control instruction under the cooperative operation mode is adopted as the desired control instruction of the underwater manipulator ; When the visual trust factor and the master trust factor both belong to the medium M class, a priority selection is adopted, i.e. the control instruction in the direct teleoperation mode is adopted as the desired control instruction of the underwater manipulator ; when the visual trust factor belongs to the small S class, the master trust factor belongs to the medium M class or the large L class, or the visual trust factor belongs to the medium M class, the master trust factor belongs to the large L class, then the control command in the direct teleoperation mode is adopted as the desired control command of the underwater manipulator ; when the master trust factor belongs to the small S class, the visual trust factor belongs to the medium M class or the large L class, or the master trust factor belongs to the medium M class, the visual trust factor belongs to the large L class, then the control command in the visual servo mode is adopted as the desired control command of the underwater manipulator , the end pose of the underwater manipulator is finally controlled.

Images