A Force Control Insertion and Screwing Method for a Robot-Dexterous Hand System Based on Two-Level Search of Translation-Rotation

Abstract

The present invention relates to the technical field of intelligent robots, and particularly to a force control jacking and screwing method for a robot-dexterous hand system based on two-stage translational-rotational search. It includes: S1, generating grasping and screwing configurations based on the models of the dexterous hand of the robot-dexterous hand system and the tool for screwing the screw; S2, using a camera to collect the pose of the screw at the screw hole to be jacked, and moving the robot-dexterous hand system to the position of the hole to be jacked according to the screw pose; S3, the robot-dexterous hand system carrying the tool moves towards the screw in a direction perpendicular to the screw surface for preliminary jacking, and performs two-stage translational-rotational search for jacking on the basis of the preliminary jacking; S4, during the preliminary jacking process, based on the contact force collected by the force sensor on the robot-dexterous hand system for judgment, when the collected contact force reaches the expected force, record the movement amount of the tool in the direction perpendicular to the screw surface, and judge the relationship between the tool and the screw according to the movement amount. This solution can successfully grasp the tool and screw the screw of the special-shaped shaft hole.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent robots, and particularly relates to a force-controlled jacking and screwing method for a robot-dexterous hand system based on two-stage translation-rotation search. Background Art

[0002] At present, with the development of artificial intelligence and robot technology, due to its versatility and flexibility, the use of a robotic arm and a dexterous hand system for fine and complex operations has great application potential and can be applied to multiple fields such as industrial manufacturing and aerospace. For the task of using a robotic arm and a dexterous hand system to screw an electric tool, it is necessary to complete the jacking and screwing of special-shaped shaft holes such as hexagon socket head cap screws, and it faces difficulties and adjustments in robot planning and control. In terms of robot planning, the method needs to adapt to conditions such as the screw measurement accuracy not meeting the assembly tolerance and the difficulty in accurately aligning special-shaped shaft holes; in terms of force control during the robot jacking and screwing process, due to the certain offset of the grasping position of the dexterous hand, jamming is likely to occur, and the force control process needs to ensure stability, reliability, timely identify the state during the assembly process and handle abnormal situations. Therefore, to solve the above problems, it is of great significance to study a force-controlled jacking and screwing method for a robot-dexterous hand system based on two-stage translation-rotation search. Summary of the Invention

[0003] An embodiment of the present invention provides a force-controlled jacking and screwing method for a robot-dexterous hand system based on two-stage translation-rotation search, which can successfully grasp the tool and screw the screw of the special-shaped shaft hole.

[0004] An embodiment of the present invention provides a force-controlled jacking and screwing method for a robot-dexterous hand system based on two-stage translation-rotation search, including:

[0005] S1, generating a grasping and screwing configuration based on the models of the dexterous hand of the robot-dexterous hand system and the tool for screwing the screw; wherein, the robot-dexterous hand system includes a robotic arm and a dexterous hand fixed at the end of the robotic arm, and the dexterous hand is fixed with a tool;

[0006] S2, using a camera to collect the screw pose at the screw hole to be jacked, and moving the robot-dexterous hand system to the position to be jacked according to the screw pose;

[0007] S3, the robot-dexterous hand system moves the tool along the direction perpendicular to the screw surface towards the screw to perform preliminary jacking;

[0008] S4. During the initial jacking process, based on the contact force collected by the force sensor on the robot-dexterous hand system, when the collected contact force reaches the expected force, record the movement amount of the tool in the direction perpendicular to the screw surface, and determine the relationship between the tool and the screw according to the movement amount. The relationship between the tool and the screw includes the tool deviating from the screw, the tool being inserted into the screw hole, and the tool abutting against the screw surface. If the tool deviates from the screw, return to S2; if the tool is inserted into the screw hole, perform the tightening operation; if the tool abuts against the screw surface, move the robot-dexterous hand system to search for the jacking hole, then insert it into the screw hole and perform the tightening operation.

[0009] Optionally, in S4, the relationship between the tool and the screw further includes that the tool has not fully entered the screw hole. The moving the robot-dexterous hand system to search for the jacking hole includes:

[0010] The robot-dexterous hand system carries the tool to translate and search for the jacking hole based on the Lissajous curve, and determines the relationship between the tool and the screw according to the movement amount in the direction perpendicular to the screw surface during the translation search. If the tool deviates from the screw, return to S2, and when re-performing the translation search for the jacking hole, change the direction of the Lissajous curve and search again. If the tool is inserted into the screw hole, perform the tightening operation. If the tool has not fully entered the screw hole, perform a rotational search for the jacking hole.

[0011] When performing the rotational search for the jacking hole, the robot-dexterous hand system carries the tool to move and search within a preset angle range, and determines the relationship between the tool and the screw according to the movement amount in the direction perpendicular to the screw surface during the rotational search. If the tool deviates from the screw, return to S2, and when re-performing the translation search for the jacking hole, change the direction of the Lissajous curve and search again. If the tool is inserted into the screw hole, perform the tightening operation.

[0012] Optionally, in S4, the tightening operation includes:

[0013] Use the index finger of the dexterous hand to control the screwing of the tool, and the index finger adopts a force control mode; the remaining fingers adopt position control to maintain stability during the screwing process.

[0014] Optionally, in S4, during the tightening operation, determine whether the screw is tightened by the following method:

[0015] Judge the screwing relationship according to the movement amount in the direction perpendicular to the screw surface during the rotational search. The screwing relationship includes screwing deviation and successful screwing. If there is a screwing deviation, return to S2. If the screwing is successful, the robot-dexterous hand system carries the tool to the position of the jacking hole to be processed.

[0016] Optionally, in S4, when judging the relationship between the tool and the screw, it further includes:

[0017] Perform auxiliary determination by combining the template image and the real-time image.

[0018] Optionally, when determining the screwing relationship, it further includes:

[0019] Perform auxiliary determination by combining the template image and the real-time image.

[0020] Optionally, during the initial jacking process, constant-force compliance control is performed.

[0021] Optionally, it further includes:

[0022] Compare the force collected by the force sensor with the safety threshold in real time. If it exceeds the safety threshold, return to S2.

[0023] Optionally, the robotic arm has 6 active degrees of freedom, among which, 3 are translational and 3 are rotational. The dexterous hand includes five fingers, and the fingers have joints to achieve grasping of the electric tool. The tool includes an electric drill, and the camera is an RGBD camera.

[0024] The present invention has at least the following beneficial effects compared with the prior art:

[0025] The advantages of the present invention compared with the prior art are:

[0026] (1) In the jacking process of the robotic arm and dexterous hand system of the present invention, two-stage search is adopted. Compared with the existing methods, it can solve the assembly jamming problem caused by the difficulty in aligning the special-shaped shapes such as hexagon socket head cap screws.

[0027] (2) In the screwing process of the present invention, the difference between the screwing of the dexterous hand and the screwing of the ordinary mechanism is considered. Compared with the existing methods, it can adapt to the situation where the force application position of the wrist is offset from the screwing position, and prevent jamming during the assembly process.

[0028] (3) This method considers the safety during the jacking process. Compared with the existing methods, it can identify and adjust abnormal situations such as slipping.

[0029] (4) This method comprehensively considers the jacking and screwing processes. Compared with the existing methods, it can achieve stability during the transition between the two processes. Description of the Drawings

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0031] Figure 1Flowchart of the force-controlled jacking and screwing method for the robot-dexterous hand system based on two-stage translation-rotation search according to the present invention;

[0032] Figure 2 Configuration image of the dexterous hand grasping and screwing generated in the embodiment of the present invention;

[0033] Figure 3 Lissajous curve image adopted for translation search in the embodiment of the present invention;

[0034] Figure 4 Image of the robotic arm and dexterous hand performing jacking in the embodiment of the present invention;

[0035] Figure 5 Image of the robotic arm and dexterous hand performing screwing in the embodiment of the present invention. Detailed implementation manners

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0037] In the description of the embodiments of the present invention, unless otherwise clearly defined and limited, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance; unless otherwise specified or stated, the term "plurality" means two or more; the terms "connection", "fixation", etc. shall be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, an integral connection, or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0038] In the description of this specification, it should be understood that the orientation terms such as "upper" and "lower" described in the embodiments of the present invention are described from the angles shown in the accompanying drawings and should not be construed as limiting the embodiments of the present invention. In addition, in the context, it should also be understood that when it is mentioned that an element is connected "above" or "below" another element, it can not only be directly connected "above" or "below" another element, but also be indirectly connected "above" or "below" another element through an intermediate element.

[0039] As Figure 1 shown, the embodiments of the present invention provide a force-controlled jacking and screwing method for a robot-dexterous hand system based on two-stage translation-rotation search, including:

[0040] S1. Generate grasping and screwing configurations based on the models of the dexterous hand and the tool for screwing in the robot-dexterous hand system; wherein, the robot-dexterous hand system includes a robotic arm and a dexterous hand fixed at the end of the robotic arm, and a tool is fixed to the dexterous hand.

[0041] S2. Use a camera to collect the screw pose at the screw insertion hole, and move the robot-dexterous hand system to the position of the insertion hole according to the screw pose.

[0042] S3. The robot-dexterous hand system moves the tool towards the screw in a direction perpendicular to the screw surface for preliminary insertion.

[0043] S4. During the preliminary insertion process, judge based on the contact force collected by the force sensor on the robot-dexterous hand system. When the collected contact force reaches the expected force, record the movement amount of the tool in the direction perpendicular to the screw surface, and judge the relationship between the tool and the screw according to the movement amount. The relationship between the tool and the screw includes the tool deviating from the screw, the tool being inserted into the screw hole, and the tool abutting against the screw surface. If the tool deviates from the screw, return to S2; if the tool is inserted into the screw hole, perform the tightening operation; if the tool abuts against the screw surface, move the robot-dexterous hand system to search for the insertion hole, then insert it into the screw hole and perform the tightening operation.

[0044] Regarding S1, in this embodiment, the configuration generation result is as Figure 2 shown, and the specific process is as follows:

[0045] (1) Use a five-finger dexterous hand, and its simulation model is imported into the Mujoco engine in the form of URDF. Manually adjust the relative pose between the dexterous hand and the power tool and the angles of each joint in the simulation platform, and perform grasping simulation in Mujoco to adjust to a reasonable configuration where the tool is not dropped.

[0046] (2) In the physical environment, based on the configuration obtained in step (1), perform grasping tests to optimize the grasping configuration. And perform screwing configuration tests for turning on the tool, analyze the stability of the dexterous hand during the screwing process, and optimize a reasonable screwing configuration.

[0047] Regarding S2, in this embodiment, an RGBD camera and a pose estimation method are used to measure the screw pose T s at the screw insertion hole, and define the direction perpendicular to the screw surface and towards the inside of the screw as the z-axis, and its vector is az.

[0048] Regarding S3, in this embodiment, after moving to the pose of the insertion hole, perform robotic arm force control insertion based on the z-axis direction, and the specific process is as follows:

[0049] (1) Calculate the pose of the insertion hole T ps = T sT z T c , T s is the screw pose obtained through S2, T z is the adjusted pose, whose rotation part is the identity matrix and only the z-axis of the translation part has a non-zero component, meaning moving the tool to a certain distance from the screw surface along the screw axis, T c is the transformation relationship between the screw pose and the end-effector pose of the jacking state, which is obtained by calculating the three-dimensional models of the screw, electric tool, dexterous hand, and end-effector of the robotic arm. In this embodiment, a UR5e robot is used, and the MoveL instruction is used to move to the pose to be jacked.

[0050] (2) Perform robotic arm force control based on the z-axis direction of the screw, and the movement in other directions remains unchanged. In this embodiment, constant-force compliance control along the z-axis of the screw is adopted, and its control method is as follows:

[0051] (2.1) When the robotic arm is at the pose to be jacked, obtain the force / torque information of the current wrist force sensor at the end of the UR5e robot, and record the current information as the reference force / torque; to compensate for the gravity of the dexterous hand and electric tool, the real-time force during the subsequent movement is the force of the sensor data minus the reference force T z ;

[0052] (2.2) Based on the transformation relationship in (1), obtain the real-time force of the force sensor in the same direction as the z-axis of the screw as F Z , and set the desired force in this direction as F d .

[0053] (2.3) Adopt a constant-force compliance control method based on admittance control:

[0054] (2.3.1) Calculate the desired input force of the virtual system as F u = F d + k p F e + k i ∫F e dt, where F e = F d - F Z is the force error. k p and k i are the proportional gain and integral gain;

[0055] (2.3.2) Based on the admittance control rate Convert it to the discrete form, that is where M, B, and K represent virtual inertia, damping, and stiffness respectively;

[0056] (2.3.3) Based on step (2.3.2) and The output of the admittance control along the z-axis of the screw can be obtained

[0057] (2.4) Based on the output in step (2.2), transform the movement amount to the base coordinate system of the robotic arm, and use the ServoL instruction for real-time control to complete the real-time adjustment during the jacking process.

[0058] Regarding the judgment of the relationship between the tool and the screw in S4, the specific judgment method is as follows:

[0059] (1) When the contact force F Z first reaches F d , record the movement amount t0 of the robotic arm along the z-axis at this time, and judge whether it is greater than the threshold value t1. If it is greater than this value, it means that the jacking is off-center, and execute S2;

[0060] (2) If the movement amount t0 is between the threshold values t2 and t3, it means that the tool is directly inserted into the screw hole, judge that the jacking task is completed, and directly perform screwing;

[0061] (3) If the movement amount t0 is less than the threshold value t2, it means that the tool is at the surface of the screw and has not entered the screw hole, and judge that a search is needed.

[0062] In some embodiments of the present invention, in S4, the relationship between the tool and the screw further includes that the tool has not completely entered the screw hole, and the mobile robot - dexterous hand system searches for the jacking hole, including:

[0063] The robot - dexterous hand system carries the tool to perform translational search for the jacking hole based on the Lissajous curve, and judges the relationship between the tool and the screw according to the movement amount along the direction perpendicular to the screw surface during the translational search. If the tool deviates from the screw, return to S2, and when re-performing the translational search for the jacking hole, change the direction of the Lissajous curve and search again. If the tool is inserted into the screw hole, perform the tightening operation. If the tool has not completely entered the screw hole, perform rotational search for the jacking hole;

[0064] When performing rotational search for the jacking hole, the robot - dexterous hand system carries the tool to move and search within a preset angle range, and judges the relationship between the tool and the screw according to the movement amount along the direction perpendicular to the screw surface during the rotational search. If the tool deviates from the screw, return to S2, and when re-performing the translational search for the jacking hole, change the direction of the Lissajous curve and search again. If the tool is inserted into the screw hole, perform the tightening operation.

[0065] Regarding the translational search, the specific translation process is as follows:

[0066] (1) Calculate the direction vectors of the x-axis and y-axis of the screw in the coordinate system of the robot - dexterous hand system;

[0067] (2) Based on the direction vector calculated in step (1), generate the three-dimensional coordinates of each point on the Lissajous curve. The Lissajous curve is as shown in Figure 3 , and its calculation method is as follows:

[0068] Δx = a·sin(b x ·Δt)

[0069]

[0070] where a is the search range, and b x , b y and determine the shape of the Lissajous curve;

[0071] (3) Use the ServoL instruction to control the movement of the robot. Among them, the movement along the z-axis of the screw still uses the value calculated in real time in S3, and the movements along the x-axis and y-axis of the screw use the values calculated in real time in step (2).

[0072] For translational search, the judgment process is as follows:

[0073] Contact state judgment:

[0074] (1) Record the movement amount t of the robotic arm along the z-axis in real time r and the contact force F Z The difference t between the movement amount t when first reaching F d is judged whether it is greater than the threshold value t4. If it is greater than this value, it means that the jack is off, and S2 is executed.

[0075] (2) If the movement amount t r is between the threshold values t5 and t6, it means that the tool is directly inserted into the screw hole, and it is judged that the jacking task is completed, and the screwing is directly carried out;

[0076] (3) If the movement amount t r is between the threshold values t7 and t8, it means that the tool is inserted into a certain part on the screw surface, but due to the blockage of the internal hexagon shape, the internal hexagon ball head of the tool head has not completely entered the screw hole, and rotational search is required.

[0077] For rotational search, the rotation process is as follows:

[0078] (1) Establish the coordinate system of the end axis endpoint of the tool. Its z-axis is the tool axis outward, the x-axis is obtained by projecting the x-axis of the tool coordinate system onto the plane perpendicular to the z-axis, and the y-axis is obtained by cross-multiplication using the right-hand rule;

[0079] (2) According to the real-time pose tracking result of the power tool, calculate the pose of the current end axis endpoint coordinate system of the power tool relative to the end coordinate system of the robotic arm;

[0080] (3) Plan the range and speed of rotational search. In this embodiment, uniform motion search is adopted within the range of ±35°, and the expected pose is discretely sampled.

[0081] (4) Calculate the search path of the end point of the robotic arm based on the expected pose calculated in step (3) and the transformation relationship obtained in step (2).

[0082] (5) Use the ServoL instruction to control the movement of the robot. Among them, the movement along the z-axis of the screw still uses the value calculated in real time in S3, and the movement in other directions uses the value calculated based on step (4).

[0083] For the rotational search, the judgment process is as follows:

[0084] (1) Record in real time the relative movement amount t of the robotic arm along the z-axis after the search is started. rr , and judge whether it is greater than the threshold value t9. If it is greater than this value, it means that the insertion hole is off, and S2 is executed.

[0085] (2) If the movement amount t rr is between the threshold value t 10 and t 11 , it means that the tool has been inserted into the screw hole, judge that the insertion hole task is completed, and directly perform screwing.

[0086] In this embodiment, the complete insertion hole process is as Figure 4 shown. During the process of deviating and executing S2, the robotic arm moves upward along the z-axis of the screw. After leaving the screw range, it returns to S2 and repeats.

[0087] As Figure 5 shown, in some embodiments of the present invention, in S4, the tightening operation includes:

[0088] Use the index finger of the dexterous hand to control the screwing of the tool, and the index finger adopts a force control mode; the other fingers adopt position control to maintain stability during the screwing process.

[0089] The dexterous hand moves to the screwing configuration generated by S1. In this embodiment, the index finger of the dexterous hand is used to control the screwing of the tool, and the index finger adopts a force control mode; the other fingers adopt position control to maintain stability during the screwing process; during the screwing process, the robotic arm adopts the same control method as S3 to ensure following during the screwing process.

[0090] In some embodiments of the present invention, in S4, during the tightening operation, the following method is used to determine whether the screw is tightened:

[0091] The screwing relationship is judged according to the amount of movement in the direction perpendicular to the screw surface during the rotation search process. The screwing relationship includes screwing deviation and successful screwing. If there is a screwing deviation, return to S2. If the screwing is successful, the robot-dexterous hand system carries the tool to the position of the hole to be inserted.

[0092] The judgment of tightening is as follows:

[0093] (1) Record in real time the relative movement amount t of the robotic arm in the z-axis direction after the start of screwing rx , and judge whether it is greater than the threshold value t 10 . If it is greater than this value, it means that the screwing is off, and it is judged that the task needs to be stopped and S2 is executed.

[0094] (2) If it is expected that the screw is screwed a certain distance, the movement amount t rx is between the threshold values t 11 and t 12 , which means that the screwing is successful and the task is judged to be completed.

[0095] (3) If it is expected that the screw is completely screwed to the tightened state, then judge the change amount Δt of the movement amount t rx within a period of time t h , and the movement amount t rx is between the threshold values t 13 and t 14 , and the movement amount t rx is between the threshold values t 15 and t 16 , which means that the screwing is successful and the task is judged to be completed.

[0096] (4) Detect whether the screwing time is greater than the threshold value. If it exceeds, it means that an abnormal state has occurred. After manual confirmation, execute S2.

[0097] In some embodiments of the present invention, in S4, when judging the relationship between the tool and the screw, it further includes:

[0098] Auxiliary determination is performed by combining the template image and the real-time image.

[0099] In some embodiments of the present invention, when judging the screwing relationship, it further includes:

[0100] Auxiliary determination is performed by combining the template image and the real-time image.

[0101] During the first test, images of the above three states are collected as template images. The real-time image is compared with the template image based on the ssim method, and the judgment is assisted by combining the comparison value.

[0102] In some embodiments of the present invention, during the initial hole insertion process, constant force compliance control is performed.

[0103] In some embodiments of the present invention, it further includes:

[0104] Compare the force collected by the force sensor with the safety threshold in real time. If it exceeds the safety threshold, return to S2.

[0105] In some embodiments of the present invention, the robotic arm has 6 active degrees of freedom, among which, there are 3 translations and 3 rotations. The dexterous hand includes five fingers, and the fingers have joints to realize the grasping of an electric tool. The tool includes an electric drill, and the camera is an RGBD camera.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A robot-dexterous hand system force control jack and screwing method based on translation-rotation two-level search, characterized in that: include: S1, generating a grasping and screwing configuration based on a model of a dexterous hand of a robot-dexterous hand system and a tool for screwing a screw; wherein the robot-dexterous hand system comprises a mechanical arm and a dexterous hand fixed at the end of the mechanical arm, and the dexterous hand is fixed with a tool; S2, using a camera to collect the screw posture at the hole where the screw is to be inserted, and moving the robot-dexterous hand system to the position of the hole to be inserted according to the screw posture; S3, the robot-dexterous hand system carries the tool and moves toward the screw in a direction perpendicular to the surface of the screw to perform preliminary insertion; S4, in the process of preliminary plugging, a judgment is made based on the contact force collected by the force sensor on the robot-dexterous hand system. When the collected contact force reaches the expected force, the movement amount of the tool in the direction perpendicular to the surface of the screw is recorded, and the relationship between the tool and the screw is judged according to the movement amount. The relationship between the tool and the screw includes the tool deviating from the screw, the tool being inserted into the screw hole, and the tool abutting the screw surface. If the tool deviates from the screw, return to S2. If the tool is inserted into the screw hole, a tightening operation is performed. If the tool abuts the screw surface, the robot-dexterous hand system is moved to search for the plugging hole, and then the tool is inserted into the screw hole and a tightening operation is performed. In S4, the relationship between the tool and the screw further includes that the tool does not completely enter the screw hole, and the moving the robot-dexterous hand system to search for the socket includes: The robot-dexterous hand system carries a tool to search for a socket based on the Lissajous curve translation, and judges the relationship between the tool and the screw according to the amount of movement in the direction perpendicular to the screw surface during the translation search. If the tool deviates from the screw, it returns to S2, and when re-translationally searching for the socket, the direction of the Lissajous curve is changed to search again. If the tool is inserted into the screw hole, a tightening operation is performed. If the tool does not completely enter the screw hole, a rotation search for the socket is performed; When performing a rotational search for a socket, the robot-dexterous hand system carries the tool and moves within a preset angle range to search. The relationship between the tool and the screw is determined based on the amount of movement in a direction perpendicular to the screw surface during the rotational search. If the tool deviates from the screw, it returns to S2 and when re-translationally searching for the socket, the direction of the Lisa curve is changed to search again. If the tool is inserted into the screw hole, the tightening operation is performed.

2. The method according to claim 1, characterized in that In S4, the tightening operation includes: The index finger of the dexterous hand is used to control the screwing of the tool, and the index finger adopts the force control mode; the other fingers adopt the position control mode to maintain the stability during the screwing process.

3. The method according to claim 1, characterized in that In S4, during the tightening operation, whether the screw is tightened is determined by: The screwing relationship is judged according to the amount of movement in the direction perpendicular to the screw surface during the rotation search process. The screwing relationship includes deviation and successful screwing. If the screwing is deviation, it returns to S2. If the screwing is successful, the robot-dexterous hand system carries the tool to the position of the hole to be inserted.

4. The method according to claim 1, characterized in that: In S4, when judging the relationship between the tool and the screw, it also includes: Combine template image and real-time image for auxiliary judgment.

5. The method according to claim 3, characterized in that: When judging the twisting relationship, the method further includes: Combine template image and real-time image for auxiliary judgment.

6. The method according to claim 1, characterized in that During the initial insertion process, constant force and smooth control are performed.

7. The method according to claim 1, characterized in that Also includes: The force collected by the force sensor is compared with the safety threshold in real time. If the force exceeds the safety threshold, the process returns to S2.

8. The method according to claim 1, characterized in that The robotic arm has 6 active degrees of freedom, including 3 translations and 3 rotations. The dexterous hand includes five fingers, and the fingers have joints to enable grasping of electric tools. The tools include an electric drill. The camera is an RGBD camera.

Images