Robot-dexterous hand system force control jack and screwing method based on translation-rotation two-stage search

Abstract

The invention relates to the technical field of intelligent robots, in particular to a robot-dexterous hand system force control jack and screwing method based on translation-rotation two-stage search. Comprising the following steps: S1, generating a grasping and screwing configuration based on a dexterous hand of a robot-dexterous hand system and a model of a tool for screwing a screw; s2, a camera is adopted to collect the position and posture of a screw to be inserted into a hole, and the robot-dexterous hand system is moved to the position to be inserted into the hole according to the position and posture of the screw; s3, the robot-dexterous hand system carries a tool to move towards the screw in the direction perpendicular to the surface of the screw so as to carry out preliminary insertion, and insertion holes are searched in a translation-rotation two-stage mode on the basis of preliminary insertion; and S4, in the preliminary hole inserting process, judgment is conducted based on contact force collected by a force sensor on the robot-dexterous hand system, when the collected contact force reaches expected stress, the motion amount of the tool in the direction perpendicular to the surface of the screw is recorded, and the relation between the tool and the screw is judged according to the motion amount. According to the scheme, the tool can be smoothly grabbed to screw the screw with the special-shaped shaft hole.

Description

Technical Field

[0001] The invention relates to the technical field of intelligent robots, and in particular to a robot-dexterous hand system force-controlled jack and screwing method based on translation-rotation two-level search. Background Art

[0002] At present, with the development of artificial intelligence and robotics technology, the use of robotic arms and dexterous hand systems for fine and complex operations has great application potential due to its versatility and flexibility, and can be applied to industrial manufacturing, aerospace and other fields. For the task of screwing electric tools using robotic arms and dexterous hand systems, it is necessary to complete the insertion and screwing of special-shaped shaft holes such as hexagon sockets, which faces difficulties and adjustments such as robot planning and control. In terms of robot planning, the method needs to adapt to the conditions that the screw measurement accuracy does not meet the assembly tolerance and the special-shaped shaft holes are difficult to align accurately; in terms of force control during robot insertion and screwing, due to the certain bias of the grasping position of the dexterous hand, it is easy to get stuck, and the force control process needs to be stable and reliable, and the state of the assembly process must be identified in time and abnormal situations must be handled. Therefore, in order to solve the above problems, it is of great significance to study a translation-rotation two-level search robot-dexterous hand system force control insertion and screwing method. Summary of the invention

[0003] The embodiment of the present invention provides a robot-dexterous hand system force-controlled jack and screwing method based on translation-rotation two-level search, which can smoothly grasp the tool to screw the screw of the special-shaped shaft hole.

[0004] The embodiment of the present invention provides a robot-dexterous hand system force control jack and screwing method based on translation-rotation two-level search, including:

[0005] S1, generating a grasping and screwing configuration based on a model of a dexterous hand of a robot-dexterous hand system and a tool for screwing a screw; wherein the robot-dexterous hand system comprises a mechanical arm and a dexterous hand fixed at the end of the mechanical arm, and the dexterous hand is fixed with a tool;

[0006] S2, using a camera to collect the screw posture at the hole where the screw is to be inserted, and moving the robot-dexterous hand system to the position of the hole to be inserted according to the screw posture;

[0007] S3, the robot-dexterous hand system carries the tool and moves toward the screw in a direction perpendicular to the surface of the screw to perform preliminary insertion;

[0008] S4, during the initial jacking process, a judgment is made based on the contact force collected by the force sensor on the robot-dexterous hand system. When the collected contact force reaches the expected force, the movement of the tool in a direction perpendicular to the surface of the screw is recorded, and the relationship between the tool and the screw is judged based on the movement. The relationship between the tool and the screw includes the tool deviating from the screw, the tool being inserted into the screw hole, and the tool abutting the screw surface. If the tool deviates from the screw, return to S2. If the tool is inserted into the screw hole, a tightening operation is performed. If the tool abuts the screw surface, the robot-dexterous hand system is moved to search for the jack, and then the tool is inserted into the screw hole and a tightening operation is performed.

[0009] Optionally, in S4, the relationship between the tool and the screw further includes that the tool does not completely enter the screw hole, and the moving the robot-dexterous hand system to search for the socket includes:

[0010] The robot-dexterous hand system carries a tool to search for a socket based on the Lissajous curve translation, and judges the relationship between the tool and the screw according to the amount of movement in the direction perpendicular to the screw surface during the translation search. If the tool deviates from the screw, it returns to S2, and when re-translationally searching for the socket, the direction of the Lissajous curve is changed to search again. If the tool is inserted into the screw hole, a tightening operation is performed. If the tool does not completely enter the screw hole, a rotation search for the socket is performed;

[0011] When performing a rotational search for a socket, the robot-dexterous hand system carries the tool and moves within a preset angle range to search. The relationship between the tool and the screw is determined based on the amount of movement in a direction perpendicular to the screw surface during the rotational search. If the tool deviates from the screw, it returns to S2 and when re-translationally searching for the socket, the direction of the Lisa curve is changed to search again. If the tool is inserted into the screw hole, the tightening operation is performed.

[0012] Optionally, in S4, the tightening operation includes:

[0013] The index finger of the dexterous hand is used to control the screwing of the tool, and the index finger adopts the force control mode; the other fingers adopt the position control mode to maintain the stability during the screwing process.

[0014] Optionally, in S4, during the tightening operation, determining whether the screw is tightened is performed in the following manner:

[0015] The screwing relationship is judged according to the amount of movement in the direction perpendicular to the screw surface during the rotation search process. The screwing relationship includes deviation and successful screwing. If the screwing is deviation, it returns to S2. If the screwing is successful, the robot-dexterous hand system carries the tool to the position of the hole to be inserted.

[0016] Optionally, in S4, when determining the relationship between the tool and the screw, the following step is further included:

[0017] Combine template image and real-time image for auxiliary judgment.

[0018] Optionally, when determining the twisting relationship, the method further includes:

[0019] Combine template image and real-time image for auxiliary judgment.

[0020] Optionally, during the initial insertion process, constant force compliance control is performed.

[0021] Optionally, it also includes:

[0022] The force collected by the force sensor is compared with the safety threshold in real time. If the force exceeds the safety threshold, the process returns to S2.

[0023] Optionally, the robotic arm has 6 active degrees of freedom, including 3 translations and 3 rotations. The dexterous hand includes five fingers, and the fingers have joints for grasping electric tools. The tools include electric drills, and the camera is an RGBD camera.

[0024] Compared with the prior art, the present invention has at least the following beneficial effects:

[0025] The advantages of the present invention compared with the prior art are:

[0026] (1) The present invention adopts a two-stage search in the process of inserting holes in the robot arm and the dexterous hand system. Compared with the existing method, it can solve the assembly obstruction problem caused by the difficulty in aligning special shapes such as hexagon sockets;

[0027] (2) The present invention takes into account the difference between dexterous hand screwing and ordinary mechanism screwing during the screwing process. Compared with the existing method, it can adapt to the situation where the wrist force application position and the screwing position are offset, thereby preventing obstruction during the assembly process;

[0028] (3) This method takes into account the safety during the insertion process and can identify and adjust for abnormal situations such as slipping compared to existing methods;

[0029] (4) This method comprehensively considers the plug-in and screw-in processes and can achieve stability in the transition switching between the two processes compared to the existing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1It is a flow chart of the robot-dexterous hand system force control jack and screwing method based on translation-rotation two-stage search of the present invention;

[0032] Figure 2 Configuration images of dexterous hand grasping and twisting generated for an embodiment of the present invention;

[0033] Figure 3 The Lissajous curve image used in the translation search of the embodiment of the present invention;

[0034] Figure 4 An image of a mechanical arm and a dexterous hand performing a jack insertion in an embodiment of the present invention;

[0035] Figure 5 This is an image of a robotic arm and a dexterous hand performing twisting in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0037] In the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance; unless otherwise specified or explained, the term "multiple" refers to two or more; the terms "connected" and "fixed" should be understood in a broad sense, for example, "connected" can be a fixed connection, a detachable connection, an integral connection, or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0038] In the description of this specification, it should be understood that the directional words such as "upper" and "lower" described in the embodiments of the present invention are described at the angles shown in the drawings and should not be understood as limiting the embodiments of the present invention. In addition, in the context, it should also be understood that when it is mentioned that an element is connected to another element "upper" or "lower", it can not only be directly connected to another element "upper" or "lower", but also indirectly connected to another element "upper" or "lower" through an intermediate element.

[0039] like Figure 1 As shown, an embodiment of the present invention provides a robot-dexterous hand system force control jack and screwing method based on translation-rotation two-level search, including:

[0040] S1, generating a grasping and screwing configuration based on a model of a dexterous hand of a robot-dexterous hand system and a tool for screwing a screw; wherein the robot-dexterous hand system comprises a robotic arm and a dexterous hand fixed at the end of the robotic arm, and the dexterous hand is fixed with a tool;

[0041] S2, using a camera to collect the screw posture at the hole where the screw is to be inserted, and moving the robot-dexterous hand system to the position of the hole to be inserted according to the screw posture;

[0042] S3, the robot-dexterous hand system carries the tool and moves toward the screw in a direction perpendicular to the screw surface to perform preliminary insertion;

[0043] S4, during the initial plugging process, the contact force collected by the force sensor on the robot-dexterous hand system is judged. When the collected contact force reaches the expected force, the movement of the tool in the direction perpendicular to the surface of the screw is recorded, and the relationship between the tool and the screw is judged based on the movement. The relationship between the tool and the screw includes the tool deviating from the screw, the tool inserted into the screw hole, and the tool abutting the screw surface. If the tool deviates from the screw, return to S2. If the tool is inserted into the screw hole, a tightening operation is performed. If the tool abuts the screw surface, the mobile robot-dexterous hand system searches for the plugging hole, inserts the tool into the screw hole, and performs a tightening operation.

[0044] For S1, the configuration generation result in this embodiment is as follows Figure 2 As shown, the specific process is as follows:

[0045] (1) A five-finger dexterous hand was used, and its simulation model was imported into the Mujoco engine in the form of URDF. The relative posture of the dexterous hand and the power tool and the angles of each joint were manually adjusted on the simulation platform. The grasping simulation was performed in Mujoco to adjust the grasping tool to a reasonable configuration without dropping it.

[0046] (2) In a physical environment, based on the configuration obtained in step (1), a gripping test is performed to optimize the gripping configuration. A twisting configuration test of the opening tool is also performed to analyze the stability of the dexterous hand during the twisting process and optimize a reasonable twisting configuration.

[0047] For S2, in this embodiment, an RGBD camera and a posture estimation method are used to measure the screw posture T at the screw hole to be inserted. s , define the direction perpendicular to the screw surface towards the inside of the screw as the z-axis, and its vector as az.

[0048] For S3, in this embodiment, after moving to the position to be inserted, the mechanical arm force based on the z-axis direction is used to control the insertion. The specific process is as follows:

[0049] (1) Calculate the position T of the socket to be inserted ps =T sT z T c , T s is the screw position obtained by S2, T z To adjust the posture, the rotation part is the unit matrix, and the translation part has only a non-zero component in the z-axis, which means moving the tool to a position along the screw axis and at a certain distance from the screw surface. c The transformation relationship between the screw posture and the end posture of the robot arm in the socket state is obtained by calculating the three-dimensional model of the screw, the power tool, the dexterous hand and the end of the robot arm. In this embodiment, the UR5e robot is used, and the MoveL instruction is used to move to the posture to be inserted into the socket.

[0050] (2) The mechanical arm force control based on the z-axis direction of the screw is performed, and the movement in other directions remains unchanged. This embodiment adopts constant force compliance control along the z-axis direction of the screw, and its control method is as follows:

[0051] (2.1) The robotic arm is in the position to be inserted into the socket. The force / torque information of the current UR5e robot end wrist force sensor is obtained and the current information is recorded as the reference force / torque. To compensate for the gravity of the dexterous hand and the power tool, the real-time force in the subsequent movement process is the force of the sensor data minus the reference force T z ;

[0052] (2.2) Based on the transformation relationship in (1), the real-time force of the force sensor in the same direction as the z-axis of the screw is obtained as F Z , set the expected force in this direction to F d .

[0053] (2.3) Adopt constant force compliance control method based on admittance control:

[0054] (2.3.1) Calculate the expected input force of the virtual system as F u =F d +k p F e +k i ∫F e dt, where F e =F d -F Z is the force error. k p and k i are proportional gain and integral gain;

[0055] (2.3.2) Based on the admittance control rate Converted into discrete form, that is Where M, B, and K represent virtual inertia, damping, and stiffness, respectively;

[0056] (2.3.3) Based on step (2.3.2) and The output of the admittance control along the z-axis direction of the screw can be obtained

[0057] (2.4) Based on the output in step (2.2), the movement amount is transformed into the base coordinate system of the robot arm, and the ServoL instruction is used for real-time control to complete the real-time adjustment during the insertion process.

[0058] Regarding the determination of the relationship between the tool and the screw in S4, the specific determination method is as follows:

[0059] (1) When the contact force F Z First time reaching F d When , record the movement amount t0 of the robot arm along the z-axis direction at this time, and determine whether it is greater than the threshold t1. If it is greater than the threshold, it means that the jack is out of alignment, and execute S2;

[0060] (2) If the movement amount t0 is between the thresholds t2 and t3, it means that the tool is directly inserted into the screw hole, and the insertion task is judged to be completed, and screwing is directly performed;

[0061] (3) If the movement amount t0 is less than the threshold t2, it means that the tool is on the surface of the screw and has not entered the screw hole, and it is determined that a search is needed.

[0062] In some embodiments of the present invention, in S4, the relationship between the tool and the screw further includes that the tool does not completely enter the screw hole, and the mobile robot-dexterous hand system searches for the socket, including:

[0063] The robot-dexterous hand system carries a tool to search for a socket based on the Lissajous curve translation. The relationship between the tool and the screw is determined according to the amount of movement in the direction perpendicular to the screw surface during the translation search. If the tool deviates from the screw, it returns to S2 and searches again by changing the direction of the Lissajous curve when re-translating the socket. If the tool is inserted into the screw hole, a tightening operation is performed. If the tool does not fully enter the screw hole, a rotation search for the socket is performed.

[0064] When performing a rotational search for the socket, the robot-dexterous hand system carries the tool and moves within a preset angle range to search. The relationship between the tool and the screw is judged based on the amount of movement in a direction perpendicular to the screw surface during the rotational search. If the tool deviates from the screw, it returns to S2 and when re-translationally searching for the socket, the direction of the Lisa curve is changed to search again. If the tool is inserted into the screw hole, the tightening operation is performed.

[0065] For translation search, the translation process is as follows:

[0066] (1) Calculate the direction vectors of the x-axis and y-axis of the screw in the robot-dexterous hand system coordinate system;

[0067] (2) Based on the direction vector calculated in step (1), the three-dimensional coordinates of each point in the Lissajous curve are generated. Figure 3 As shown, the calculation method is as follows:

[0068] Δx=a·sin(b x ·Δt)

[0069]

[0070] Where a is the search range, b x , b y and Determine the shape of the Lissajous curve;

[0071] (3) ServoL instructions are used to control the robot movement, wherein the movement along the z-axis of the screw still uses the value calculated in real time in S3, and the movement along the x-axis and y-axis of the screw uses the value calculated in real time in step (2).

[0072] For translation search, the judgment process is as follows:

[0073] Contact status judgment:

[0074] (1) Real-time recording of the movement of the robot arm along the z-axis t r With contact force F Z First time reaching F d The difference t of the movement amount t0 at that time is determined to determine whether it is greater than a threshold value t4. If it is greater than this value, it indicates that the jack is out of alignment, and S2 is executed.

[0075] (2) If the amount of exercise t r If the value is between the threshold values ​​t5 and t6, it means that the tool is directly inserted into the screw hole, and the insertion task is judged to be completed, and screwing is directly performed;

[0076] (3) If the amount of exercise t r If it is between the threshold values ​​t7 and t8, it means that the tool is inserted into the screw surface to a certain extent, but due to the obstruction of the hexagonal shape, the hexagonal ball head of the tool head has not completely entered the screw hole, and a rotation search is required.

[0077] For rotation search, the rotation process is as follows:

[0078] (1) Establish a tool end axis endpoint coordinate system, where the z-axis is outward from the tool axis, the x-axis is obtained by projecting the x-axis of the tool coordinate system onto a plane perpendicular to the z-axis, and the y-axis is obtained by cross product using the right-hand rule;

[0079] (2) calculating the pose of the current electric tool end-axis end coordinate system relative to the robot arm end coordinate system based on the real-time pose tracking result of the electric tool;

[0080] (3) Planning the range and speed of the rotation search. In this embodiment, a uniform motion search is used within a range of ±35°, and the desired posture is discretely sampled;

[0081] (4) calculating the search path of the end point of the robot arm based on the desired posture calculated in step (3) and the transformation relationship obtained in step (2);

[0082] (5) ServoL instructions are used to control the robot movement, where the movement along the z-axis of the screw still uses the value calculated in real time in S3, and the movement in other directions uses the values ​​calculated based on step (4).

[0083] For rotation search, the judgment process is as follows:

[0084] (1) After the real-time recording search is turned on, the relative movement of the robot along the z-axis t rr , determine whether it is greater than the threshold t9. If it is greater than this value, it means that the jack is out of place, and execute S2.

[0085] (2) If the amount of exercise t rr At the threshold t 10 to 11 If the tool is inserted into the screw hole, the insertion task is completed and screwing can be performed directly.

[0086] The complete jack insertion process in this embodiment is as follows Figure 4 As shown, during the process of deviating from the execution of S2, the robot arm moves upward along the z-axis of the screw until it is out of the screw range, and then returns to S2 and starts again.

[0087] like Figure 5 As shown, in some embodiments of the present invention, in S4, the tightening operation includes:

[0088] The index finger of the dexterous hand is used to control the screwing of the tool, and the index finger adopts the force control mode; the other fingers adopt the position control mode to maintain the stability during the screwing process.

[0089] The dexterous hand moves to the screwing configuration generated by S1. In this embodiment, the index finger of the dexterous hand is used to control the screwing of the tool, and the index finger adopts the force control mode; the other fingers adopt position control to maintain stability during the screwing process; during the screwing process, the robotic arm adopts the same control method as S3 to ensure following during the screwing process.

[0090] In some embodiments of the present invention, in S4, during the tightening operation, it is determined whether the screw is tightened by:

[0091] The screwing relationship is judged according to the movement amount in the direction perpendicular to the screw surface during the rotation search process. The screwing relationship includes screwing deviation and screwing success. If screwing deviation occurs, it returns to S2. If screwing is successful, the robot-dexterous hand system carries the tool to the position to be inserted into the hole.

[0092] The tightening judgment is as follows:

[0093] (1) Real-time recording of the relative motion of the robot along the z-axis after the screw is turned on. rx , determine whether it is greater than the threshold t 10 If it is greater than this value, it means that the screw is out of alignment, and it is necessary to stop the task and execute S2.

[0094] (2) If the screw is expected to be screwed a certain distance, the movement t rx At the threshold t 11 to 12 If the screwing is successful, the task is completed.

[0095] (3) If the screw is expected to be fully tightened, the movement amount t rx In a period of time h The change of Δt rx At the threshold t 13 to 14 Between, and the amount of movement t rx At the threshold t 15 to 16 If the screwing is successful, the task is completed.

[0096] (4) Check whether the screwing time is greater than a threshold. If so, it indicates an abnormal state. After manual confirmation, execute S2.

[0097] In some embodiments of the present invention, in S4, when determining the relationship between the tool and the screw, it also includes:

[0098] Combine template image and real-time image for auxiliary judgment.

[0099] In some embodiments of the present invention, when determining the screwing relationship, the method further includes:

[0100] Combine template image and real-time image for auxiliary judgment.

[0101] In the first test, the images of the above three states are collected as template images. The real-time image is compared with the template image based on the SSIM method, and the comparison value is used to assist in judgment.

[0102] In some embodiments of the present invention, constant force compliance control is performed during the initial insertion process.

[0103] In some embodiments of the present invention, it further includes:

[0104] The force collected by the force sensor is compared with the safety threshold in real time. If the safety threshold is exceeded, the process returns to S2.

[0105] In some embodiments of the present invention, the robotic arm has 6 active degrees of freedom, including 3 translations and 3 rotations. The dexterous hand includes five fingers. The fingers have joints to enable grasping of electric tools. The tools include an electric drill. The camera is an RGBD camera.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A robot-dexterous hand system force control jack and screwing method based on translation-rotation two-level search, characterized in that: include: S1, generating a grasping and screwing configuration based on a model of a dexterous hand of a robot-dexterous hand system and a tool for screwing a screw; wherein the robot-dexterous hand system comprises a mechanical arm and a dexterous hand fixed at the end of the mechanical arm, and the dexterous hand is fixed with a tool; S2, using a camera to collect the screw posture at the hole where the screw is to be inserted, and moving the robot-dexterous hand system to the position of the hole to be inserted according to the screw posture; S3, the robot-dexterous hand system carries the tool and moves toward the screw in a direction perpendicular to the surface of the screw to perform preliminary insertion; S4, during the initial jacking process, a judgment is made based on the contact force collected by the force sensor on the robot-dexterous hand system. When the collected contact force reaches the expected force, the movement of the tool in a direction perpendicular to the surface of the screw is recorded, and the relationship between the tool and the screw is judged based on the movement. The relationship between the tool and the screw includes the tool deviating from the screw, the tool being inserted into the screw hole, and the tool abutting the screw surface. If the tool deviates from the screw, return to S2. If the tool is inserted into the screw hole, a tightening operation is performed. If the tool abuts the screw surface, the robot-dexterous hand system is moved to search for the jack, and then the tool is inserted into the screw hole and a tightening operation is performed.

2. The method according to claim 1, characterized in that In S4, the relationship between the tool and the screw further includes that the tool does not completely enter the screw hole, and the moving the robot-dexterous hand system to search for the socket includes: The robot-dexterous hand system carries a tool to search for a socket based on the Lissajous curve translation, and judges the relationship between the tool and the screw according to the amount of movement in the direction perpendicular to the screw surface during the translation search. If the tool deviates from the screw, it returns to S2, and when re-translationally searching for the socket, the direction of the Lissajous curve is changed to search again. If the tool is inserted into the screw hole, a tightening operation is performed. If the tool does not completely enter the screw hole, a rotation search for the socket is performed; When performing a rotational search for a socket, the robot-dexterous hand system carries the tool and moves within a preset angle range to search. The relationship between the tool and the screw is determined based on the amount of movement in a direction perpendicular to the screw surface during the rotational search. If the tool deviates from the screw, it returns to S2 and when re-translationally searching for the socket, the direction of the Lisa curve is changed to search again. If the tool is inserted into the screw hole, the tightening operation is performed.

3. The method according to claim 1, characterized in that In S4, the tightening operation includes: The index finger of the dexterous hand is used to control the screwing of the tool, and the index finger adopts the force control mode; the other fingers adopt the position control mode to maintain the stability during the screwing process.

4. The method according to claim 1, characterized in that: In S4, during the tightening operation, whether the screw is tightened is determined by: The screwing relationship is judged according to the amount of movement in the direction perpendicular to the screw surface during the rotation search process. The screwing relationship includes deviation and successful screwing. If the screwing is deviation, it returns to S2. If the screwing is successful, the robot-dexterous hand system carries the tool to the position of the hole to be inserted.

5. The method according to claim 1, characterized in that In S4, when judging the relationship between the tool and the screw, it also includes: Combine template image and real-time image for auxiliary judgment.

6. The method according to claim 4, characterized in that When judging the twisting relationship, the method further includes: Combine template image and real-time image for auxiliary judgment.

7. The method according to claim 1, characterized in that During the initial insertion process, constant force and smooth control are performed.

8. The method according to claim 1, characterized in that Also includes: The force collected by the force sensor is compared with the safety threshold in real time. If the force exceeds the safety threshold, the process returns to S2.

9. The method according to claim 1, characterized in that: The robotic arm has 6 active degrees of freedom, including 3 translations and 3 rotations. The dexterous hand includes five fingers, and the fingers have joints to enable grasping of electric tools. The tools include an electric drill. The camera is an RGBD camera.

Images