Robot control method, device, equipment, medium and program product

Abstract

The invention provides a robot control method, device and equipment, a medium and a program product, and can be applied to the technical field of robot control. The method comprises the steps that a calibration point located on a target object, a center point of a robot tail end flange and an initial tip point of an instrument assembled by the robot are obtained, the robot tail end flange is rigidly connected with the instrument through an adapter, and the instrument is used for forming a notch in the target object or executing operation in the target object; in response to a received target tip point of the instrument selected by the target user, determining a target center point of the robot tail end flange based on the target tip point, the calibration point, the center point of the robot tail end flange and the initial tip point; generating a first control instruction based on the target center point of the tail end flange; and the first control instruction is sent to an execution end, so that the execution end drives a joint motor of the robot to move the center point of the tail end flange to the target center point.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and more specifically to a robot control method, apparatus, device, medium, and program product. Background Technology

[0002] In related technologies, the dexterous hand of a general-purpose robot is generally used to hold laparoscopic instruments to perform surgery on patients. However, the contact gap between the dexterous hand and the handle of the laparoscopic instrument and the flexible deformation limit the motion constraint accuracy of the instrument rod around the preset surgical position, making it difficult to stably meet the positioning requirements of minimally invasive surgery. Summary of the Invention

[0003] In view of the above problems, this application provides robot control methods, apparatus, devices, media and program products.

[0004] According to a first aspect of this application, a robot control method is provided, comprising: acquiring a calibration point located on a target object, a center point of a robot end flange, and an initial tip point of a device assembled on the robot, wherein the robot end flange and the device are rigidly connected via an adapter, and the device is used to form a cut on the target object or perform an operation within the target object; in response to receiving a target tip point of the device selected by a target user, determining a target center point of the robot end flange based on the target tip point, the calibration point, the center point of the robot end flange, and the initial tip point; generating a first control command based on the target center point of the end flange; and sending the first control command to an actuator, such that the actuator drives a joint motor of the robot to move the center point of the end flange to the target center point.

[0005] According to an embodiment of this application, determining the target center point of the robot end flange based on the target tip point, the calibration point, the center point of the robot end flange, and the initial tip point includes: determining the direction vector of the device based on the target tip point and the calibration point; determining the length of the device based on the center point of the robot end flange and the initial tip point; and determining the target center point of the robot end flange based on the length of the device and the direction vector.

[0006] According to an embodiment of this application, the aforementioned instrument includes a cutting instrument, and the method further includes: in response to receiving force data sent by the execution terminal, decomposing the force data into lateral shear force data and axial force data, wherein the axial force data characterizes the insertion resistance encountered by the cutting instrument when cutting the target object, and the lateral shear force data is the shear force by which the cutting instrument tears the target object; when the lateral shear force data is greater than or equal to a preset threshold, acquiring the cutting displacement and relative velocity deviation between the cutting instrument and the cut of the target object, wherein the calibration point is located at the center of the cut; generating a second control command based on the cutting displacement and the relative velocity deviation, and sending the second control command to the execution terminal to cause the cutting instrument to return to the calibration point.

[0007] According to an embodiment of this application, the above-mentioned generation of the second control command based on the cutting displacement and the relative velocity deviation includes: generating a virtual restoring force based on the cutting displacement, the relative velocity deviation, preset damping, and preset stiffness; using a Jacobian matrix, inversely mapping the virtual restoring force into the joint torque of the robot joint motor; and generating the second control command based on the joint torque.

[0008] According to an embodiment of this application, the device further includes a clamping device, and the adapter includes a rotation drive module and an opening and closing drive module. The rotation drive module is used to control the rotation of the clamping device, and the opening and closing drive module is used to control the opening and closing of the clamping device.

[0009] According to an embodiment of this application, the method further includes: obtaining the target rotation angle of the rotation drive module and the target opening / closing angle of the opening / closing drive module; determining the parasitic displacement corresponding to the target rotation angle from a preset mapping table, wherein the parasitic displacement represents the opening / closing amount of the opening / closing drive module caused by the rotation of the rotation drive module; determining a feedforward compensation amount based on the parasitic displacement; compensating the opening / closing angle of the opening / closing drive module using the feedforward compensation amount to generate a compensation instruction; and sending the compensation instruction to the execution terminal so that the opening / closing angle of the opening / closing drive module reaches the target opening / closing angle.

[0010] A second aspect of this application provides a robot control device, comprising: an acquisition module for acquiring a calibration point located on a target object, a center point of the robot end flange, and an initial tip point of a device assembled on the robot, wherein the robot end flange and the device are rigidly connected via an adapter, and the device is used to form a cut on the target object or perform an operation within the target object; a determination module for determining a target center point of the robot end flange based on the target tip point, the calibration point, the center point of the robot end flange, and the initial tip point in response to receiving a target tip point of the device selected by a target user; a generation module for generating a first control command based on the target center point of the end flange; and a sending module for sending the first control command to an execution end, such that the execution end drives the joint motor of the robot to move the center point of the end flange to the target center point.

[0011] A third aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0012] A fourth aspect of this application also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0013] The fifth aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0014] According to the embodiments of this application, by rigidly connecting the robot end flange to the instrument via an adapter, the flexible joints in the mechanical transmission chain are eliminated, improving the system rigidity and the positioning accuracy of the instrument end. This further simplifies the system's motion model. In the process of calculating the robot's motion position, only the target tip point, calibration point, center point of the end flange, and initial tip point are needed to determine the target center point of the robot end flange, reducing the amount of computation and thus alleviating the hardware computational burden on the operator. At the same time, due to the reduced amount of computation, the first control command can be generated and sent to the execution end for response more quickly, at least partially improving the response efficiency of the execution end, thereby improving the user experience of the operator. Attached Figure Description

[0015] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0016] Figure 1This illustration schematically depicts an application scenario of a robot control method and apparatus according to embodiments of this application.

[0017] Figure 2 schematically illustrates a flowchart of a robot control method according to an embodiment of this application;

[0018] Figure 3 A schematic diagram of an actuator assembly device according to an embodiment of this application is shown.

[0019] Figure 4 A schematic diagram illustrating the structure of a robot control device according to an embodiment of this application is shown; and

[0020] Figure 5 A block diagram schematically illustrates an electronic device suitable for implementing a robot control method according to an embodiment of this application. Detailed Implementation

[0021] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0022] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0023] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0024] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0025] Minimally invasive surgery requires surgical instruments to move around the incision point on the body surface to avoid tearing patient tissue. While general-purpose humanoid robots have a human-like dual-arm structure and are suitable for working in unmodified environments, they lack a dedicated kinematic model for minimally invasive surgery. If the dexterous hand of a humanoid robot is used directly to operate manual instruments, there are problems such as large contact gaps in the handles and insufficient rigidity, making it impossible to ensure that the instrument rods move strictly around the incision point on the body surface, and it is difficult to transmit precise high-frequency force feedback.

[0026] In view of this, embodiments of this application provide a robot control method, comprising: acquiring a calibration point located on a target object, a center point of a robot end flange, and an initial tip point of an instrument assembled on the robot, wherein the robot end flange and the instrument are rigidly connected via an adapter, and the instrument is used to form a cut on the target object or perform an operation within the target object; in response to receiving a target tip point of the instrument selected by a target user, determining a target center point of the robot end flange based on the target tip point, the calibration point, the center point of the robot end flange, and the initial tip point; generating a first control command based on the target center point of the end flange; and sending the first control command to an actuator, such that the actuator drives the joint motors of the robot to move the center point of the end flange to the target center point.

[0027] Figure 1 The illustration shows an application scenario of the robot control method and apparatus according to embodiments of this application.

[0028] like Figure 1 As shown, application scenario 100 according to this embodiment may include a head-mounted display 101, an operation terminal 102, an execution terminal 103, and a network 104. The network 104 is used as a medium to provide a communication link between the head-mounted display 101, the operation terminal 102, and the execution terminal 103. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0029] The head-mounted display 101 can receive stereoscopic video streams sent from the humanoid robot at the actuator 103, providing immersive feedback to the operator.

[0030] The operating terminal 102 includes a main hand controller and a foot pedal. The main hand controller is the core interface for the operator to interact with the execution terminal 103. The operator commands the execution terminal 103 to complete actions by operating the main hand controller.

[0031] The actuator 103 is a general-purpose dual-arm humanoid robot. Both arms of the robot have had their original dexterous end effectors removed and are connected to the instrument via flanges and adapters. The aforementioned instrument can be, for example, a standard laparoscopic instrument, which can be directly driven by the adapter.

[0032] It should be noted that the robot control method provided in this application embodiment can generally be executed by the operating terminal 102. Accordingly, the robot control device provided in this application embodiment can generally be set in the operating terminal 102.

[0033] It should be understood that Figure 1 The number of headsets, operating terminals, networks, and execution terminals shown in the diagram is merely illustrative. Depending on implementation requirements, any number of headsets, operating terminals, networks, and execution terminals can be included.

[0034] The following will be based on Figure 1 The described scene, through Figures 2-3 The robot control method of the application embodiments will be described in detail.

[0035] Figure 2 A flowchart illustrating a robot control method according to an embodiment of this application is shown schematically.

[0036] like Figure 2 As shown, the robot control method of this embodiment includes operations S210 to S240.

[0037] In operation S210, the calibration point on the target object, the center point of the robot end flange, and the initial tip point of the instrument assembled on the robot are obtained.

[0038] The robot's end flange is rigidly connected to the instrument via an adapter. The instrument is used to make cuts on the target object or to perform operations within the target object.

[0039] The initial tip point is the initial position of the instrument's farthest working point. For example, when the instrument is a scalpel, the initial tip point is the blade tip. The target object is the object to be operated on or manipulated, and the calibration point is a reference point on the target object that is manually selected and has a clear spatial position.

[0040] Figure 3 A schematic diagram of an actuator assembly device according to an embodiment of this application is shown.

[0041] like Figure 3 As shown, the actuator 103 includes an end flange 301, an adapter 302, and a device 303 (for demonstration purposes). Figure 3 (The device in the image has been enlarged.) It can be seen that one end of the adapter 302 is fixed to the end joint of the humanoid robot arm via a standard mechanical interface bolt, replacing the original dexterous hand. Meanwhile, the other end of the adapter 302 is designed with a quick-locking interface to secure the device's rod.

[0042] In operation S220, in response to receiving the target tip point of the instrument selected by the target user, the target center point of the robot end flange is determined based on the target tip point, calibration point, center point of the robot end flange, and initial tip point.

[0043] According to the embodiments of this application, since the instrument and the flange are rigidly connected, the position of the instrument's tip relative to the center point of the robot's end flange remains unchanged regardless of how the spatial position of the instrument's tip changes (assuming an ideal state). Therefore, when the target user selects the target tip point (i.e., the position that the instrument tip is about to reach), the geometric dimensions (length) of the instrument itself can be calculated using the initial tip point and the center point of the robot's end flange. Then, the target center point of the robot's end flange can be determined by the length and direction between the target tip point and the calibration point.

[0044] In operation S230, the first control command is generated based on the target center point of the end flange.

[0045] In operation S240, the first control command is sent to the actuator so that the actuator drives the robot's joint motors to move the center point of the end flange to the target center point.

[0046] According to embodiments of this application, the operating end and the executing end can be interconnected via a high-speed communication link based on the SSH (Secure Shell) protocol. The operating end can be equipped with a policy server, and the first control command can be, for example, a high-frequency action sequence. The executing end can autonomously execute short-sequence predicted actions or execute preset tasks (such as standardized tasks like knot tying) when communication latency is high.

[0047] According to the embodiments of this application, the two arms of the aforementioned execution-end robot can perform cooperative operations (e.g., the two arms can pass a ring to each other). During the process of moving the center point of the end flange to the target center point, the elbow and shoulder postures of the execution-end robot can be automatically adjusted by using redundant degree of freedom zero-space projection technology, thereby avoiding the two arms from colliding with each other or with the torso. Furthermore, the rigid connection enables the operation accuracy of the execution end to reach the sub-millimeter level, which is suitable for operations such as fine fitting with high precision requirements.

[0048] According to the embodiments of this application, by rigidly connecting the robot end flange to the instrument via an adapter, the flexible joints in the mechanical transmission chain are eliminated, improving the system rigidity and the positioning accuracy of the instrument end. This further simplifies the system's motion model. In the process of calculating the robot's motion position, only the target tip point, calibration point, center point of the end flange, and initial tip point are needed to determine the target center point of the robot end flange, reducing the amount of computation and thus alleviating the hardware computational burden on the operator. At the same time, due to the reduced amount of computation, the first control command can be generated and sent to the execution end for response more quickly, at least partially improving the response efficiency of the execution end, thereby improving the user experience of the operator.

[0049] According to an embodiment of this application, the above-mentioned determination of the target center point of the robot end flange based on the target tip point, calibration point, center point of the robot end flange, and initial tip point includes: determining the direction vector of the instrument based on the target tip point and calibration point; determining the length of the instrument based on the center point of the robot end flange and initial tip point; and determining the target center point of the robot end flange based on the length of the instrument and the direction vector.

[0050] According to the embodiments of this application, since the robot end flange is directly connected to the instrument via an adapter, the kinematic model is simplified and optimized. Therefore, the center point of the robot end flange, the adapter, and the instrument can be considered as a rigid body linkage model to calculate the target center point of the robot end flange. Since the instrument rod at the actuator end must always pass through the calibration point, under the direct-connection rigid body model, this physical requirement is transformed into a spatial three-point collinear geometric constraint. That is, the vector formed by the end flange center point and the calibration point, the flange calibration vector, and the vector formed by the calibration point and the target tip point are collinear and coplanar. Therefore, under this collinear constraint, the depth of the instrument tip within the target object... ,in, To delve deeper into the length, For the target tip, The calibration point is the distance from the center point of the end flange to the outer edge of the calibration point. Forming a linear complementary relationship ( ),in, For the outer length, For the length of the instrument, As a calibration point, The center point of the end flange.

[0051] The direction vector of the instrument can be calculated using the following formula (1).

[0052] (1)

[0053] in, Let be the direction vector of the instrument. For the target tip, This is the calibration point.

[0054] According to the embodiments of this application, the direction vector of the instrument is the absolute spatial pose that the instrument rod must maintain. After the spatial straight line is determined, the target center point of the robot end flange can be reversed from the target tip point to the back along the opposite direction of the direction vector of the instrument, based on the length of the instrument.

[0055] The target center point of the end flange can be calculated using the following formula (2).

[0056] (2)

[0057] in, The target center point of the robot's end flange. For the target tip, For the length of the instrument, Let be the direction vector of the instrument.

[0058] According to an embodiment of this application, the target center point of the robot end flange and the direction vector of the instrument (constraining pitch and yaw angles) can be further sent to the execution end as the first control command of a 5-DOF Cartesian axis with axial constraints. The execution end can use the remaining degrees of freedom (rotation around the tool axis) to avoid obstacles or optimize joint limits through the underlying inverse kinematics solver, provided that the axial space constraints are satisfied. The rotation of the instrument is completely executed independently by the adapter.

[0059] According to the embodiments of this application, the spatial attitude of the device is determined by the target tip point and the calibration point, and the inherent length of the device is determined based on the center point of the flange and the initial tip point. In the process of motion control of the actuator, only the direction vector and the length of the device are needed to determine the target center point of the robot end flange, which reduces the amount of calculation and thus reduces the hardware calculation burden of the operating end.

[0060] According to an embodiment of this application, the aforementioned instrument includes a cutting instrument, and the method further includes: in response to receiving force data sent by the execution end, decomposing the force data into lateral shear force data and axial force data, wherein the axial force data characterizes the insertion resistance encountered by the cutting instrument when cutting the target object, and the lateral shear force data is the shear force by which the cutting instrument tears the target object; if the lateral shear force data is greater than or equal to a preset threshold, obtaining the cutting displacement and relative velocity deviation between the cutting instrument and the cut of the target object, wherein the calibration point is located at the center of the cut; generating a second control command based on the cutting displacement and relative velocity deviation, and sending the second control command to the execution end to cause the cutting instrument to return to the calibration point.

[0061] According to an embodiment of this application, the above-mentioned generation of the second control command based on cutting displacement and relative velocity deviation includes: generating a virtual restoring force based on cutting displacement, relative velocity deviation, preset damping and preset stiffness; using the Jacobian matrix to inversely map the virtual restoring force into the joint torque of the robot joint motor; and generating the second control command based on the joint torque.

[0062] According to the embodiments of this application, in order to overcome the limitation of "excessive rigidity and easy tissue tearing" when faced with external physical interference, this application integrates dynamic compensation and impedance control layers below the kinematic planning layer. The whole-body nonlinear rigid body dynamic equation of the actuator robot connected by the flange, adapter and instrument can be expressed by the following formula (3).

[0063] (3)

[0064] in, For the joint angles of the actuator robot, For the joint speed of the actuator robot, For the joint acceleration of the actuator robot, The inertia matrix of the actuator robot reflects the inertial effect of mass distribution during joint movement. The Coriolis force and centrifugal force matrix of the actuator robot reflects the nonlinear coupling force during high-speed compound motion of the arm. This is a gravity compensation vector used to counteract the downward pressure caused by the weight of the robotic arm and its end effector. This refers to the internal control torque (i.e., the actual drive control torque output by the motors of each joint of the robot). External disturbance torque refers to the equivalent disturbance resistance torque applied by the external environment (such as collisions, the resistance of the target object, etc.) and converted to the joint end of the robot motor.

[0065] In the above formula (3), the left side of the equation represents the nonlinear physical resistance of the robot itself at the execution end, which can be used to control the internal torque. Set as Substituting the internal control torque into formula (3), the inertia matrix, Coriolis force and centrifugal force matrix, and gravity compensation vector on both sides of formula (3) are canceled out, making the dynamic response of the actuator approximate as a linear second-order system with tracking error as input. This compensation mechanism can reduce the influence of the nonlinearity of the robotic arm dynamics on trajectory tracking and maintain high tracking accuracy even at high speeds. Therefore, there is no need to install a six-dimensional force sensor at the actuator; it is only necessary to use a nonlinear disturbance observer to monitor the residual between the current feedback of the bottom joint and the desired dynamic model in real time, thereby accurately estimating the force data of the joint layer.

[0066] According to the embodiments of this application, the Jacobian matrix can be used for transpose operation to map the force data (external disturbance torque) to the spatial Cartesian force borne by the instrument rod at the incision calibration point. This force is then decomposed into axial force data along the instrument rod direction and lateral shear force data perpendicular to the instrument rod direction. When the lateral shear force data is detected to be greater than a preset threshold, it is determined that the instrument rod may be subjected to lateral compression, posing a risk of deviating from the calibration point. At this time, the actuator will immediately switch from "normal rigid mode" to "compliant safety mode". In "compliant safety mode", a virtual restoring force can be generated using the following formula (4).

[0067] (4)

[0068] in, For virtual restoring force, To preset virtual stiffness, To preset virtual damping, For cutting displacement, This represents the relative velocity deviation.

[0069] The virtual restoring force always points towards the calibration point. Furthermore, the virtual restoring force can be mapped to the joint torque of the robot's joint motor through the Jacobian matrix and superimposed on the internal control torque. This enables the robotic arm to generate compliant movement when it receives external disturbances, while maintaining the positioning constraint of the lever relative to the calibration point and reducing the risk of damage to the target object due to rigid resistance.

[0070] According to the embodiments of this application, by receiving the force data from the actuator in real time and decomposing the force data to obtain lateral shear force and axial force data, a second control command is generated in a timely manner when the lateral shear force data is greater than or equal to a preset threshold, so that the cutting instrument returns to the calibration point. This enables the actuator to have the ability to sense and respond to external disturbances. When an accidental collision occurs, the actuator exhibits compliance rather than rigid resistance, which can reduce the risk of tearing of the cut surface of the target object caused by external force and improve operational safety.

[0071] According to an embodiment of this application, the above-mentioned device further includes a clamping device, and the adapter includes a rotation drive module and an opening and closing drive module. The rotation drive module is used to control the rotation of the clamping device, and the opening and closing drive module is used to control the opening and closing of the clamping device.

[0072] According to embodiments of this application, the device can retain only the core rod (including the outer shell tube and push-pull wire). The adapter can employ a coaxial dual-path servo direct-drive mechanism to rigidly connect with the device's drive shaft. The aforementioned rotation drive module and opening / closing drive module are parallel to each other. The aforementioned rotation drive module can use a hollow torque motor in conjunction with a high-precision harmonic reducer. The hollow shaft of the motor is directly and rigidly locked to the outer sleeve of the device via a chuck. The rotation of the motor directly drives the device rod to achieve 360° unlimited rotation. The aforementioned opening / closing drive module can use a micro servo motor to drive a micro ball screw, converting the motor's rotational motion into linear push-pull motion. The screw slider is rigidly connected to the push-pull wire (used to control the opening and closing of the end jaws) inside the device via a micro thrust bearing. For example, with a screw lead of 1mm and a motor rotation of 1°, the theoretical linear propulsion resolution is approximately 2.7μm.

[0073] According to embodiments of this application, the robot at the execution end can also perform other auxiliary or nursing tasks, and can be used in surgical mode simply by replacing the end adapter, thereby improving the overall utilization efficiency of the equipment.

[0074] According to an embodiment of this application, the above method further includes: obtaining the target rotation angle of the rotation drive module and the target opening / closing angle of the opening / closing drive module; determining the parasitic displacement corresponding to the target rotation angle from a preset mapping table, wherein the parasitic displacement represents the opening / closing amount of the opening / closing drive module caused by the rotation of the rotation drive module; determining the feedforward compensation amount based on the parasitic displacement; compensating the opening / closing angle of the opening / closing drive module using the feedforward compensation amount to generate a compensation instruction; and sending the compensation instruction to the execution end so that the opening / closing angle of the opening / closing drive module reaches the target opening / closing angle.

[0075] In a compact coaxial mechanism, if the motor of the opening and closing drive module is fixed on the rotation shaft of the rotation drive module, the end moment of inertia will be greatly increased. If the motor of the opening and closing drive module is fixed on the external support, when the instrument rotates, the internal push and pull wires will generate parasitic displacement due to relative torsion (i.e., the coupled disturbance of "rotation causing accidental opening and closing of the jaws").

[0076] To this end, this application constructs a preset mapping table by statistically analyzing the mapping relationship between the rotation angle and the parasitic linear displacement of the push-pull wire. Thus, when the instrument needs to rotate and open / close, the corresponding parasitic displacement is calculated in real time in each control cycle, thereby determining the feedforward compensation amount and then determining the actual opening / closing angle that needs to be sent to the opening / closing drive module.

[0077] The actual opening and closing angle can be calculated using the following formula (5).

[0078] (5)

[0079] in, This refers to the actual opening and closing angle. For the target opening angle, This is the feedforward compensation amount.

[0080] According to embodiments of this application, the geometric coupling mapping function between the rotation angle and the parasitic linear displacement of the push-pull wire can also be determined by mechanism kinematic calibration or polynomial fitting, and this application does not limit this.

[0081] According to embodiments of this application, the arm of the actuator robot is primarily responsible for adjusting the insertion depth of the instrument within the target object (Z-axis translation) and the swing angle around the target point (pitch and yaw); while the motor (micro-motion) inside the adapter is dedicated to the rotation of the instrument and the opening and closing of the jaws. The two work together to complete the full end-effector multi-degree-of-freedom operation. That is, through the macro-micro composite configuration, the decoupling of large-scale spatial positioning and fine end-effector operation is achieved.

[0082] For example, when performing the action of "intra-abdominal suturing and knotting", the robot's large arm first moves the suture needle to the surface of the tissue (target object) and adjusts the tilt angle of the needle insertion. Then, the large arm remains stationary, the opening and closing motor in the adapter drives the jaws to clamp the suture needle, and the rotation motor drives the instrument rod to rotate 360° with extremely high precision, causing the suture needle to penetrate the tissue. This avoids the clumsiness of using a large robotic arm to draw a circle and improves the precision and success rate of suturing.

[0083] According to the embodiments of this application, the rotation drive module and the opening and closing drive module are decoupled by a preset mapping table, which avoids the decoupling accuracy reduction caused by wear and loosening due to the use of complex pure mechanical steel wire pulley groups in related methods. This maintains stable decoupling accuracy throughout the entire life cycle of the execution end, avoids performance degradation due to mechanical wear, and ensures that the rotation and opening and closing actions do not interfere with each other.

[0084] Based on the above-described robot control method, this application also provides a robot control device. The following will be combined with... Figure 4 The device is described in detail.

[0085] Figure 4 A schematic block diagram of a robot control device according to an embodiment of this application is shown.

[0086] like Figure 4 As shown, the robot control device 400 in this embodiment includes an acquisition module 410, a determination module 420, a generation module 430, and a sending module 440.

[0087] The acquisition module 410 is used to acquire the calibration point located on the target object, the center point of the robot end flange, and the initial tip point of the instrument assembled on the robot. The robot end flange and the instrument are rigidly connected via an adapter. The instrument is used to form a cut on the target object or perform an operation within the target object. In one embodiment, the acquisition module 410 can be used to perform the operation S210 described above, which will not be repeated here.

[0088] The determining module 420 is configured to, in response to receiving the target tip point of the instrument selected by the target user, determine the target center point of the robot end flange based on the target tip point, the calibration point, the center point of the robot end flange, and the initial tip point. In one embodiment, the determining module 420 may be used to perform the operation S220 described above, which will not be repeated here.

[0089] The generation module 430 is used to generate a first control command based on the target center point of the aforementioned end flange. In one embodiment, the generation module 430 can be used to execute the operation S230 described above, which will not be repeated here.

[0090] The sending module 440 is used to send the first control command to the execution end, so that the execution end drives the joint motor of the robot to move the center point of the end flange to the target center point. In one embodiment, the sending module 440 can be used to perform the operation S240 described above, which will not be repeated here.

[0091] According to an embodiment of this application, the determination module 420 includes: a vector determination submodule, used to determine the direction vector of the device based on the target tip point and the calibration point; a length determination submodule, used to determine the length of the device based on the center point of the robot end flange and the initial tip point; and a center point determination submodule, used to determine the target center point of the robot end flange based on the length of the device and the direction vector.

[0092] According to an embodiment of this application, the aforementioned instrument includes a cutting instrument, and the aforementioned device further includes: a decomposition module, configured to decompose the force data into lateral shear force data and axial force data in response to receiving force data sent by the aforementioned execution terminal, wherein the axial force data characterizes the insertion resistance encountered by the cutting instrument when cutting the target object, and the lateral shear force data is the shear force by which the cutting instrument tears the target object; a displacement acquisition module, configured to acquire the cutting displacement and relative velocity deviation between the cutting instrument and the cut of the target object when the lateral shear force data is greater than or equal to a preset threshold, wherein the calibration point is located at the center of the cut; and a second sending module, configured to generate a second control command based on the cutting displacement and the relative velocity deviation, and send the second control command to the aforementioned execution terminal to cause the cutting instrument to return to the calibration point.

[0093] According to an embodiment of this application, the second sending module includes: a restoring force generation submodule, used to generate a virtual restoring force based on the cutting displacement, the relative velocity deviation, the preset damping, and the preset stiffness; an inverse mapping submodule, used to inversely map the virtual restoring force into the joint torque of the robot joint motor using a Jacobian matrix; and a second instruction generation submodule, used to generate the second control instruction based on the joint torque.

[0094] According to an embodiment of this application, the device further includes a clamping device, and the adapter includes a rotation drive module and an opening and closing drive module. The rotation drive module is used to control the rotation of the clamping device, and the opening and closing drive module is used to control the opening and closing of the clamping device.

[0095] According to an embodiment of this application, the above-mentioned device further includes: a target opening / closing angle acquisition module, used to acquire the target rotation angle of the rotation drive module and the target opening / closing angle of the opening / closing drive module; a parasitic displacement determination module, used to determine the parasitic displacement corresponding to the target rotation angle from a preset mapping table, wherein the parasitic displacement represents the opening / closing amount of the opening / closing drive module caused by the rotation of the rotation drive module; a compensation amount determination module, used to determine a feedforward compensation amount based on the parasitic displacement amount; a compensation module, used to compensate the opening / closing angle of the opening / closing drive module using the feedforward compensation amount, and generate a compensation instruction; and a compensation instruction sending module, used to send the compensation instruction to the execution end so that the opening / closing angle of the opening / closing drive module reaches the target opening / closing angle.

[0096] According to embodiments of this application, any multiple modules among the acquisition module 410, determination module 420, generation module 430, and transmission module 440 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the acquisition module 410, determination module 420, generation module 430, and transmission module 440 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), programmable logic array (PLA), system-on-a-chip, system-on-a-substrate, system-on-package, application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any appropriate combination of any of these three implementation methods. Alternatively, at least one of the acquisition module 410, determination module 420, generation module 430, and transmission module 440 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0097] Figure 5 A block diagram schematically illustrates an electronic device suitable for implementing a robot control method according to an embodiment of this application.

[0098] like Figure 5 As shown, an electronic device 500 according to an embodiment of this application includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage portion 508 into a random access memory (RAM) 503. The processor 501 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 501 may also include onboard memory for caching purposes. The processor 501 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0099] RAM 503 stores various programs and data required for the operation of electronic device 500. Processor 501, ROM 502, and RAM 503 are interconnected via bus 504. Processor 501 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 502 and / or RAM 503. It should be noted that the programs may also be stored in one or more memories other than ROM 502 and RAM 503. Processor 501 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0100] According to embodiments of this application, the electronic device 500 may further include an input / output (I / O) interface 505, which is also connected to a bus 504. The electronic device 500 may also include one or more of the following components connected to the input / output (I / O) interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 510 as needed so that computer programs read from it can be installed into the storage section 508 as needed.

[0101] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0102] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 502 and / or RAM 503 and / or one or more memories other than ROM 502 and RAM 503 described above.

[0103] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the robot control method provided in the embodiments of this application.

[0104] When the computer program is executed by the processor 501, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0105] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 509, and / or installed from a removable medium 511. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0106] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by processor 501, it performs the functions defined in the system of this application embodiment. According to embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0107] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0110] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The application does not depart from its scope, and those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A robot control method, characterized in that, The method is applied to the operating terminal, and the method includes: The calibration point located on the target object, the center point of the robot end flange, and the initial tip point of the instrument assembled on the robot are obtained. The robot end flange and the instrument are rigidly connected via an adapter. The instrument is used to form a cut on the target object or to perform an operation within the target object. In response to receiving the target tip point of the instrument selected by the target user, the target center point of the robot end flange is determined based on the target tip point, the calibration point, the center point of the robot end flange, and the initial tip point; A first control command is generated based on the target center point of the end flange. The first control command is sent to the execution end, so that the execution end drives the joint motor of the robot to move the center point of the end flange to the target center point.

2. The method according to claim 1, characterized in that, Determining the target center point of the robot end flange based on the target tip point, the calibration point, the center point of the robot end flange, and the initial tip point includes: The orientation vector of the instrument is determined based on the target tip and the calibration point; The length of the instrument is determined based on the center point of the robot's end flange and the initial tip point; The target center point of the robot end flange is determined based on the length of the instrument and the direction vector.

3. The method according to claim 1, characterized in that, The instrument includes a cutting instrument, and the method further includes: In response to receiving the force data sent by the execution end, the force data is decomposed into lateral shear force data and axial force data. The axial force data characterizes the insertion resistance encountered by the cutting instrument when cutting the target object, and the lateral shear force data is the shear force that causes the cutting instrument to tear the target object. When the lateral shear force data is greater than or equal to a preset threshold, the cutting displacement and relative velocity deviation between the cutting instrument and the cut of the target object are obtained, and the calibration point is located at the center of the cut; A second control command is generated based on the cutting displacement and the relative speed deviation, and the second control command is sent to the execution end so that the cutting instrument returns to the calibration point.

4. The method according to claim 3, characterized in that, The generation of the second control command based on the cutting displacement and the relative velocity deviation includes: A virtual restoring force is generated based on the cutting displacement, the relative velocity deviation, the preset damping, and the preset stiffness; Using the Jacobian matrix, the virtual restoring force is inversely mapped to the joint torque of the robot's joint motor; The second control command is generated based on the joint torque.

5. The method according to claim 1, characterized in that, The device also includes a clamping device, and the adapter includes a rotation drive module and an opening and closing drive module. The rotation drive module is used to control the rotation of the clamping device, and the opening and closing drive module is used to control the opening and closing of the clamping device.

6. The method according to claim 5, characterized in that, The method further includes: Obtain the target rotation angle of the rotation drive module and the target opening / closing angle of the opening / closing drive module; The parasitic displacement corresponding to the target rotation angle is determined from the preset mapping table. The parasitic displacement represents the opening and closing amount of the opening and closing drive module caused by the rotation of the rotation drive module. The feedforward compensation amount is determined based on the parasitic displacement. The opening and closing angle of the opening and closing drive module is compensated using the feedforward compensation amount, and a compensation command is generated. The compensation command is sent to the execution end so that the opening and closing angle of the opening and closing drive module reaches the target opening and closing angle.

7. A robot control device, characterized in that, The device includes: The acquisition module is used to acquire the calibration point located on the target object, the center point of the robot end flange, and the initial tip point of the instrument assembled on the robot. The robot end flange and the instrument are rigidly connected via an adapter. The instrument is used to form a cut on the target object or perform an operation within the target object. A determination module is configured to, in response to receiving a target tip point of the instrument selected by a target user, determine the target center point of the robot end flange based on the target tip point, the calibration point, the center point of the robot end flange, and the initial tip point; The generation module is used to generate a first control command based on the target center point of the end flange; The sending module is used to send the first control command to the execution end, so that the execution end drives the joint motor of the robot to move the center point of the end flange to the target center point.

8. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 6.

Images