Method for determining external force of deformable robot arm based on pose and robot system

Abstract

The present disclosure relates to the field of mechanics, and discloses a method for determining external force of a deformable robot arm. The deformable robot arm comprises at least one structure bone, a fixed disc and at least one spacer disc, the at least one structure bone passes through the at least one spacer disc and the end is fixedly connected with the fixed disc. The method comprises: obtaining the pose of the end of the deformable robot arm; obtaining the driving information of the at least one structure bone of the deformable robot arm; and determining the external force of the deformable robot arm based on the pose of the end of the deformable robot arm, the driving information of the at least one structure bone and the mechanical model of the deformable robot arm, wherein the mechanical model is based on the distribution of the at least one structure bone on the cross section of the deformable robot arm and the physical properties of the at least one structure bone.

Description

Technical Field

[0001] This disclosure relates to the field of mechanics, and more particularly to a method, computer equipment, and robot system for determining the external forces acting on a deformable robotic arm based on its posture. Background Technology

[0002] Deformable robotic arms can be used in robotic-assisted minimally invasive surgical tasks. During surgery, the operator (e.g., a physician) needs to obtain the external forces acting on the deformable robotic arm (e.g., the external forces generated by the interaction between the deformable robotic arm and human tissue) in order to perform surgical operations based on these external forces.

[0003] Typically, a force sensor can be installed at the end of a deformable robotic arm. The force sensor measures the external force on the deformable robotic arm and feeds the external force back to the operator to guide the operator's surgical procedures.

[0004] However, installing force sensors at the end effector of deformable robotic arms to measure external forces presents at least the following technical challenges: 1) Integration difficulties. Force sensors occupy a large space and require cable transmission. Medical deformable robotic arms and surgical instruments have complex and compact structures, making it difficult to allocate space for force sensor integration during the design process. 2) High cost. Surgical instruments are often discarded or recycled after a few uses, while force sensors are expensive, thus limiting their widespread application in surgical instruments. 3) Poor placement flexibility. Force sensors can only measure external forces at their installation location, making it difficult or impossible to cover external forces across the entire surgical instrument. Therefore, sensor placement must be tailored to the force conditions of the deformable robotic arm. 4) Poor compatibility. Using force sensors can cause issues related to sterilization and electromagnetic compatibility.

[0005] Therefore, there is a need to provide a method for obtaining the external force of a deformable robotic arm that can meet practical needs, in order to assist the operator in surgical operations, improve the accuracy of surgical operations, and reduce the cost of surgical operations. Summary of the Invention

[0006] In some embodiments, this disclosure provides a method for determining the external forces acting on a deformable robotic arm. The deformable robotic arm includes at least one structural bone, a fixed disk, and at least one spacer disk. The at least one structural bone passes through at least one spacer disk and its end is fixedly connected to the fixed disk. The method may include: obtaining the pose of the end effector of the deformable robotic arm; obtaining actuation information of at least one structural bone of the deformable robotic arm; and determining the external forces acting on the deformable robotic arm based on the pose of the end effector, the actuation information of at least one structural bone, and a mechanical model of the deformable robotic arm, wherein the mechanical model is based on the distribution of at least one structural bone of the deformable robotic arm across the cross-section of the deformable robotic arm and the physical properties of at least one structural bone.

[0007] In some embodiments, this disclosure provides a computer device comprising: a memory for storing at least one instruction; and a processor coupled to the memory and configured to execute at least one instruction to perform the method provided in any of the foregoing embodiments for determining external forces on a deformable robotic arm.

[0008] In some embodiments, this disclosure provides a computer-readable storage medium for storing at least one instruction, which, when executed by a computer, causes a robot system to implement a method for determining external forces on a deformable robotic arm according to any embodiment of this disclosure.

[0009] In some embodiments, this disclosure provides a robotic system comprising: at least one deformable robotic arm including at least one structural bone, a fixed disk, and at least one spacer disk, the at least one structural bone passing through the at least one spacer disk and having its end fixedly connected to the fixed disk; and a control device configured to perform the method of any embodiment of this disclosure to determine external forces acting on the deformable robotic arm. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments of this disclosure will be briefly introduced below. The accompanying drawings described below only show some embodiments of this disclosure. For those skilled in the art, other embodiments can be obtained based on the content of the embodiments of this disclosure and these drawings without creative effort.

[0011] Figure 1 A flowchart of a method for determining external forces on a deformable robotic arm according to some embodiments of the present disclosure is shown;

[0012] Figure 2 A schematic diagram of the structure of a robot system according to some embodiments of the present disclosure is shown;

[0013] Figure 3A schematic diagram of a deformable robotic arm according to some embodiments of the present disclosure is shown;

[0014] Figure 4 A schematic diagram of a system for measuring the pose of a deformable robotic arm end effector according to some embodiments of the present disclosure is shown.

[0015] Figure 5 A schematic diagram of the deformation of a deformable robotic arm under actuation according to some embodiments of the present disclosure is shown.

[0016] Figure 6 A schematic diagram of the deformation of a deformable robotic arm according to some embodiments of the present disclosure when subjected to driving and external forces is shown.

[0017] Figure 7 A force diagram of a deformable robotic arm according to some embodiments of the present disclosure is shown;

[0018] Figure 8 A schematic diagram of the coordinate system and reference coordinate system of a deformable robotic arm according to some embodiments of the present disclosure is shown. Detailed Implementation

[0019] To make the technical problems solved by this disclosure, the technical solutions adopted, and the technical effects achieved clearer, the technical solutions of the embodiments of this disclosure will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely exemplary embodiments of this disclosure, and not all embodiments.

[0020] In the description of this disclosure, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this disclosure. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0021] In the description disclosed in this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "coupling" should be interpreted broadly. For example, they can refer to fixed connections or detachable connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention disclosure according to the specific circumstances. In this invention disclosure, the end closer to the operator (e.g., a doctor) is defined as the proximal end, proximal or rear end, or rear part, and the end closer to the surgical patient is defined as the distal end, distal end, distal or front end, or front part. Those skilled in the art will understand that the embodiments of this disclosure can be used in medical devices or surgical robots, as well as other non-medical devices.

[0022] In this disclosure, the term "position" refers to the location of an object or part of an object in three-dimensional space (e.g., three translational degrees of freedom can be described using variations in Cartesian X, Y, and Z coordinates, such as three translational degrees of freedom along the Cartesian X, Y, and Z axes, respectively). In this disclosure, the term "attitude" refers to the rotational setting of an object or part of an object (e.g., three rotational degrees of freedom, which can be described using roll, pitch, and yaw). In this disclosure, the term "pose" refers to a combination of the position and attitude of an object or part of an object, which can be described, for example, using six parameters from the six degrees of freedom mentioned above.

[0023] In this disclosure, a reference coordinate system can be understood as a coordinate system capable of describing the pose of an object. Depending on the actual positioning requirements, the reference coordinate system can be selected with the origin of a virtual reference object or the origin of a physical reference object as its origin. In some embodiments, the reference coordinate system can be a world coordinate system, a camera coordinate system, or the operator's own perception coordinate system, etc.

[0024] In this disclosure, an object can be understood as an object or target that needs to be positioned, such as a deformable manipulator or the end effector of a deformable manipulator. The pose of the deformable manipulator or a portion thereof (e.g., the end effector) can refer to the pose of the coordinate system defined by the deformable manipulator or a portion thereof relative to a reference coordinate system.

[0025] Figure 1 A flowchart 100 illustrates a method for determining external forces on a deformable robotic arm according to some embodiments of the present disclosure. Figure 2 A schematic diagram 200 of a robot system according to some embodiments of the present disclosure is shown. Method 100 may be implemented or executed by hardware, software, or firmware. In some embodiments, method 100 may be implemented by a robot system (e.g., Figure 2The robot system 200 shown is executed. In some embodiments, method 100 can be implemented as computer-readable instructions. These instructions can be executed by a general-purpose processor or a special-purpose processor (e.g., [missing information]). Figure 2 The control device 220 shown reads and executes the instructions. For example, the control device for the robot system 200 may include a processor configured to execute method 100. In some embodiments, these instructions may be stored on a computer-readable medium.

[0026] In some embodiments, such as Figure 2 As shown, the robot system 200 may include a master control carriage 210, a surgical carriage 230, and a control device 220. The control device 220 can be communicatively connected to the master control carriage 210 and the surgical carriage 230, for example, via cable or wireless connection, to enable communication between them. The master control carriage 210 includes a master manipulator for remote operation by the operator and a display for showing an image of the operating area. The surgical carriage 230 includes a slave tool for performing surgery, comprising a deformable robotic arm and an end effector disposed at the end of the deformable robotic arm. The control device 220 enables master-slave mapping between the master manipulator in the master control carriage and the slave tool in the surgical carriage, allowing the master manipulator to control the motion of the slave tool. In some embodiments, the slave tool is configured to enter the operating area through a sheath and contact and exert force on the tissue to be operated on (e.g., human tissue) in the operating area. The sheath can be fixed to the patient's surgical opening (e.g., an incision or natural opening), and the operating area can be the area where the surgery is performed. The end effectors of the surgical instruments can include, but are not limited to, surgical forceps, electrosurgical units, and electro-hooks. Those skilled in the art will understand that the main control carriage 210 and the operating carriage 230 can adopt other structures or forms, such as bases, supports, or buildings.

[0027] Figure 3 A schematic diagram of a deformable robotic arm 300 according to some embodiments of the present disclosure is shown. See also Figure 3The deformable robotic arm includes one or more structural bones 310, a fixed plate 330, and at least one spacer plate 320. The structural bone 310 passes through at least one spacer plate 320 and its end is fixedly connected to the fixed plate 330. The spacer plates 320 and the fixed plate 330 are spaced apart axially from the structural bone 310. The spacer plate 320 has a through hole 3201 through which the structural bone 310 passes, and the fixed plate 330 has a fixing hole 3301 for fixing the structural bone 310. The end of the structural bone 310 can be connected to the fixed plate 330, and the proximal end of the structural bone 310 can be connected to a drive mechanism. When the structural bone 310 is driven by the drive mechanism, the structural bone 310 can move along the through hole 3201. When the deformable robotic arm is subjected to an external force, the structural bone 310 can move along the through hole 3201. In some embodiments, at least one spacer plate 320 can form a continuous structure, such as a bellows. In some embodiments, the deformable robotic arm 300 may be covered with a covering layer or a cover.

[0028] In some embodiments, the number of structural bones 310 is one or more, and they are uniformly or non-uniformly distributed on the cross-section of the spacer disc, for example, located at the center or distributed circumferentially. In some embodiments, the number of spacer discs 320 is one or more. One or more through holes 3201 may be formed on the cross-section of the spacer disc 320, and the shape of the through holes 3201 matches or substantially matches the shape of the cross-section of the structural bones, so that the structural bones 310 can pass through the through holes 3201. In some embodiments, the number of through holes 3201 formed on the cross-section of the spacer disc 320 is the same as the number of structural bones 310, so that one structural bone 310 can pass through each through hole 3201. In some embodiments, the cross-sectional shape of the spacer disc 320 is circular, the through holes 3201 on the spacer disc are circular holes, and the cross-section of the structural bones 310 is circular. In other embodiments, the cross-sectional shape of the spacer disc 320 is rectangular, the through holes on the spacer disc are polygonal holes, and the cross-section of the structural bones is polygonal, etc.

[0029] Figure 1 A flowchart illustrating a method 100 for determining external forces acting on a deformable robotic arm according to some embodiments of the present disclosure is shown. Figure 1 As shown, in step 101, the pose of the end effector of the deformable robotic arm can be obtained. In some embodiments, signals can be received from sensors disposed at the end effector of the deformable robotic arm, and the pose of the end effector of the deformable robotic arm can be calculated based on the received signals. For example, one or more electromagnetic or infrared sensors can be disposed on the end effector of the deformable robotic arm, and the pose of the end effector of the deformable robotic arm can be calculated based on electromagnetic signals received from the electromagnetic sensors or infrared signals received from the infrared sensors.

[0030] In some embodiments, an image of the deformable robotic arm can be obtained and analyzed to determine the pose of the end effector of the deformable robotic arm. For example, a positioning marker can be set on the end effector of the deformable robotic arm, and the pose of the end effector of the deformable robotic arm can be obtained by processing the image of the positioning marker.

[0031] Figure 4 A schematic diagram of a system 400 for measuring the pose of a deformable robotic arm end effector according to some embodiments of the present disclosure is shown. Figure 4 As shown, system 400 may include a control device 420, a signal acquisition device 440, and at least one deformable robotic arm 450. The signal acquisition device 440 may be communicatively connected to the control device 420. In some embodiments, the control device 420 may be communicatively connected to a drive device of at least one robotic arm 450 for controlling the movement of at least one robotic arm 450 to adjust the pose of at least one robotic arm 450, coordinate with each other, etc. In some embodiments, the robotic arm 450 may be, for example... Figure 3 The deformable robotic arm shown is implemented. In some embodiments, at least one robotic arm 450 may include a robotic arm end effector 451 at its distal or end end. A control device 420 can control the movement of at least one robotic arm 450 to move the robotic arm end effector 451 to a desired position and orientation. Those skilled in the art will understand that the robotic arm control system 400 can be applied to surgical robot systems, such as laparoscopic surgical robot systems. For example, a surgical actuator 460 may be disposed at the distal end of the robotic arm end effector 451, such as… Figure 4 As shown. It should be understood that the robotic arm control system 400 can also be applied to dedicated or general-purpose robotic systems in other fields (e.g., manufacturing, machinery, etc.).

[0032] In some embodiments, a sensor, such as an electromagnetic sensor or an infrared sensor, may be provided on the end effector 451 of the robotic arm. Figure 4 As shown, the end effector 451 of the robotic arm is within the working range 441 of the signal acquisition device 440. The signal acquisition device 440 can acquire signals from sensors, such as electromagnetic signals from an electromagnetic sensor or infrared signals from an infrared sensor. The control device 420 can calculate the pose of the end effector based on the signals acquired by the signal acquisition device 440.

[0033] In some embodiments, a positioning mark may be provided on the end effector 451 of the robotic arm, such as... Figure 4As shown. The signal acquisition device 440 may include, but is not limited to, a dual-lens image acquisition device or a single-lens image acquisition device, such as a binocular or monocular camera. The end effector 451 of the robotic arm is within the field of view 441 of the signal acquisition device 440, which can be used to acquire positioning images. The positioning images may include part or all of the image of the robotic arm 450. For example, the signal acquisition device 440 can be used to acquire images of the end effector 451 of the robotic arm. The control device 420 can receive the positioning images from the signal acquisition device 440 and process the positioning images. For example, the control device 420 can identify positioning markers located on the robotic arm 450 in the positioning images and determine the pose of the robotic arm 450.

[0034] like Figure 4 As shown, the end effector 451 or actuator 460 of the robotic arm 450 may be subjected to an external force F. The control device 420 can also determine the external force F of the deformable robotic arm based on the pose of the robotic arm 450.

[0035] In some embodiments, images of the deformable robotic arm can also be acquired from medical imaging devices such as computed tomography (CT), magnetic resonance imaging (MRI), and stereo vision, and the pose of the end effector of the deformable robotic arm can be determined based on the analysis of the images.

[0036] In step 103, the driving information of the structural bone of the deformable robotic arm can be obtained. In some embodiments, the proximal end of the structural bone can be connected to a driving mechanism, which can be used to drive the structural bone to move within the through-hole of the spacer disc. For example, the operator of the surgical robot issues a control command to control the movement of the deformable robotic arm (e.g., the control command includes driving information), and the driving mechanism responds to the control command by pushing and / or pulling the structural bone to achieve the movement of the structural bone, thereby meeting the operator's operational needs for the deformable robotic arm.

[0037] In some embodiments, the structural skeleton can be made of an elastic material, possessing a certain degree of flexibility. For example, the material of the structural skeleton can be a superelastic alloy, a gas / liquid cavity, a shape memory alloy, a polymer structural material, such as a nickel-titanium alloy. Based on the elastic properties of the deformable robotic arm, when the structural skeleton is subjected to external forces and / or the driving action of a driving mechanism (e.g., push-pull action), the deformable robotic arm can deform. For example, the shape change of the deformable robotic arm can manifest as bending deformation, telescopic deformation, or torsional deformation.

[0038] See Figure 5 , Figure 5 A schematic diagram showing the deformation of a deformable robotic arm 500 according to some embodiments of the present disclosure when driven. For example... Figure 5As shown, when the structural bone 310 in the deformable robotic arm is pushed or pulled, the deformable robotic arm deforms. For example, when the structural bone in the deformable robotic arm is driven by a drive mechanism (not shown), the structural bone 310 can move along the through hole 3201 on the spacer 320. The drive mechanism may include pushing (e.g., Figure 5 The direction of pushing) and pulling (e.g., Figure 5 (The direction of pulling). For example, when the drive mechanism applies a downward pulling force to the structural bone, the structural bone deforms, for example, the length of the structural bone becomes greater than its length in the static state. When the drive mechanism applies an upward pushing force to the structural bone, the structural bone deforms, for example, the length of the structural bone becomes less than its length in the static state.

[0039] Figure 6 A schematic diagram of the deformation of a deformable robotic arm 600 according to some embodiments of the present disclosure under a driving force and an external force F is shown. In some embodiments, when the deformable robotic arm is subjected to an external force F, the structural bone 110 can move along a through-hole 3201 on the spacer disc 320. The external force F can be a force acting at the end position of the deformable robotic arm or a force acting at other positions of the deformable robotic arm. For example, the external force can include the force generated when the deformable robotic arm comes into contact with the tissue being operated on during a surgical procedure.

[0040] In step 105, the external forces acting on the deformable manipulator can be determined based on the end-effector pose, the driving information of the structural bones, and the mechanical model of the deformable manipulator. The mechanical model is based on the distribution of the structural bones of the deformable manipulator across its cross-section and the physical properties of the structural bones.

[0041] A deformable robotic arm may include an elastic structural skeleton. When subjected to external forces or driving forces from a drive mechanism, the pose of the deformable robotic arm (e.g., end-effector pose) will change. The pose of the deformable robotic arm reflects the driving information and external forces acting on it; therefore, given the driving information and the pose of the deformable robotic arm, the external forces acting on it can be calculated. Figure 3 As shown, the deformable robotic arm is in a static state, and its end effector pose does not change. Figure 5 as well as Figure 6 As shown, when the deformable robotic arm is subjected to a driving force or an external force, the pose of the deformable robotic arm changes.

[0042] In some embodiments, a mechanical model of the deformable manipulator can be constructed based on the relationship between the pose of the deformable manipulator, the external forces acting on the deformable manipulator, and the driving information of the deformable manipulator. Thus, given the pose changes and driving information of the deformable manipulator, the external forces acting on the deformable manipulator can be calculated based on the constructed mechanical model.

[0043] The mechanical model of a deformable robotic arm can be constructed based on the distribution of the structural bones of the deformable robotic arm across its cross-section and the physical properties of the structural bones. In some embodiments, the distribution of the structural bones across the cross-section of the deformable robotic arm can be represented by a position vector of the structural bones within the cross-section. For example, it could be a position offset vector of the structural bones relative to the center reference line of the deformable robotic arm. See also Figure 5 The position vector of the j-th structural bone in the cross-section corresponds to r. j In some embodiments, the physical properties of the structural bone can be used to characterize the elastic deformation properties of the structural bone, for example, to characterize the deformation of the deformable manipulator when it is subjected to external forces and / or driving forces.

[0044] In some embodiments, the pose of points on the deformable robotic arm can be represented based on a coordinate system of points on the central axis of the deformable robotic arm. For example, the central axis can be a virtual centerline extending axially along the deformable robotic arm (e.g., Figure 3 (340 in the text). In some embodiments, the deformable robot coordinate system can be a local dynamic coordinate system, with each axial point corresponding to a deformable robot coordinate system. Figure 8 A schematic diagram of coordinate system 800 according to some embodiments of this disclosure is shown. See also Figure 8 The starting point can be the proximal end of the deformable robot arm (e.g., s = 0), and the ending point can be the distal end of the deformable robot arm (e.g., s = L). A coordinate system for the deformable robot arm is established along the axis of the reference line for each axial point. Figure 8 In the coordinate system of the deformable robotic arm, x is... d y d z d The reference coordinate system is x w y w z w .

[0045] In some embodiments, the deformable robotic arm coordinate system includes a first coordinate direction, a second coordinate direction, and a third coordinate direction. For example, the tangent between the axial point and the central axis can be used as the first coordinate direction, which can be the z-axis of the deformable robotic arm coordinate system. d Axis. The line segment pointing from the axial point to the structural bone can be used as the second coordinate direction. The second coordinate direction can be the x-axis of the deformable robot arm coordinate system. dThe axis. A third coordinate direction can be determined based on the first and second coordinate directions, and this third coordinate direction can be used as the y-axis of the deformable robotic arm. d Axis. A deformable robot arm coordinate system x is constructed based on the first, second, and third coordinate directions, with points along the axis. d y d z d ,like Figure 8 As shown.

[0046] In some embodiments, the calculated pose of the deformable manipulator can be determined based on the calculated pose of the central axis of the deformable manipulator and the distribution of the structural bones on the cross-section of the deformable manipulator. The pose of the deformable manipulator can be represented by formulas (1) and (2).

[0047] p j =p+Rr j (1)

[0048] R j =R (2)

[0049] In formulas (1) and (2), p j Here, p is the position of the j-th structural bone in the deformable robotic arm, and r is the position of the central axis of the deformable robotic arm. j R represents the distribution of the j-th structural bone on the cross-section of the deformable robotic arm. j R is the pose of the j-th structural bone in the deformable manipulator, and R is the pose of the central axis of the deformable manipulator.

[0050] In some embodiments, the pose of the central axis can be the pose of the central axis in a reference coordinate system. The change of the coordinate system of the central axis of the robotic arm along the axial direction can be represented by formulas (3) and (4).

[0051] p′=Rv (3)

[0052] R′=Ru^ (4)

[0053] In formulas (3) and (4), p is the position of the central axis in the reference coordinate system, R is the orientation of the central axis in the reference coordinate system, ()′ represents the derivative with respect to length s, and ()^ represents the operation of converting a vector into a skew-symmetric matrix: v is linear velocity, u is angular velocity, and R represents rotation. R can transform the described coordinate system; for example, R can transform the coordinate system of a deformable robotic arm to a reference coordinate system. The coordinate system of the deformable robotic arm can be the coordinate system corresponding to the axial points on the central axis, and it can change as the axial points change. The reference coordinate system can be the coordinate system of the host control vehicle, the world coordinate system, the camera coordinate system, etc., and can be an invariant coordinate system.

[0054] In some embodiments, the mechanical model may include constitutive relations related to the structural skeleton. Constitutive relations can represent the material properties of the structural skeleton, for example, by expressing the material properties through the internal forces and deformations of the structural skeleton.

[0055] In some embodiments, the deformable robotic arm includes at least one structural bone (j is the structural bone number, j = 1, 2, 3…m), and the internal force constitutive relation of the deformable robotic arm can be determined based on the internal force constitutive relation of the structural bone. The internal force constitutive relation of the deformable robotic arm can be determined based on the shear-tensile stiffness matrix of the structural bone.

[0056] In some embodiments, the deformable robotic arm includes a constraint structure (e.g., a spacer disk, a fixed disk, a covering layer, etc.) and a structural skeleton. The internal force constitutive relationship of the deformable robotic arm can be determined based on the internal force constitutive relationship of the constraint structure and the structural skeleton. For example, the internal force constitutive relationship of the deformable robotic arm is given by formula (5).

[0057]

[0058] In formula (5), n all The internal forces of the deformable robotic arm are given by R, which is the rotation matrix, and K. SE K is the shear and tensile stiffness matrix of the constraint structure of the deformable robotic arm. SEj Let v be the shear-tensile stiffness matrix of the j-th structural bone in the deformable manipulator, and v be the linear velocity of the deformable manipulator's pose changing along the arc length of a reference line. For example, the reference line could be the central axis of the deformable manipulator. min This is the linear velocity of the deformable robotic arm in its natural state. For example, v min It can be the linear velocity when there is no external force or driving force, v min =[0 0 1] T .

[0059] In some embodiments, the deformable robotic arm includes multiple structural bones (j is the structural bone number, j = 1, 2, 3…m), and the internal moment constitutive relation of the deformable robotic arm can be determined based on the internal moment constitutive relation of the structural bones. The internal moment constitutive relation of the deformable robotic arm can be determined based on the bending and torsional stiffness matrix of the structural bones.

[0060] In some embodiments, the deformable manipulator includes a constraint structure and a structural skeleton. The internal moment constitutive relation of the deformable manipulator can be determined based on the internal force constitutive relation of the constraint structure and the structural skeleton. For example, the internal moment constitutive relation of the deformable manipulator is given by equation (6).

[0061]

[0062] In formula (6), m all R is the internal torque of the deformable robotic arm, R is the rotation matrix, and K is the internal torque of the deformable robotic arm. BT Let K be the bending and torsional stiffness matrix of the constraint structure of the deformable robotic arm. BTj Let be the bending and torsional stiffness matrix of the j-th structural bone in the deformable manipulator, and u be the angular velocity of the deformable manipulator's pose changing along the arc length of the reference line. min It refers to the angular velocity under natural conditions, such as the angular velocity when there is no external force or driving force. min =0.

[0063] In some embodiments, the mechanical model includes mechanical equilibrium relationships related to the structural skeleton. These mechanical equilibrium relationships include the force equilibrium relationships of the structural skeleton, which include the force equilibrium at various locations along the axial direction of the structural skeleton. In some embodiments, the force equilibrium relationships include the equilibrium between external and internal forces acting on the structural skeleton along the axial direction at the points of force application.

[0064] In some embodiments, the mechanical balance relationship includes the force balance relationship of the deformable robotic arm, which includes the force balance of the structural bones at various points along the axial direction. In some embodiments, the force balance relationship of the deformable robotic arm includes the external force and internal force of the deformable robotic arm at the point of force application along the axial direction being in balance.

[0065] Figure 7 A force diagram of a deformable robotic arm 700 according to some embodiments is shown. See also Figure 7 On the left side, for a structural unit [s,s+Δs] of the deformable robotic arm, the force balance relationship of the structural unit [s,s+Δs] is given by formula (7).

[0066]

[0067] A deformable robotic arm may include a constraint structure (e.g., a spacer disk, a fixed disk, a covering layer, etc.) and a structural skeleton. In formula (7), n(s) represents the internal force of the skeleton of the deformable robotic arm at point s, n(s+Δs) represents the internal force of the constraint structure of the deformable robotic arm at point (s+Δs), where Δs is a small increment, f e (ξ) represents the distributed external force at point ξ in the deformable robotic arm. j (s) represents the internal force on the j-th structural bone at point s, n j (s+Δs) represents the internal force on the j-th structural bone at (s+Δs).

[0068] Based on formula (7), the force balance relationship of the deformable robotic arm is obtained, as shown in formula (8).

[0069] n′ all +f e =0 (8)

[0070] In formula (8), n all The internal forces of the deformable robotic arm are given by ()′, which represents differentiation, f. e This is a distributed external force (such as gravity). In some embodiments, the distributed external force can be ignored.

[0071] In some embodiments, the mechanical equilibrium relationship of the deformable robotic arm includes a torque balance relationship, which includes the axial torque of the structural skeleton. See also Figure 7 On the right side, for a structural unit [s,s+Δs] of the deformable robotic arm, the torque balance relationship of the structural unit [s,s+Δs] is given by formula (9).

[0072]

[0073] In formula (9), m(s) represents the internal torque of the constraint structure of the deformable robot arm at point s, m(s+Δs) represents the internal torque of the constraint structure of the deformable robot arm at point s+Δs, and Δs is a small increment. p(s+Δs) represents the position of the constraint structure of the deformable robot arm at point s+Δs, and n(s+Δs) represents the internal force on the constraint structure of the deformable robot arm at point s+Δs. e (ξ) represents the distributed torque of the constraint structure of the deformable manipulator at ξ (in some embodiments, the distributed torque can be ignored), p(ξ) is the position of the constraint structure of the deformable manipulator at ξ, and f e (ξ) represents the distributed external force at ξ on the constraint structure of the deformable robotic arm. j (s) represents the internal torque of the j-th structural bone at point s, m j (s+Δs) represents the internal torque of the j-th structural bone at s+Δs. j (s+Δs) represents the position of the j-th structural bone at s+Δs, where n j (s+Δs) represents the internal force on the j-th structural bone at s+Δs. j (s) represents the position of the j-th structural bone at s, n j (s) represents the internal force on the j-th structural bone at point s.

[0074] Based on formula (9), the torque balance relationship of the deformable robotic arm at position s is obtained, see formula (10).

[0075]

[0076] In formula (10), m all is the internal torque of the deformable robotic arm, p is the position of the reference line of the deformable robotic arm, and n is the torque of the internal torque of the deformable robotic arm. all It is the internal force of the deformable robotic arm, l e R is the distributed torque of the deformable robotic arm; in some embodiments, the distributed torque can be ignored. R is the rotation matrix of the reference line of the deformable robotic arm, r j It is the distribution of the j-th structural bone on the cross section of the deformable robotic arm, for example, r. j It can be the coordinates of the structural bones on the cross-section of the deformable manipulator, u is the angular velocity of the deformable manipulator's pose changing along the arc length of the reference line, and K is the coordinates of the structural bones on the cross-section of the deformable manipulator. SEj The shear-tensile stiffness matrix of the j-th structural bone. It is linear strain, representing the difference in linear velocity of the pose of the j-th structural bone along the arc length of the reference line before and after deformation. In some embodiments, the moment boundary condition at the end of the deformable manipulator is applied, which includes the sum of the internal moments of the deformable manipulator at the end being zero, see formula (11).

[0077]

[0078] In formula (11), m e is the external torque of the deformable manipulator at position L, m(L) is the internal torque of the deformable manipulator at position L, and R(L) is the rotation matrix of the deformable manipulator at position L.

[0079] In some embodiments, the mechanical model of the deformable manipulator includes the relationship between the axial length variation of the structural bones and the distribution of the structural bones on the cross-section of the deformable manipulator. For example, the axial length variation of each structural bone (j is the structural bone number, j = 1, 2, 3, ..., m) in the deformable manipulator is given by formula (12).

[0080] q′ j =||v+u^r j ||-1 (12)

[0081] In formula (12), q j r is the change in length of the j-th structural bone along the axial direction. j It refers to the distribution of structural bones across the cross-section of a deformable robotic arm, for example, r. j It can be the coordinates of the distribution of the structural bone on the cross-section of the deformable manipulator, v is the linear velocity of the deformable manipulator's pose along the arc length of the reference line, and u is the angular velocity of the deformable manipulator's pose along the arc length of the reference line.

[0082] In some embodiments, the length change q of the structural bone j (j is the structural bone number, j = 1, 2, 3, ..., m) can be related to drive information; for example, drive information can be information about the drive mechanism driving the movement of the structural bone. In some embodiments, the length change of the structural bone can also be related to the deformation of the deformable manipulator. For example, the deformation of the deformable manipulator can be the extension and retraction deformation of its structural bone. In some embodiments, a length boundary condition is applied to the structural bone, which includes the length change q of the structural bone at its end. j (L) equals the length driving quantity q aj With length deformation ε j The sum of these. The driving information for the structural bones includes the length driving amount. For example, the axial length changes of all structural bones in a deformable robotic arm are given by Equation (13).

[0083] q(L)=q a +L all ε (13)

[0084] In formula (13), q(L)=[q1(L)q2(L)...q m (L)] T q represents the length of each structural bone at s = L. a =[q a1 q a2 ... q am ] T L represents the driving length of each bone structure. all Representing the total length of the structural bone, ε = [ε1 ε2 ... ε m ] T This represents the linear strain of expansion and contraction on each structural skeleton. For example, ε j It can be a percentage, L all ε j This represents the stretching deformation of the j-th structural bone.

[0085] In some embodiments, the mechanical model of the deformable manipulator is solved analytically to determine the external forces acting on the manipulator. In some embodiments, the mechanical model of the deformable manipulator is solved using a target-shooting method to determine the external forces acting on the manipulator.

[0086] In some embodiments, solving the mechanical model of the deformable manipulator based on the shooting method to determine the external forces acting on the deformable manipulator may include determining the calculated pose of the end effector of the deformable manipulator based on the mechanical model, and determining the external forces acting on the deformable manipulator based on the calculated pose.

[0087] In some embodiments, the calculated pose of the end effector of the deformable manipulator is determined based on the initial force and initial torque of the deformable manipulator at its initial position, the length deformation and driving information of the structural bone, and the mechanical model of the deformable manipulator. The driving information includes the length driving amount. For example, the initial position can be the proximal position of the deformable manipulator near the driving mechanism (e.g., the proximal fixed position of the deformable manipulator or the position where the sheath extends out), which can be denoted as position 0. Based on the initial force n(0), initial torque m(0), length deformation ε(0) of the structural bone, and driving information q of the deformable manipulator at its initial position... a (0) and the mechanical model of the deformable manipulator, determine the calculated pose of the end effector of the deformable manipulator. For example, the calculated pose of the end effector of the deformable manipulator can be obtained by performing differential calculations along the axis of the deformable manipulator, with the initial position as the starting point and the end position as the ending point.

[0088] In some embodiments, the calculated pose of the deformable manipulator can be based on the calculated pose of the manipulator's central axis. For example, it can be based on the estimated initial force n(0), initial torque m(0), length deformation ε(0) of the structural bone at the initial position, and known actuation information q. a (0), calculate the pose of the deformable robotic arm using formulas (1)-(13).

[0089] In some embodiments, the pose of the central axis can be determined based on the position and orientation of each axial point on the central axis. For example, a deformable robotic arm coordinate system can be constructed for each axial point, such as... Figure 8 As shown, the position and orientation of each axial point are determined based on the coordinate system of each deformable robotic arm.

[0090] In some embodiments, in response to the difference between the calculated pose and the obtained pose meeting an error requirement, the external forces on the deformable manipulator are determined based on the initial forces and initial moments. For example, when the difference between the calculated pose determined based on the initial values ​​of the deformable manipulator at its initial position (e.g., initial forces and initial moments, length deformation of the structural skeleton, and actuation information) and the pose obtained based on the measurement meets an error requirement (e.g., less than the error), it indicates that the initial values ​​at the initial position meet the requirements for calculating the external forces. Therefore, the external forces on the deformable manipulator are determined based on the initial values ​​of the deformable manipulator at its initial position.

[0091] In some embodiments, solving the mechanical model of the deformable manipulator based on the target-shooting method to determine the external forces acting on the manipulator further includes adjusting the initial forces, initial torques, and length deformations in response to the pose difference between the calculated pose and the obtained pose not meeting the error requirements. Based on the adjusted initial forces, initial torques, length deformations, driving information, and the mechanical model of the deformable manipulator, the calculated pose of the end effector of the deformable manipulator is determined until the pose difference between the calculated pose and the obtained pose meets the error requirements. When the pose difference does not meet the error requirements, it indicates that there is a certain gap between the calculated pose calculated based on the initial values ​​and the measured pose, and the initial values ​​need to be adjusted. The initial values ​​are iteratively adjusted, and the calculated pose of the end effector of the deformable manipulator is determined based on the adjusted initial values ​​until the pose difference between the calculated pose and the obtained pose meets the error requirements. When the pose difference determined by the adjusted initial values ​​meets the error requirements, the external forces are calculated. In this way, the accuracy of the initial values ​​is verified by the pose difference, improving the accuracy of the external forces calculated from the initial values.

[0092] In one embodiment, the difference between the calculated pose and the obtained pose is determined by formulas (14) and (15).

[0093] Δp=p(L)-p marker (L) (14)

[0094] Δω=log(R T (L)R marker (L)) ∨ (15)

[0095] In formula (14), Δp is the measurement position p of the deformable robotic arm. marker The position difference between (L) and the calculated position p(L). In formula (15), Δω is the measured posture R of the reactive deformable manipulator. marker The difference in attitude between (L) and the calculated attitude R(L), T To represent the matrix transpose, () ∨ Operators that convert skew-symmetric matrices into vectors:

[0096] In some embodiments, the adjustment amount of the initial value vector can be determined based on the influence matrix of the change of the initial value vector on the boundary condition value vector. The initial value vector includes the initial force, the initial torque, and the length deformation. The initial value vector is adjusted based on the adjustment amount. For example, the initial value vector x(0) can be determined based on the initial force n(0), the initial torque m(0), and the length deformation ε of the deformable manipulator at the initial value. The boundary condition value vector b(L) can be determined based on the length boundary condition of the structural bone, the torque boundary condition and the pose boundary condition of the deformable manipulator at the end. The influence matrix J of the change of the initial value vector x(0) on the boundary condition value vector b(L) is given by formula (16).

[0097]

[0098] In some embodiments, the elements of the influence matrix can be calculated by perturbing the elements of the initial value vector. The elements of the initial value vector are then adjusted based on the elements of the influence matrix. For example, the boundary condition value b(L) cannot be explicitly expressed as an equation about the initial value vector x(0), and J can be obtained numerically. For the i-th column of J, a perturbation can be applied to the i-th element of the initial value vector x(0) to calculate the elements of the influence matrix J, see formula (17).

[0099]

[0100] In formula (17), e i It is a vector where the i-th element is 1 and all other elements are 0. δ is a very small positive number. b is the boundary value when the initial value vector x(0) is perturbed by δ at the i-th element. x(0) (L) is the boundary value when the initial value vector is x(0). Based on formula (17), the elements of the influence matrix J are obtained as J = [J [1] J [2] ... J [n] ], based on the elements J of the influence matrix J = [J [1] J [2] ... J [n] And formula (17) adjusts the elements of the initial value vector x(0), see formula (18),

[0101] x(0)=x(0)-(J T J+λI) -1 J T b(L) (18)

[0102] In formula (18), λ is a positive number. In some embodiments, the initial value vector can be iteratively adjusted until the error of the boundary condition value vector meets the requirements. The boundary condition value vector includes the length of the structural bone at the end being equal to the sum of the length driving amount and the length deformation included in the driving information, the difference between the calculated pose and the obtained pose meeting the error requirements, and the torque balance of the deformable manipulator at the end. For example, based on formula (18), through several iterations, an adjusted initial value vector x(0) can be obtained, making the error of the boundary condition value vector b(L) = 0 sufficiently small. In this way, the initial value of the proximal end is adjusted by the target method, and the external force of the deformable manipulator is calculated based on the adjusted initial value of the proximal end, improving the accuracy of the external force calculation. For example, based on the initial force, the initial torque, and the force balance relationship of the deformable manipulator, the external force at the end of the deformable manipulator is calculated.

[0103] In some embodiments, a deformable robotic arm can be used as a surgical tool. The surgical tool may also include an actuator disposed at the end of the deformable robotic arm. The surgical tool can perform robot-assisted minimally invasive surgical tasks. During surgery, the operator needs to sense the external forces acting on the deformable robotic arm, such as the force exerted by the deformable robotic arm on human tissue, and then guide the operator's surgical procedures based on these external forces. For example, feeding back the external forces acting on the deformable robotic arm to the operator can help the operator correctly apply force, palpate the lesion area, or avoid erroneous operations, thereby improving surgical efficiency and reducing surgical risks.

[0104] In the above embodiments, based on the principle that the pose of the deformable robotic arm is constrained by driving information and external forces, a mechanical model is pre-constructed to describe the relationship between the pose of the deformable robotic arm, the external forces acting on the deformable robotic arm, and the driving information of the deformable robotic arm. In some embodiments, the external forces acting on the deformable robotic arm can be calculated using the mechanical model of the deformable robotic arm based on the acquired pose and driving information. The obtained external forces acting on the deformable robotic arm guide the operator (e.g., a doctor) at the control console to control the movement of the deformable robotic arm and perform surgical operations.

[0105] In some embodiments, the pose of a deformable robotic arm can be measured using a pose sensor or marker, and the external forces acting on the deformable robotic arm can be calculated based on the measured pose and a mechanical model. Compared to using a mechanical sensor to measure the external forces acting on a deformable robotic arm, pose sensors or markers offer advantages such as low cost, simple assembly, small footprint, flexible use, strong applicability, and high practicality. Furthermore, the constructed mechanical model can be reused, achieving the efficient effect of repeated calculations after a single modeling.

[0106] Note that the above are merely exemplary embodiments and technical principles of this disclosure. Those skilled in the art will understand that this disclosure is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this disclosure. Therefore, although this disclosure has been described in detail through the above embodiments, this disclosure is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this disclosure, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for determining external force of a morphable manipulator, the morphable manipulator comprising at least one structural bone, a fixed disk and at least one spacer disk, the at least one structural bone passing through the at least one spacer disk and being fixedly connected with the fixed disk, the morphable manipulator being provided with a positioning mark at the end thereof, the method comprising: obtaining a pose of the end of the morphable manipulator; obtaining driving information of the at least one structural bone of the morphable manipulator; and determining external force of the morphable manipulator based on the pose of the end of the morphable manipulator, the driving information of the at least one structural bone and a mechanical model of the morphable manipulator, wherein the mechanical model is based on distribution of the at least one structural bone on a cross section of the morphable manipulator and physical properties of the at least one structural bone; obtaining the pose of the end of the morphable manipulator comprises: obtaining an image of the morphable manipulator; and identifying the positioning mark in the image to determine the pose of the end of the morphable manipulator; the mechanical model comprises a constitutive relation and a mechanical equilibrium relation related to the at least one structural bone; the constitutive relation comprises an internal force constitutive relation and an internal moment constitutive relation, the internal force constitutive relation being based on a shear tensile stiffness matrix of the at least one structural bone, and the internal moment constitutive relation being based on a bending torsional stiffness matrix of the at least one structural bone; the mechanical equilibrium relation comprises a force equilibrium relation and a moment equilibrium relation of the morphable manipulator, the force equilibrium relation comprising force along an axial direction of the at least one structural bone, and the moment equilibrium relation comprising moment along the axial direction of the at least one structural bone. the force equilibrium relation comprises: external force and internal force of the morphable manipulator along the axial direction at the force are in equilibrium, and internal force at the end is in equilibrium; or external force and internal force of the morphable manipulator along the axial direction at the end are in equilibrium; the moment of the at least one structural bone is determined based on the distribution of the at least one structural bone on the cross section of the morphable manipulator and the shear tensile stiffness matrix of the at least one structural bone; the mechanical model of the morphable manipulator comprises a relation between a length variation amount of the at least one structural bone along the axial direction and the distribution of the at least one structural bone on the cross section of the morphable manipulator; further comprising: applying a length boundary condition of the at least one structural bone, the length boundary condition comprising that a length of the at least one structural bone at the end is equal to a sum of a length driving amount and a length morphing amount, and the driving information of the at least one structural bone comprises the length driving amount; further comprising: applying a moment boundary condition of the morphable manipulator at the end, the moment boundary condition comprising that a sum of internal moments of the morphable manipulator at the end is zero; further comprising: solving the mechanical model of the morphable manipulator based on an analytical method to determine the external force of the morphable manipulator; further comprising: ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ 2. The method of claim 1, wherein, ​ ​ ​ 3. The method of claim 1, wherein, ​ 4. The method of claim 1, wherein, ​ 5. The method of claim 1, wherein, ​ ​ 6. The method of claim 1, wherein, ​ ​ 7. The method according to any one of claims 1 to 6, characterized in that, ​ ​ 8. The method according to any one of claims 1-6, characterized in that, ​ solving a mechanical model of the morphable robotic arm based on a shooting method to determine external forces of the morphable robotic arm.

9. The method of claim 8, wherein, solving a mechanical model of the morphable robotic arm based on a shooting method to determine external forces of the morphable robotic arm includes: determining a calculated pose of an end of the morphable robotic arm based on initial forces and initial moments of the morphable robotic arm at an initial position, length deformation variables of the at least one structural bone, and driving information, wherein the driving information includes length driving variables; and in response to a difference between the calculated pose and the obtained pose satisfying an error requirement, determining the external forces of the morphable robotic arm based on the initial forces and the initial moments.

10. The method of claim 9, wherein, solving a mechanical model of the morphable robotic arm based on a shooting method to determine external forces of the morphable robotic arm further includes: in response to the difference between the calculated pose and the obtained pose not satisfying the error requirement, adjusting the initial forces, the initial moments, and the length deformation variables; and determining a calculated pose of an end of the morphable robotic arm based on the adjusted initial forces, the initial moments, and the length deformation variables, the driving information, and the mechanical model of the morphable robotic arm, until the difference between the calculated pose and the obtained pose satisfies the error requirement.

11. The method of claim 9, wherein, determining the external forces of the morphable robotic arm includes: calculating the external forces of an end of the morphable robotic arm based on the initial forces, the initial moments, and a force balance relationship of the morphable robotic arm.

12. The method of claim 10, wherein, further includes: determining an adjustment of an initial value vector based on an influence matrix of a change of the initial value vector on a boundary condition value vector, wherein the initial value vector includes the initial forces, the initial moments, and the length deformation variables; and adjusting the initial value vector based on the adjustment.

13. The method of claim 12, wherein, further includes: calculating elements of the influence matrix by applying perturbations to elements of the initial value vector; and adjusting elements of the initial value vector based on elements of the influence matrix.

14. The method of claim 12, wherein, further includes: iteratively adjusting the initial value vector until an error of the boundary condition value vector satisfies a requirement, wherein the boundary condition value vector includes that a length of the at least one structural bone at the end is equal to a sum of the length driving variables and the length deformation variables included in the driving information, the difference between the calculated pose and the obtained pose satisfies the error requirement, and a moment balance of the morphable robotic arm at the end is satisfied.

15. A computer device, comprising: a memory configured to store at least one instruction; and a processor coupled to the memory and configured to execute the at least one instruction to perform a method for determining external forces of a morphable robotic arm according to any one of claims 1-14.

16. A computer-readable storage medium configured to store at least one instruction, which, when executed by a computer, causes the computer to perform a method for determining external forces of a morphable robotic arm according to any one of claims 1-14.

17. A robotic system, comprising: at least one morphable robotic arm; and A control device configured to perform the method according to any one of claims 1-14 to determine external forces on the morphable robotic arm.

Images