Control methods for manipulators and surgical robot systems

Abstract

This disclosure relates to the field of robot control, and discloses a control method for a manipulator and a surgical robot system. The control method for the manipulator includes: obtaining the current actual pose of a target portion of the manipulator; obtaining the target pose of the target portion of the manipulator; responding to a first force condition of the target portion of the manipulator, in a first control mode of the manipulator, determining a drive signal for the manipulator based on the current actual pose and the target pose; and responding to a second force condition of the target portion of the manipulator, in a second control mode of the manipulator, obtaining the current theoretical pose of the target portion of the manipulator, and determining the drive signal for the manipulator based on the current theoretical pose and the target pose.

Description

Technical Field

[0001] This disclosure relates to the field of medical devices, and more particularly to a control method for a manipulator and a surgical robot system. Background Technology

[0002] In recent years, many surgical robot systems using robotic arms have been developed for cardiovascular surgery, neurosurgery, and endoscopic surgery, among others. During surgical procedures, the robotic arms need to be controlled to perform surgical actions. Summary of the Invention

[0003] In some embodiments, this disclosure provides a control method for a manipulator, comprising: obtaining a current actual pose of a target portion of the manipulator; obtaining a target pose of the target portion of the manipulator; in response to a first force condition of the target portion of the manipulator, in a first control mode of the manipulator, determining a drive signal of the manipulator based on the current actual pose and the target pose; and in response to a second force condition of the target portion of the manipulator, in a second control mode of the manipulator, obtaining a current theoretical pose of the target portion of the manipulator, and determining the drive signal of the manipulator based on the current theoretical pose and the target pose.

[0004] In some embodiments, this disclosure provides a surgical robot system, including: a surgical tool, the surgical tool including a manipulator arm and an end effector disposed at the distal end of the manipulator arm; and a processor for performing the method of any one of the embodiments of this disclosure.

[0005] In some embodiments, this disclosure provides a computer device including: a memory for storing at least one instruction; and a processor coupled to the memory and configured to execute at least one instruction to perform a method according to any of some embodiments of this disclosure.

[0006] In some embodiments, this disclosure provides a computer-readable storage medium for storing at least one instruction, which, when executed by a computer, causes a robot system to perform a method as described in any of some embodiments of this disclosure. Attached Figure Description

[0007] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments of this disclosure will be briefly introduced below. The accompanying drawings described below only show some embodiments of this disclosure. For those skilled in the art, other embodiments can be obtained based on the content of the embodiments of this disclosure and these drawings without creative effort.

[0008] Figure 1 A flowchart illustrating a control method for an operating arm according to some embodiments of the present disclosure is shown;

[0009] Figure 2 This diagram illustrates a structural block diagram of a surgical robot system according to some embodiments of the present disclosure;

[0010] Figure 3 A schematic diagram of the structure of an operating arm according to some embodiments of the present disclosure is shown;

[0011] Figure 4 A schematic diagram of the structure of an operating arm according to some embodiments of the present disclosure is shown;

[0012] Figure 5 A schematic diagram showing the acquisition of the current actual position of the end effector of a surgical robot system according to some embodiments of the present disclosure;

[0013] Figure 6 A schematic diagram is shown of a positioning tag including multiple pose markers and multiple angle markers according to some embodiments of the present disclosure;

[0014] Figure 7 A schematic diagram showing a positioning label disposed on the periphery of the end of an operating arm and formed into a cylindrical shape according to some embodiments of the present disclosure;

[0015] Figure 8 A schematic diagram showing the end-effector pose of a manipulator obtained by a binocular endoscope-based visual detection algorithm according to some embodiments of the present disclosure;

[0016] Figure 9 A flowchart illustrating a method for determining the force conditions of a target portion of an operating arm according to some embodiments of the present disclosure;

[0017] Figure 10 A flowchart illustrating a method for determining the motion state of an operating arm according to some embodiments of the present disclosure;

[0018] Figure 11 A schematic diagram of the forces acting on an operating arm according to some embodiments of the present disclosure is shown;

[0019] Figure 12 A schematic diagram showing the forces acting on the end of an operating arm according to some embodiments of the present disclosure;

[0020] Figure 13 A logic block diagram of a control method for a manipulator according to some embodiments of the present disclosure is shown;

[0021] Figure 14 A schematic block diagram of a computer device according to some embodiments of the present disclosure is shown;

[0022] Figure 15 A schematic diagram of a surgical robot system according to some embodiments of the present disclosure is shown. Detailed Implementation

[0023] To make the technical problems solved by this disclosure, the technical solutions adopted, and the technical effects achieved clearer, the technical solutions of the embodiments of this disclosure will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely exemplary embodiments of this disclosure, and not all embodiments.

[0024] In the description of this disclosure, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing this disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this disclosure. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this disclosure, it should be noted that unless otherwise expressly specified and limited, the terms "installed," "connected," "coupled," and "coupled" should be interpreted broadly. For example, they can refer to fixed connections or detachable connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this disclosure according to the specific circumstances.

[0025] Those skilled in the art will understand that the embodiments of this disclosure are applicable to mechanical devices (e.g., surgical robots) operating in various environments, including but not limited to those on the surface, underground, underwater, in space, and within living organisms. In this disclosure, the end closer to the operator (e.g., a doctor) is defined as the proximal end, proximal or rear end, or rear portion, and the end closer to the object of work (e.g., a surgical patient) is defined as the distal end, distal or front end, or front portion. In this disclosure, the manipulator arm may include a main body and a distal end portion of the main body, the distal end of which may be fitted with an end effector (e.g., an imaging device, a surgical actuator, etc.). In this disclosure, the target portion of the manipulator arm may include the end effector of the manipulator arm or a portion of the main body. For clarity, in this disclosure, the end of the manipulator refers to the farthest part of the manipulator and its end effector. For example, when an end effector (such as an imaging device, surgical actuator, etc.) is installed on the distal end of the manipulator body, the end of the manipulator may refer to the end effector; when no end effector (such as an imaging device, surgical actuator, etc.) is installed on the distal end of the manipulator body, the end of the manipulator refers to the distal end of the manipulator body.

[0026] In this disclosure, the term "position" refers to the location of an object or a portion of an object in three-dimensional space (e.g., three translational degrees of freedom can be described using variations in Cartesian X, Y, and Z coordinates, such as three translational degrees of freedom along the Cartesian X, Y, and Z axes, respectively). In this disclosure, the term "pose" refers to the rotational setting of an object or a portion of an object (e.g., three rotational degrees of freedom, which can be described using roll, pitch, and yaw). In this disclosure, the term "pose" refers to a combination of the position and pose of an object or a portion of an object, which can be described, for example, using six parameters from the six degrees of freedom mentioned above.

[0027] Some embodiments of this disclosure provide a control method for a manipulator. Figure 1 A flowchart is shown of a control method 100 for a manipulator (hereinafter also referred to as "method 100") according to some embodiments of the present disclosure. Method 100 may be implemented or performed by hardware, software, or firmware. In some embodiments, method 100 may be performed by a surgical robot system (e.g., Figure 2 The surgical robot system 200 shown Figure 5 The surgical robot system 500 shown is... Figure 15 The surgical robot system 1500 shown is executed. In some embodiments, method 100 can be implemented as computer-readable instructions. These instructions can be executed by a general-purpose processor or a special-purpose processor (e.g., Figure 2 The control device 220 shown or Figure 15 The processor 1530 shown reads and executes data. For example, the control unit of a surgical robot system (e.g., Figure 2 The control device 220 shown may include a processor configured to execute method 100. In some embodiments, these instructions may be stored on a computer-readable medium.

[0028] Some embodiments of this disclosure provide a surgical robot system. Figure 2 A structural block diagram of a surgical robot system 200 according to some embodiments of the present disclosure is shown. Figure 2As shown, the surgical robot system 200 may include a master control carriage 210, a surgical carriage 230, and a control device 220. The control device 220 can communicate with the master control carriage 210 and the surgical carriage 230, for example, via cable or wireless connection, to achieve communication between them. The master control carriage 210 includes a master manipulator for remote operation by the operator and a display for showing images of the operating area. The surgical carriage 230 includes surgical tools for performing surgery, including a manipulator arm and end effectors (such as imaging devices, surgical actuators, etc.) located at the distal end of the manipulator arm. The control device 220 enables a master-slave mapping between the master manipulator in the master control carriage 210 and the surgical tools in the surgical carriage 230, allowing the master manipulator to control the motion of the surgical tools. In some embodiments, the surgical cart 230 includes multiple surgical tools configured to enter the operating area through a sheath. At least one surgical tool's end effector (e.g., an imaging device) can acquire images of the object to be operated on (e.g., human tissue) within the operating area, and at least one surgical tool's end effector (e.g., a surgical actuator) can perform surgical actions on the object to be operated on (e.g., human tissue) within the operating area. The sheath can be fixed to the patient's surgical opening (e.g., an incision or natural opening), and the operating area can be the area where the surgery is performed. In some embodiments, the imaging device may include, but is not limited to, a monocular endoscope, a binocular endoscope, etc., and the surgical actuator may include, but is not limited to, surgical forceps, an electrosurgical unit, an electrocautery hook, etc.

[0029] In some embodiments, the manipulator may include a deformable robotic arm, such as a multi-degree-of-freedom robotic arm composed of multiple joints, such as a robotic arm capable of 6 degrees of freedom of motion, or a deformable continuum robotic arm.

[0030] Figure 3 A schematic diagram of a component 300 of an operating arm according to some embodiments of the present disclosure is shown. The operating arm (e.g., Figure 4 The shown operating arm 400 Figure 5 The surgical execution arm 510 and visual guidance arm 520 shown are... Figure 8 The shown operating arm 800 or Figure 15 The illustrated manipulator 1511 may include at least one deformable segment 300. For example... Figure 3 As shown, the deformable segment 300 includes a fixed disk 310 and multiple structural bones 320. A first end of each structural bone 320 is fixedly connected to the fixed disk 310, and a second end is connected to a drive unit (not shown). In some embodiments, the fixed disk 310 may be, but is not limited to, a ring-shaped structure, a disc-shaped structure, etc., and its cross-section may be circular, rectangular, polygonal, or various other shapes.

[0031] In some embodiments, the structural skeleton 320 in the deformable segment 300 can be made of an elastic material, possessing a certain degree of flexibility. For example, the material of the structural skeleton 320 can be a hyperelastic alloy, a gas / liquid cavity, a shape memory alloy, a polymer structural material, such as a nickel-titanium alloy. Based on the elastic properties of the structural skeleton 320, the deformable segment 300 can undergo shape changes when subjected to external forces and / or the driving action of the driving unit (e.g., push-pull action). The shape change of the deformable segment 300 can manifest as bending deformation, stretching deformation, or torsional deformation, etc. For example, the driving unit drives the structural skeleton 320 to bring the segment 300 into a position such as... Figure 2 The bending state is shown. In some embodiments, the second end of the multiple structural bones 320 passes through the base plate 330 and is connected to the driving unit, so that the driving unit drives the structural bones 320 to change the shape of the deformable segment 300. In some embodiments, similar to the fixed plate 210, the base plate 330 may be, but is not limited to, a ring structure, a disc structure, etc., and the cross-section may be a circle, a rectangle, a polygon, etc. In some embodiments, the driving unit may include a linear motion mechanism, a driving segment, or a combination of both. The linear motion mechanism may be connected to the multiple structural bones 320 to push or pull the multiple structural bones 320, thereby driving the segment 300 to bend. The driving segment may include a fixed plate and multiple structural bones, wherein one end of the multiple structural bones is fixedly connected to the fixed plate. The other end of the multiple structural bones of the driving segment is connected to or integrally formed with the multiple structural bones 320, so as to drive the bending of the deformable segment 300 by bending the driving segment.

[0032] In some embodiments, at least one spacer disk 340 is further included between the fixed disk 310 and the base disk 330, and the first ends of the plurality of structural bones 320 pass through the at least one spacer disk 340 and are fixedly connected to the fixed disk 310. Similarly, the drive segment may also include a spacer disk.

[0033] Figure 4 A schematic diagram of the structure of an operating arm 400 according to some embodiments of the present disclosure is shown. For example... Figure 4 As shown, the manipulator 400 is a continuous robotic arm, which may include a distal arm body 410 and an arm body main body 420. The arm body main body 420 may include one or more segments, such as a first segment 4201 and a second segment 4202. In some embodiments, the structures of the first segment 4201 and the second segment 4202 may be consistent with... Figure 3 The shown component 300 is similar. In some implementations, such as... Figure 4As shown, the main body 420 of the arm also includes a first straight rod segment 4203 located between the first segment 4201 and the second segment 4202. The first end of the first straight rod segment 4203 is connected to the base plate of the second segment 4202, and the second end is connected to the fixing plate of the first segment 4201. In some embodiments, such as... Figure 4 As shown, the manipulator body 420 further includes a second straight rod segment 4204, the first end of which is connected to the base plate of the first component 4201. In some embodiments, the distal end portion 410 of the arm body is located at the distal end of the arm body 420. In some embodiments, the distal end portion 410 of the arm body may include a columnar portion located at the distal end of the arm body 420. In some embodiments, an end effector (e.g., an imaging tool, surgical actuator, etc.) 430 may be disposed at the distal end of the distal end portion 410 of the arm body. In some embodiments, the manipulator 400 may be covered with a covering layer or a cover.

[0034] In some embodiments, the manipulator 400 and its constituent segments can be described by a kinematic model. In some embodiments, the structure of each segment can be specifically as follows: Figure 3 The shown component is 300. (As shown in the image) Figure 3 As shown, the base coordinate system The base plate 330 is attached to the i-th (i = 1, 2, 3...) segment, with its origin located at the center of the base plate 330, and the XY plane coincides with the plane of the base plate 330. Pointing from the center of base plate 330 to the first structural bone (the first structural bone can be understood as any one of the multiple structural bones 320 designated as a reference). Curved plane coordinate system. Its origin coincides with the origin of the base coordinate system, and the XY plane coincides with the bending plane. and Coincident. Fixed disk coordinate system. The origin of the fixed disk 310 attached to the i-th segment is located at the center of the fixed disk 310, and the XY plane coincides with the plane of the fixed disk 310. Pointing from the center of the fixed plate 310 to the first structural bone. Curved plane coordinate system. Its origin is located at the center of the fixed disk 310, and the XY plane coincides with the bending plane. and coincide.

[0035] In some embodiments, such as Figure 3 The single segment 300 shown can be described by a kinematic model. The position of the i-th segment end (e.g., in the fixed disk coordinate system {ie}) relative to the base disk coordinate system {ib}. ib p ie ,attitude ib R ie As shown in formulas (1) and (2) below:

[0036]

[0037] ib R ie = ib R i1 i1 R i2 i2 R ie (2)

[0038] Among them, L i The virtual structural skeleton for the i-th segment (e.g., Figure 3 The length of the virtual structural bone 321 shown in the figure; θ i Let be the bending angle of the i-th structural segment, indicating that in the i-th structural segment, about or Rotate to Required rotation angle; δ i Let be the bending direction angle of the i-th segment, and let represent the bending plane and in the i-th segment. The included angle; ib R i1 Let {i1} be the orientation of the bending plane coordinate system 1{i1} of the i-th segment relative to the base disk coordinate system {ib}; i1 R i2 Let 2{i2} be the orientation of the bending plane coordinate system 2{i2} of the i-th segment relative to the bending plane coordinate system 1{i1}; i2 R ie Let {ie} be the orientation of the fixed disk coordinate system {i2} of the i-th segment relative to the curved plane coordinate system {i2}.

[0039] ib R i1 , i1 R i2 and i2 R ie It can be determined based on the following formulas (3), (4) and (5):

[0040]

[0041]

[0042]

[0043] like Figure 3 The segment parameter ψ of the single segment 300 shown i It can be determined based on the following formula (6):

[0044] ψ i =[θi ,δ i ] T (6)

[0045] In some embodiments, the driving amount of multiple structural bones has a known mapping relationship with the segmental parameters. Based on the target segmental parameters and the mapping relationship, the driving amount of the multiple structural bones can be determined. The driving amount of the multiple structural bones can be understood as moving a single segment from its initial state (e.g., θ) i =0) The length of the structural bone subjected to push or tension when bent to the target bending angle. In some embodiments, the mapping relationship between the driving amount of multiple structural bones and the joint parameters can be determined based on the following formula (7):

[0046] q ij =-r ij θ i cos(δ i -β ij (7)

[0047] Where, q ij r is the driving force of the j-th structural bone in the i-th segment. ij β is the distance from the j-th structural bone in the i-th segment to the virtual structural bone. ij Let be the angle between the j-th structural bone and the first structural bone in the i-th segment. The driving signal of the driving unit can be determined based on the driving amount of multiple structural bones.

[0048] In some embodiments, the feed amount of the manipulator 400 is based on the overall rotation angle of the manipulator, the feed length of the manipulator (e.g., the overall feed length of the manipulator, the feed length of a segment in the manipulator, or the feed length of a straight rod segment in the manipulator), and the individual segments constituting the manipulator (e.g., ...). Figure 4 The configuration parameters of the first component 4201 and the second component 4202 shown can determine the configuration parameters of the manipulator. For example, the configuration parameters of the manipulator can be determined based on the following formula (8):

[0049]

[0050] in, d represents the overall rotation angle of the manipulator, and d represents the overall feed length of the manipulator. In some embodiments, the overall rotation angle of the manipulator... The overall feed length d of the manipulator is provided by the drive unit; for example, the overall feed length d of the manipulator is provided by a linear drive mechanism that drives the linear feed of the manipulator, and the overall rotation angle of the manipulator... Provided by a rotary drive mechanism that drives the manipulator to rotate about its own central axis.

[0051] Those skilled in the art should understand that the manipulator has different configuration parameters in different working states. For example, Figure 4 The manipulator 400 shown includes at least four operating states, which correspond to four different configurations of the manipulator 400, and can be denoted as configurations C1-C4. The four operating states of the manipulator 400 are described below:

[0052] First working state (C1 position): Only the second component 4202 participates in the pose control of the control device (for example, only the second component 4202 enters the workspace), and the position parameters of the manipulator 400 at this time are as shown in the following formula (9):

[0053]

[0054] Where, ψ c1 These are the position parameters of the manipulator 400 in its first working state. L1 is the overall rotation angle of the manipulator 400, L2 is the feed length of the second component 3202, and L2 is related to... Figure 3 In the structure shown, L in section 300 t The physical meaning is the same. ψ2 is the segment parameter of the second segment 4202. ψ2 can be determined by the above formula (6).

[0055] Second working state (C2 position): The second component 4202 and the first linear segment 4203 participate in the position control of the control device (for example, the second component 4202 is fully in the working space, and the first linear segment 4203 is partially in the working space). At this time, the position parameters of the manipulator 400 are as shown in the following formula (10):

[0056]

[0057] Where, ψ c2 For the configuration parameters of the manipulator 400 in the second working state, L r This is the feed length of the first straight segment 4203.

[0058] The third working state (C3 position): The second segment 4202, the first linear segment 4203, and the first segment 4201 participate in the position control of the actuator (for example, the second segment 4202 is fully in the workspace, the first linear segment 4203 is fully in the workspace, and the first segment 4201 is partially in the workspace). At this time, the position parameters of the manipulator 400 are as shown in the following formula (11):

[0059]

[0060] Where, ψ c3For the configuration parameters of the operating arm 400 in the third working state, L1 is the feed length of the first component 4201, and L1 is related to... Figure 3 In the structure shown, L in section 300 t The physical meanings are the same. ψ1 is the segment parameter of the first segment 4201, and ψ2 is the segment parameter of the second segment 4202. ψ1 and ψ2 can be determined by the above formula (6).

[0061] Fourth working state (C4 position): The second component 4202, the first linear segment 4203, the first component 4201, and the second linear segment 4204 participate in the position control of the actuator (for example, the second component 4202 is fully in the workspace, the first linear segment 4203 is fully in the workspace, the first component 4201 is fully in the workspace, and the second linear segment 4204 is partially in the workspace). At this time, the position parameters of the manipulator 400 are as shown in the following formula (12):

[0062]

[0063] Where, ψ c4 For the configuration parameters of the manipulator 400 in the fourth working state, L s This is the feed length for the second straight segment 4204.

[0064] In some embodiments, similar to a single segment, the driving amount of each structural bone of each segment of the manipulator can be determined based on formula (7), and then the driving signal of the driving unit can be determined based on the driving amount.

[0065] Figure 1 A flowchart is shown for a control method 100 for a manipulator according to some embodiments of the present disclosure.

[0066] See Figure 1 In step 101, the current actual pose of the target portion of the manipulator is obtained. The current actual pose of the target portion of the manipulator includes the current actual position and the current actual orientation. In some embodiments, the current actual orientation of the target portion of the manipulator may be the current actual pose of the target portion of the manipulator relative to a reference coordinate system. In some embodiments, the reference coordinate system may be the base coordinate system of the manipulator or the world coordinate system. In some embodiments, the base coordinate system of the manipulator may be the coordinate system of the base on which the manipulator is mounted, the coordinate system of the sheath through which the manipulator passes (the coordinate system of the sheath outlet), the coordinate system of the remote center of motion (RCM) of the manipulator, etc. For example, the base coordinate system of the manipulator may be set at the sheath outlet position, and the base coordinate system of the manipulator remains fixed during teleoperation. Those skilled in the art should understand that the current actual pose of the target portion of the manipulator can be transformed by coordinate system transformation to obtain the pose relative to other coordinate systems.

[0067] In some embodiments, an electromagnetic sensor may be provided on the manipulator, and method 100 may include using the electromagnetic sensor to obtain the current actual pose of the target portion of the manipulator. The electromagnetic sensor may include an electromagnetic induction device and an electromagnetic reflector disposed on the target portion of the manipulator. The electromagnetic induction device determines the pose information (including position information and attitude information) of the electromagnetic reflector disposed on the target portion of the manipulator based on the principle of mutual inductance of electromagnetic fields, thereby obtaining the current actual pose of the target portion of the manipulator.

[0068] In some embodiments, method 100 may include obtaining a positioning image of the manipulator and analyzing the obtained positioning image to determine the current actual pose of the target portion of the manipulator.

[0069] In some embodiments, the manipulator may be equipped with positioning tags, which may include multiple pose markers and angle markers. Positioning images of the positioning tags are obtained through an image acquisition device, and the current actual position of the target portion of the manipulator is obtained by processing the positioning images. The image acquisition device includes, but is not limited to, dual-lens image acquisition devices or single-lens image acquisition devices, such as binocular or monocular cameras.

[0070] In some embodiments, the method may further include acquiring images of the manipulator using medical imaging equipment such as computed tomography (CT), magnetic resonance imaging (MRI), or stereoscopic vision, and determining the current actual pose of the target portion of the manipulator based on the analysis of the images.

[0071] In some embodiments, the manipulator may be a continuous robotic arm (e.g., Figure 4 The illustrated manipulator arm 400 includes at least one segment, which includes a fixation plate and multiple structural bones. The distal ends of the multiple structural bones are fixedly connected to the fixation plate, and the proximal ends of the multiple structural bones are connected to a drive unit. In some embodiments, the target portion includes the end effector of the manipulator arm, and the current actual pose includes the current actual pose of the end effector. In some embodiments, the end effector of the manipulator arm may include an end effector (e.g., a surgical actuator) mounted on the distal end of the arm body, and the current actual pose of the end effector may be the current actual pose of the end effector.

[0072] In some embodiments, method 100 may include using a visual detection algorithm to obtain the current actual pose of the end effector of the manipulator. In some embodiments, a positioning image is acquired using an image acquisition device, and the positioning image may include part or all of the image of the manipulator. In some embodiments, a positioning tag is provided at or near the distal end of the manipulator body, and the positioning tag is used for detecting the end effector position or pose of the manipulator. The positioning tag may include multiple pose identifiers and angle identifiers. In some embodiments, the positioning tag is within the field of view of the image acquisition device, and the acquired positioning image may include a positioning image for positioning the end effector of the manipulator, and the end effector pose of the manipulator is obtained through the positioning image. In some embodiments, method 100 may include obtaining a positioning image; identifying multiple identifiers located on the manipulator in the positioning image; and determining the current actual pose of the end effector based on the multiple identifiers.

[0073] In some embodiments, the surgical robot system may include a visual guide arm with an imaging device (e.g., a binocular endoscope) mounted at its distal end and at least one surgical execution arm with a surgical actuator mounted at its distal end. The imaging device mounted at the distal end of the visual guide arm acts as an image acquisition device, acquiring positioning images of the surgical execution arm to obtain the current actual pose of the end effector of the surgical execution arm. Figure 5 This diagram illustrates the acquisition of the current actual pose of the end effector of a surgical robot system 500 according to some embodiments of the present disclosure. Figure 5 As shown, the surgical robot system 500 includes at least one surgical execution arm 510 and a visual guidance arm 520. A binocular endoscope 521 is mounted on the distal end of the visual guidance arm 520, and a surgical actuator 511 is mounted on the distal end of the surgical execution arm 510. In some embodiments, a positioning tag is provided on the distal end of the surgical execution arm 510. Figure 5 (Not shown in the image), the distal end of the surgical arm 510 is within the field of view of the binocular endoscope 521. The positioning image acquired by the binocular endoscope 521 may include a positioning image of the distal end of the surgical arm 510, and the current actual pose of the distal end of the surgical arm 510 is obtained through the positioning image. In some embodiments, positioning images from the binocular endoscope 521 may be received and processed to obtain the current actual pose of the distal end of the surgical arm 510.

[0074] In some embodiments, such as Figure 4The illustrated manipulator 400 (e.g., on the arm body 420 or the distal end 410 of the arm body) has a plurality of pose markers and at least one angle marker distributed thereon. For example, the plurality of pose markers are distributed circumferentially on the distal end 410 of the arm body, and the plurality of angle markers are also distributed circumferentially on the distal end 410 of the arm body. The plurality of pose markers and the plurality of angle markers are arranged side by side axially on the distal end 410 of the arm body. For example, the plurality of pose markers and the plurality of angle markers are disposed on the outer surface of the columnar portion of the distal end 410 of the arm body.

[0075] In some embodiments, each angle marker has a positional association with one of the pose markers. Based on this positional association, the possible distribution area of ​​the angle markers can be determined by the position of the pose markers. Alternatively, the possible distribution area of ​​the pose markers can be determined by the position of the angle markers. The positional association can be determined according to the specific arrangement of the pose markers and angle markers, and can be pre-designed.

[0076] In some embodiments, the positional association may include an axial correspondence between angle markers and pose markers. For example, the positional association may include an axial offset. Based on the axial correspondence, given that the positions of one or more pose markers on the distal end of the arm are known, an axial offset by a certain distance can determine the area where angle markers may exist. For example, the positional association may also include axial oblique alignment, etc.

[0077] In some embodiments, multiple pose markers and multiple angle markers may be disposed on a label affixed to the periphery of the distal end of the arm. In some embodiments, the pose markers may include pose marker patterns and pose marker pattern corner dots, and the angle markers may include angle marker patterns and angle marker pattern corner dots. In some embodiments, the pose marker patterns and angle marker patterns may be disposed on a label affixed to the distal end of the arm, or may be printed on the distal end of the arm, or may be patterns formed by the physical structure of the distal end of the arm itself, for example, including recesses or protrusions and combinations thereof. In some embodiments, the pose marker patterns or angle marker patterns may include patterns formed with brightness, grayscale, color, etc. In some embodiments, the pose marker patterns and angle marker patterns may include patterns that actively (e.g., self-illuminating) or passively (e.g., reflecting light) provide information detected by an image acquisition device (e.g., an imaging device, such as a binocular endoscope, mounted on the distal end of the visual guide arm of a robotic system). Those skilled in the art should understand that, in some embodiments, the pose of the pose identifier can be represented by the pose of the corner coordinate system of the pose identifier pattern, and the pose of the angle identifier can be represented by the pose of the corner coordinate system of the angle identifier pattern.

[0078] Figure 6A schematic diagram is shown of a positioning tag 600 including a plurality of pose markers and a plurality of angle markers according to some embodiments of the present disclosure. Figure 7 A schematic diagram is shown of a positioning tag 700 disposed on the peripheral side of the distal end of the arm body and formed into a cylindrical shape according to some embodiments of the present disclosure. It will be understood that, for simplicity, the positioning tag 600 may include the same pose marking pattern and angle marking pattern as the positioning tag 700.

[0079] See Figure 6 Multiple pose markers (represented by the symbol "○" for corner points in this disclosure) and multiple angle markers (represented by the symbol "△" for corner points in this disclosure) are arranged side by side. The multiple pose marker patterns 611 may be identical or similar, and the corner points of the multiple pose marker patterns are located within the multiple pose marker patterns 611. The multiple angle marker patterns 621-626 may be different, and the corner points of the multiple angle marker patterns are located within the multiple angle marker patterns 621-626.

[0080] In some embodiments, each angle identifier may have a positional association with one of the pose identifiers. For example, such as Figure 6 As shown, in the direction indicated by the arrow, some pose markers (e.g., pose marker pattern 611) and corresponding angle markers (e.g., angle marker pattern 621) are arranged along the arrow direction and have a spacing d1. See also Figure 7 In the circumferential setting state, label 600 becomes label 700 with a spatial structure of a cylinder. The positional association between each angle identifier and one of the pose identifiers can include the angle identifier and the pose identifier in the axial direction (e.g., Figure 7 The correspondence between the angle markers and the pose markers along the positive Z-axis is established. Based on this axial correspondence, given the known positions of one or more pose markers on the distal end of the arm, the region where the angle markers may exist can be determined by offsetting by a certain distance (e.g., distance d1) along the axial direction. In some embodiments, the axial correspondence between the angle markers and the pose markers can be represented by the axial correspondence between the corner points of the angle marker pattern and the corner points of the pose marker pattern. In some embodiments, based on the axial correspondence between the angle markers and the pose markers, the projections of one of the corner points of the angle marker pattern and the corner points of the pose marker pattern along the Z-axis coincide.

[0081] In some embodiments, the about-axis angle or roll angle of the angle marker or pose marker can be represented by the about-axis angle of the corner point of the angle marker pattern or the corner point of the pose marker pattern. The corner point of the angle marker pattern is relative to the manipulator coordinate system (e.g., a coordinate system established at the distal end of the arm body, such as...). Figure 7 The angles of the XY coordinate system shown are known or predetermined, for example... Figure 7In the XY coordinate system, the angle between corner point R7 of the angle marker pattern and the X-axis is θ. Based on the positional relationship, the angle between corner point P7 of the pose marker pattern associated with its position and the X-axis can be obtained as angle θ. It should be understood that the angle θ corresponding to corner point R7 of the angle marker pattern and corner point P7 of the pose marker pattern can be called the axial angle or roll angle of the angle marker or pose marker about the Z-axis. In this disclosure, the axial angle or roll angle refers to the angle about the Z-axis. It is understood that, for clarity, Figure 7 The corner point R7 of the angle marker pattern and the corner point P7 of the pose marker pattern are shown as separate, but they are overlapping.

[0082] In some embodiments, the manipulator (e.g.,) can be obtained through a visual detection algorithm. Figure 4 The shown operating arm 400 Figure 5 The surgical execution arm 510 and visual guidance arm 520 shown are... Figure 8 The shown operating arm 800 or Figure 15 The current actual pose of the end effector of the manipulator 1511 shown. In some embodiments, method 100 may include obtaining a positioning image of a positioning tag disposed at the distal end of the arm; identifying multiple markers (including pose markers and angle markers) located on the manipulator in the positioning image; and determining the current actual pose of the end effector based on the multiple markers, wherein the current actual pose of the end effector includes the current actual position of the end effector and the current actual posture of the end effector. Figure 8 This diagram illustrates the current actual pose of the end effector of the manipulator 800 obtained using a visual detection algorithm based on a binocular endoscope 821 according to some embodiments of the present disclosure. Figure 8 In the middle, the 800-meter manipulator is as follows: Figure 4 The manipulator 400 shown may include an end effector (e.g., a surgical actuator) mounted on the distal end of the arm body. Therefore, the current actual pose of the end effector of the manipulator 800 refers to the current actual pose of the end effector (e.g., the current actual pose of the surgical actuator). In some of the following embodiments, the end effector is described using a surgical actuator as an example. The following describes... Figure 8 The definition of the coordinate system is explained below: Surgical actuator coordinate system and Figure 4 Consistent with the above, please refer to the following: Figure 4 The surgical actuator coordinate system {tip} is attached to the surgical actuator and is fixed at the distal end of the distal arm 410 at a set distance from the fixed disk coordinate system {2e} at the end of the second component 4202 in the main body of the arm 420. and Consistent direction direction such as Figure 4 or Figure 8 As shown. Sheath coordinate system. and Figure 4 It is consistent with the middle, and it is attached to the outlet of the sheath. With the first segment base disk coordinate system {1b} Consistent direction direction such as Figure 4 or Figure 8 As shown. Location label coordinate system. The positioning tag is attached to the peripheral side of the distal end of the arm body, with the origin being the center of the circle containing the corner points of multiple pose marker patterns. The axis direction points from the origin to one of the corner points of the pose marker pattern. The direction is parallel to the axis of the distal end of the arm. Axis perpendicular to Plane. Binocular endoscope left lens coordinate system. It is attached to the left lens, with its origin located at the center of the left lens. The direction is parallel to the optical axis of the left lens. direction such as Figure 8 As shown. Coordinate system of the right lens of the binocular endoscope. It is attached to the right lens, with its origin located at the center of the right lens. The direction is parallel to the optical axis of the right lens. direction such as Figure 8 As shown. In some embodiments, as Figure 8 As shown, the left and right lenses of the binocular endoscope 821 can obtain positioning images respectively. Based on one image (left lens image or right lens image) from the binocular endoscope 821, a visual detection algorithm can be used to estimate the pose of the positioning tag in the binocular endoscope lens coordinates. For example, the pose of the positioning tag in the left lens coordinate system {ll} of the binocular endoscope (…). ll p wm , ll R wm Or the pose of the positioning tag in the coordinate system {rl} of the right lens of the binocular endoscope. rl p wm , rl R wm ),in, ll p wm , rl p wm Indicates location, ll R wm , rl R wm Indicates pose. The following uses the pose of the positioning label in the coordinate system {ll} of the left lens of the binocular endoscope ( ll p wm , ll R wmLet's take an example to illustrate. The pose of the positioning tag in the coordinate system of the left lens of the binocular endoscope ( ll p wm , ll R wm It can be determined through image analysis, and the homogeneous transformation matrix of the positioning tag relative to the coordinate system of the left lens of the binocular endoscope can also be obtained. ll T wm Therefore, the homogeneous transformation matrix of the surgical actuator coordinates {tip} at the end of the manipulator 800 relative to its sheath coordinate system {tc} can be, for example, shown in the following formula (13):

[0083]

[0084] in, Let represent the homogeneous transformation matrix of the surgical actuator coordinates {tip} obtained based on the visual detection algorithm relative to its sheath coordinate system {tc}. This represents the current actual pose of the end effector of the manipulator obtained based on a visual detection algorithm (i.e., the current actual pose of the surgical actuator relative to the sheath coordinate system). This represents the current actual position of the end effector of the manipulator. This represents the current actual position of the end effector of the manipulator. tc T ll This represents the homogeneous transformation matrix of the left lens coordinate system of the binocular endoscope relative to the sheath coordinate system. ll T wm This represents the homogeneous transformation matrix of the positioning tag coordinate system relative to the coordinate system of the left lens of the binocular endoscope. wm T tip This is the homogeneous transformation matrix of the surgical actuator coordinates relative to the positioning label coordinate system.

[0085] In some embodiments, the binocular endoscope is fixed to the distal end of the visual guidance arm (e.g., as shown in the figure). Figure 5 As shown, the binocular endoscope 521 is mounted on the distal end of the visual guide arm 520. The current actual pose of the end of the visual guide arm can be obtained using methods described in some of the above embodiments (e.g., the current actual pose of the end of the visual guide arm corresponds to the current actual pose of the binocular endoscope). Based on the current actual pose of the end of the visual guide arm, the poses of the left and right lenses in the binocular endoscope can be obtained separately. This allows the determination of the transformation matrix of the binocular endoscope lens coordinate system relative to the sheath coordinate system, such as the homogeneous transformation matrix of the left lens coordinate system relative to the sheath coordinate system. tc T llIn some embodiments, the surgical actuator is fixedly disposed at the distal end of the distal end of the surgical arm, and a positioning tag is fixedly disposed at the distal end of the arm (e.g., the positioning tag is fixedly disposed on the outer surface of the columnar portion of the distal end of the arm). Therefore, wm T tip It is known or predetermined. In some embodiments, based on tc T ll , ll T wm , wm T tip It has been determined that the current actual pose of the end effector of the manipulator can be obtained based on formula (13).

[0086] Continue reading Figure 1 In step 103, the target pose of the target portion of the manipulator is obtained. In some embodiments, the target pose of the target portion of the manipulator can be the target pose of the target portion of the manipulator relative to a reference coordinate system. In some embodiments, the reference coordinate system can be the base coordinate system or the world coordinate system of the manipulator.

[0087] In some embodiments, method 100 further includes receiving a control command; and determining a target pose of a target portion of the manipulator based on the control command.

[0088] In some embodiments, the target portion includes the end effector of the manipulator, and the target pose includes the end effector target pose. In some embodiments, the end effector target pose of the manipulator can be input by a user via an input device. In some embodiments, control commands can be received based on a master-slave motion control method. For example, by acquiring the pose or joint information of the master manipulator in each motion control cycle, the end effector target pose of the manipulator can be determined. The end effector target pose of the manipulator can be the target pose of the end effector of the manipulator relative to the world coordinate system. Real-time master-slave motion control can be performed through multiple motion control cycles.

[0089] In some embodiments, method 100 may include determining the force on a target portion of the manipulator and, based on the force on the target portion, determining a force condition for the target portion. In some embodiments, based on the fact that the force on the target portion of the manipulator is not greater than an external force threshold, it is determined that the target portion of the manipulator meets a first force condition; based on the fact that the force on the target portion of the manipulator is greater than the external force threshold, it is determined that the target portion of the manipulator meets a second force condition. In this disclosure, the external force threshold may be configured as an empirical value or may be predetermined. For example, in some embodiments, the external force threshold may be set to 3N.

[0090] Figure 9A flowchart illustrating a method 900 (hereinafter also referred to as "method 900") for determining the force conditions of a target portion of a manipulator according to some embodiments of the present disclosure. Method 900 may be implemented or performed by hardware, software, or firmware. In some embodiments, method 900 may be implemented by a surgical robot system (e.g., Figure 2 The surgical robot system shown Figure 5 The surgical robot system 500 shown is... Figure 15 The surgical robot system 1500 shown is executed. In some embodiments, method 900 can be implemented as computer-readable instructions. These instructions can be executed by a general-purpose processor or a special-purpose processor (e.g., for example, ...). Figure 2 The control device 220 shown or Figure 15 The processor 1530 shown reads and executes data. For example, the control unit of a surgical robot system (e.g., Figure 2 The control device 220 shown may include a processor configured to execute method 900. In some embodiments, these instructions may be stored on a computer-readable medium.

[0091] See Figure 9 In step 901, the current actual position of the target portion of the manipulator is obtained. In some embodiments, a method similar to... Figure 1 The current actual position of the target part of the manipulator is obtained in a similar manner to step 101. This has been described in detail in some of the above embodiments and will not be repeated here.

[0092] Continue reading Figure 9 In step 903, the current actual drive amount of the manipulator is obtained. In some embodiments, the manipulator may be a continuous robotic arm (e.g., Figure 4 The manipulator 400 shown includes at least one segment, which includes a fixing plate and multiple structural bones. The distal ends of the multiple structural bones are fixedly connected to the fixing plate, and the proximal ends of the multiple structural bones are connected to a drive unit. The current actual drive amount of the manipulator may include the current actual drive amount output by the drive unit to the multiple structural bones of the manipulator.

[0093] In some embodiments, the operator issues control commands to control the movement of the manipulator (e.g., the control commands include drive information). The drive unit responds to the control commands by pushing and / or pulling the structural bone, thereby moving the structural bone to meet the operator's operational needs for the manipulator. In some embodiments, the current control commands for the manipulator can be obtained, and the drive information (e.g., drive vector) of the manipulator can be obtained from these commands. Based on the drive information, the current actual drive amount output by the drive unit to the structural bone of the manipulator can be obtained. In some embodiments, the current actual drive amount of the structural bone of the manipulator can be obtained by sensing the state (e.g., rotation angle or stroke) of the drive device (e.g., a potentiometer) using a sensor.

[0094] In some embodiments, the drive vector of the manipulator may include the drive vector q of multiple structural bones of the manipulator. ij , ij is the number of the structural bone, and the driving forces of multiple structural bones constitute the driving force vector of the structural bone, which can be denoted as q. a,bkb For example, such as Figure 4 In the illustrated manipulator 400, the deformation of the i-th segment is achieved by driving two sets of symmetrically distributed structural bones through a drive unit. In some embodiments, each set of symmetrically distributed structural bones may include two symmetrically distributed structural bones (e.g., the included angle between the two structural bones is π). In some embodiments, the deformation of the i-th segment can be achieved by driving (e.g., simultaneously pushing and pulling) the two sets of symmetrically distributed structural bones, for example, by two pairs of double-ended screw drive mechanisms. In some embodiments, the double-ended screw drive mechanism may include a double-ended screw and two threaded sliders located on the double-ended screw. When the double-ended screw rotates, the two threaded sliders located on the double-ended screw move in opposite directions at the same speed, and the two threaded sliders drive the two symmetrically distributed structural bones to move in opposite directions at the same speed, so that the two symmetrically distributed structural bones are pushed or pulled to achieve the bending deformation of the segment. In some embodiments, the two sets of structural bones driven by the i-th segment are represented by structural bones numbered i1 and i2, respectively. Therefore, the driving vector of the structural bones can be expressed as: q a,bkb =[q 11 ,q 12 ,q 21 ,q 22 ] T In some embodiments, the current driving vector of the structural bone can be obtained, and the current driving vector of the structural bone can be used as the current actual driving amount of the manipulator, which can be represented as q. a,bkb .

[0095] Continue reading Figure 9In step 905, based on the current actual driving quantity and the driving quantity residual calculation model, the first driving quantity residual of the operating arm in the free motion state is determined. In this disclosure, the free motion state of the operating arm refers to the motion state of the operating arm when there is no external load force acting on it under the drive of the drive unit.

[0096] Due to the presence of factors such as difficult-to-model and nonlinear drive hysteresis and / or tensile / compressive deformation of the structural bones within the manipulator, the drive quantity residuals of the structural bones within the manipulator always exist. In some embodiments, the first drive quantity residual of the manipulator in a free-movement state may include the first drive quantity residual values ​​of multiple structural bones of the manipulator in a free-movement state, with each structural bone corresponding to a first drive quantity residual value, and the first drive quantity residual values ​​of multiple structural bones combined to form a first drive quantity residual vector.

[0097] To compensate for the driving quantity residual that is difficult to model when the manipulator is without external load, in some embodiments of this disclosure, a driving quantity residual calculation model is trained using model design and a series of random trajectory motion data. The trained driving quantity residual calculation model can predict the driving quantity residual under actual driving quantity conditions and without external load; that is, the driving quantity residual calculation model can predict the driving quantity residual of the manipulator in free motion under actual driving quantity conditions. In this disclosure, the driving quantity residual predicted by the driving quantity residual calculation model is denoted as the first driving quantity residual, which can be expressed as: For example, consider a manipulator that drives four symmetrically distributed structural bones. ij = 11, 12, 21, 22 Let be the residual of the first driving force of the ij-th structural bone.

[0098] In some embodiments, the actuation residual calculation model can be a machine learning model. This model can be constructed and trained based on machine learning algorithms to calculate the first actuation residual of at least one structural bone (e.g., four structural bones) of a continuum manipulator in a free state. In some embodiments, the actuation residual calculation model can include a support vector regression model, referred to as an SVR (Support Vector Regression) model. The SVR model can directly predict the actuation residual of the manipulator at the current actual actuation (e.g., the actuation vector q of a given structural bone of the manipulator). a,bkb The first driving quantity residual when there is no external load force applied under the driving force can be expressed as:

[0099] In some embodiments, the training dataset for the drive quantity residual calculation model may include: a drive quantity residual dataset of the manipulator in free motion under different actual drive quantity conditions.

[0100] In some embodiments, the SVR model can transform the original data space into a high-dimensional linearly separable space through a kernel function, thereby achieving better regression results for nonlinear data. Simultaneously, the SVR model represents the entire data feature using a finite number of support vectors and corresponding weight coefficients; therefore, the SVR model enables rapid prediction calculations, ensuring the real-time performance of the manipulator control. In some embodiments, independent SVR models can be created for each structural bone within the continuous robotic arm. Exemplarily, in some embodiments, such as... Figure 4 In the manipulator 400 shown, the deformation of the i-th segment is achieved by driving two sets of symmetrically distributed structural bones through a driving unit. The driving of the two sets of structural bones in the i-th segment is represented by structural bones numbered i1 and i2, respectively. Therefore, for... Figure 4 The operating arm 400 includes a first segment and a second segment. Four independent SVR models are created for the four representative structural bones (ij = 11, 12, 21, 22) within the operating arm 400, as shown in the following formula (14):

[0101]

[0102] Among them, SV ij χ is the number of support vectors for the ij-th SVR model (the SVR model used to predict the first driving force residual of the ij-th bone structure), and χ is the feature vector of the input SVR model. ij,w and m ij,w Let w be the w-th support vector and its weights for the ij-th SVR model, and n be the weights. ij Let κ(χ) be the constant term of the ij-th SVR model. ij,w κ(χ) is the Gaussian kernel function of the SVR model. ij,w The definition of χ) can be shown in the following formula (15):

[0103]

[0104] Where, γ ij γ is an adjustable parameter of the Gaussian kernel function. ij >0. The optimal γ value before training the SVR model. ij The parameters, as well as the soft margin parameter C in the SVR model, are determined through a gridded traversal search strategy and cross-validation.

[0105] In some embodiments, considering the coupling force relationship between the structural bones within the continuum manipulator, that is, the structural bone of the i-th segment not only affects the bending deformation within its own segment but also affects the deformation of the structural bones of other segments due to the coupling effect between segments. In some embodiments, the feature vector χ of the four SVR models considers the driving effect of all structural bones, that is, the feature vector χ of each SVR model is defined consistently, as shown in the following formula (16):

[0106]

[0107] Among them, h bkb =[h 11 ,h 12 ,h 21 ,h 22 [h] is a hysteresis feature vector characterizing the hysteresis properties of the structural bone drive process, representing the historical state of the structural bone drive process. In some embodiments, for time k, h ij (k) can be represented as an accumulator with saturation properties, as shown in the following formula (17):

[0108]

[0109] Where, μ max Representing the maximum hysteresis of each structural bone, μ is relevant to the continuous manipulator in some embodiments of this disclosure. max Set to 0.2mm. Δq a,ij Let ij be the driving increment of the structural bone numbered ij during the time interval from time k-1 to time k.

[0110] In some embodiments, the SVR model for calculating the first driving quantity residual of the four structural bones is determined based on formulas (14) to (17). In some embodiments, before using the SVR model, a large amount of random dataset needs to be collected to train the SVR model. For example, for the ij-th SVR model, the collected random dataset can be represented as: ij = 11, 12, 21, 22, where χ1, χ2, ... are random data for the eigenvectors. ...represents the residual data of the first driving quantity of the manipulator in free motion under the actual driving quantity conditions corresponding to the random data χ1, χ2, ... of the feature vector. ...can be obtained through experiments.

[0111] In some embodiments, different configurations of the manipulator (e.g., C1-C4 configurations) can affect the training and prediction results of the SVR model. In some embodiments, such as... Figure 4In the C3 configuration, the overall feed length d of the manipulator 400 directly alters the feed length L1 of the first component 4201. Different L1 conditions affect the training and prediction results of the SVR model. In some embodiments, during SVR model training, L1 is first discretized from a predetermined motion range (e.g., 40mm to 60mm) at predetermined intervals (e.g., 5mm), and a sufficient amount of random data is collected for each L1 condition to train the corresponding L1 SVR model. During the calculation of the first driving quantity residual using the trained SVR model, the most recent set of SVR models is selected for prediction based on the current L1 value.

[0112] For example, L1 = L x The random data collection process under certain circumstances can be carried out in the following manner:

[0113] Set L1 = L x , Based on the maximum bending angle of the components in the manipulator (e.g., θ) 1,max =π / 2,θ 2,max =2π / 3) Determine the range of structural bone driving force, and randomly select N groups (e.g., 3000 groups) of structural bone driving force vectors q within the range of structural bone driving force. a,bkb The driving vector sum of each group of structural bones is L1 = L x , This constitutes a set of manipulator drive vectors For each set of driving quantities q a The theoretical pose of the positioning tag coordinate system {wm} relative to the binocular endoscope lens coordinate system is calculated using a kinematic model. For example, the theoretical pose of the positioning tag coordinate system {wm} relative to the binocular endoscope left lens coordinate system {ll}. ll T wmDrive vectors exceeding the visual detection range of the positioning tag on the distal end of the arm are discarded. In some embodiments, the final random drive quantity trajectory can be obtained by interpolation calculation between two adjacent drive vectors after the above-discarded drive vectors at a set interval (e.g., 0.05 mm). The manipulator is driven to each given drive quantity position within the random drive quantity trajectory using an experimental platform under no external load conditions, and the feature vector (χ) at each given drive quantity position is calculated and recorded. In some embodiments, the end-effector pose of the manipulator can be obtained by a visual detection algorithm as described in some of the embodiments above. Shape reconstruction is performed based on the end-effector pose, and the theoretical drive quantity of the manipulator under each given drive quantity is obtained based on the reconstructed shape, which can be implemented in some of the embodiments described later. In some embodiments, the drive quantity residual can be obtained based on the given drive quantity and the calculated theoretical drive quantity. Since the random data acquisition process is experimentally obtained under no external load force, the experimentally obtained drive quantity residual can be regarded as the first drive quantity residual of the manipulator in the free movement state, and the experimentally obtained first drive quantity residual is denoted as Therefore, the feature vectors of all records at all given driving quantities and the corresponding residuals of the first driving quantity constitute a random dataset. In some embodiments, a portion of the obtained random dataset (e.g., 80% of the data in the random dataset) is randomly selected as the training dataset, and the remainder (e.g., the remaining 20% ​​of the data) is used as the test dataset to verify the prediction performance of the SVR model trained on the training dataset.

[0114] Continue reading Figure 9 In step 907, the current ideal position of the operating arm is determined based on the current actual driving quantity and the residual of the first driving quantity.

[0115] In some embodiments, method 900 includes determining the ideal drive quantity corresponding to the current free movement state of the manipulator based on the current actual drive quantity and a first drive quantity residual. In this disclosure, the first drive quantity residual is the drive quantity residual of the structural bone within the manipulator, obtained by considering factors such as drive hysteresis and / or tensile / compressive deformation of the structural bone, which are difficult to model and are nonlinear within the manipulator. In some embodiments, the ideal drive quantity corresponding to the current free movement state of the manipulator includes the ideal joint drive quantity corresponding to the current free movement state of the manipulator. The ideal structural bone drive quantity of multiple structural bones corresponding to the current free movement state of the manipulator can be obtained through the current actual drive quantity and the first drive quantity residual. Based on the ideal structural bone drive quantity of multiple structural bones, the overall feed length of the manipulator, and the overall rotation angle of the manipulator, the ideal joint drive quantity corresponding to the current free movement state of the manipulator can be obtained.

[0116] For example, such as Figure 4In the illustrated manipulator arm 400, the deformation of the i-th segment is achieved by a drive unit driving two sets of symmetrically distributed structural bones. The specific method by which the drive unit drives the bending deformation of the segment is consistent with some of the embodiments described above, and will not be repeated here. Figure 4 In the illustrated manipulator 400, the actuation of the two sets of structural bones in the i-th segment is represented by structural bones numbered i1 and i2, respectively. The manipulator 400 includes two segments, namely the first segment 4201 and the second segment 4202. Therefore, the manipulator 400 has four sets of structural bones in total. The actuation of the four sets of structural bones in the manipulator 400 is represented by structural bones numbered ij = 11, 12, 21, and 22, respectively. The current actual actuation amount of the manipulator can be obtained through the methods in some of the above embodiments. The current actual actuation amount of the manipulator can be denoted as q. a,bkb =[q 11 ,q 12 ,q 21 ,q 22 ] T The first driving quantity residual of the operating arm in a free-movement state can be obtained through the methods in some of the above embodiments. The first driving quantity residual can be denoted as... Based on the current actual drive quantity q of the manipulator a,bkb The residual of the first driving force of the manipulator in free motion The ideal structural bone drive quantity of multiple structural bones corresponding to the current free movement state of the manipulator can be determined, and can be denoted as... It can be represented as in, This represents the ideal structural bone driving amount corresponding to the ij-th structural bone in the current free motion state. Based on the ideal structural bone driving amount of the structural bone in the current free motion state, the possible overall feed length d, and the possible overall rotation angle... We can obtain the ideal joint drive value corresponding to the current state of free movement, which can be denoted as... It can be represented as In some embodiments, the overall feed length d may be provided by the drive unit. In some embodiments, there may be an elongated section (e.g., longer than 200 mm) between the drive unit and the sheath, which will generate a certain torsion under external load, thus allowing for an overall rotation angle. The required drive rotation angle is provided by the drive unit (e.g., a rotary drive mechanism that can rotate about its own central axis). Add an extra offset angle to the base Right now Among them, the additional offset angle This can be obtained through shape estimation. In some embodiments, the configuration parameters of the manipulator can be obtained by shape reconstruction based on the shape information and / or end-effector pose of the manipulator (e.g., a continuous manipulator), with additional offset angles incorporated during the shape reconstruction process. As a parameter to be estimated, the estimated value of the additional deflection angle is obtained through estimation. The specific implementation can be achieved using the methods described in some of the embodiments described later. Therefore, It can be represented as

[0117] In some embodiments, method 900 includes determining the current ideal position of a target portion of the manipulator based on the ideal actuation amount corresponding to the manipulator's current free movement state. In some embodiments, the ideal pose information (including position information and attitude information) of the manipulator in its free movement state can be obtained based on the ideal actuation amount corresponding to the manipulator's current free movement state and the kinematic model of the manipulator, thereby obtaining the current ideal position of the target portion of the manipulator.

[0118] In some embodiments, the manipulator includes at least one segment, the segment including a fixing plate and multiple structural bones, the distal ends of the multiple structural bones being fixedly connected to the fixing plate, and the proximal ends of the multiple structural bones being connected to a drive unit. In some embodiments, the target portion includes the end of the manipulator, and the current ideal position includes the current ideal position of the end.

[0119] In some embodiments, the end effector of the manipulator may include an end effector (e.g., a surgical actuator) mounted on the distal end of the arm body, and the current ideal position of the end effector can be the current ideal position of the end effector. This is based on the ideal actuation amount corresponding to the manipulator's current free-movement state (e.g., ...). The kinematic model of the manipulator can be used to obtain the current ideal pose of the end effector in free motion. The current ideal pose of the end effector can be expressed as: The current ideal position of the end effector of the manipulator. This represents the current ideal posture of the end effector of the manipulator. The kinematic model of the manipulator has been described in detail in some of the embodiments above, and will not be repeated here.

[0120] See Figure 9 In step 909, the forces acting on the target portion are determined based on the current actual position and the current ideal position. In this disclosure, the forces acting on the target portion can be external forces acting on the target portion. For example, external forces can include the forces generated when the target portion of the operating arm comes into contact with the tissue being operated on during a surgical procedure.

[0121] In some embodiments, method 900 further includes obtaining the current stiffness of the target portion; and determining the forces acting on the target portion based on the current actual position, the current ideal position, and the current stiffness.

[0122] In some embodiments, the actual position deformation of the target portion can be determined based on the current actual position and the current ideal position. The actual position deformation of the target portion can be a position difference vector generated after the target undergoes position deformation.

[0123] In some embodiments, the current stiffness of the target portion corresponding to the actual position deformation direction can be obtained, and the force on the target portion can be determined based on the actual position deformation and the current stiffness. In some embodiments, based on Hooke's law, the product of the actual position deformation and the current stiffness of the target portion is determined as the force on the target portion.

[0124] In some embodiments, the manipulator includes at least one segment, the segment including a fixing plate and multiple structural bones, the distal ends of the multiple structural bones being fixedly connected to the fixing plate. In some embodiments, the target portion includes the end of the manipulator, the current actual position includes the current actual position of the end, and the current ideal position includes the current ideal position of the end. Method 900 may include determining the actual position deformation of the end of the manipulator based on the current actual position and the current ideal position of the end. Method 900 may also include obtaining the current stiffness of the end of the manipulator corresponding to the direction of the actual position deformation, and determining the forces acting on the end of the manipulator based on the actual position deformation and the current stiffness.

[0125] In some embodiments, method 900 further includes: obtaining the end-effector average stiffness of the end of the manipulator; and determining the end-effector virtual force based on the current actual position of the end of the manipulator, the current ideal position of the end of the manipulator, and the end-effector average stiffness.

[0126] In some embodiments, the end effector of the manipulator may include an end effector (e.g., a surgical actuator) mounted on the distal end of the arm body; therefore, the average end effector stiffness of the manipulator end effector may be the average stiffness of the end effector. In embodiments of this disclosure, the average end effector stiffness of the manipulator end effector may be determined based on the material properties of the manipulator end effector.

[0127] In some embodiments, the actual position deformation of the end effector is determined based on the current actual position and the current ideal position of the end effector, for example, as... Figure 12 As shown, the actual position deformation of the end of the manipulator can be determined by the following formula (18):

[0128]

[0129] Where, Δp loadThis represents the actual position deformation of the end effector arm. This represents the current actual position of the terminal. The current ideal position of the end.

[0130] Based on Hooke's Law, the product of the actual positional deformation and the average end-effector stiffness of the manipulator's end is determined as the virtual end-effector force of the manipulator. For example, ... Figure 12 As shown, the virtual force at the end of the manipulator can be determined by the following formula (19):

[0131]

[0132] Among them, f virtual For the end virtual force, For the end-effector average stiffness, for example, in some implementations, It can be set to 20 N / m, Δp load This represents the actual position deformation of the end of the manipulator.

[0133] In some embodiments, method 900 further includes determining the end-effector virtual position deformation of the end of the manipulator based on the current actual drive amount of the manipulator, the end-effector virtual force, and the mechanical model of the manipulator, wherein the mechanical model is determined based on the distribution of multiple structural bones of the manipulator on the cross section of the manipulator and the physical properties of the multiple structural bones.

[0134] In some embodiments, the mechanical model may include constitutive relations related to the structural bones. These constitutive relations can represent the material properties of the structural bones, for example, by expressing the material properties through the internal forces and deformations of the structural bones. In some embodiments, the manipulator arm includes multiple structural bones (j is the structural bone number, j = 1, 2, 3…m), and the constitutive relations of the internal forces of the manipulator arm can be determined based on the constitutive relations of the internal forces of the structural bones. The constitutive relations of the internal forces of the manipulator arm can be determined based on the shear-tensile stiffness matrices of the multiple structural bones.

[0135] In some embodiments, the manipulator includes a constraint structure (e.g., a spacer, a fixed plate, a covering layer, etc.) and a structural skeleton. The internal force constitutive relationship of the manipulator can be determined based on the internal force constitutive relationship of the constraint structure and the structural skeleton. For example, the internal force constitutive relationship of a deformable manipulator is given by formula (20):

[0136]

[0137] In formula (20), n all It represents the internal forces of the manipulator, R is the rotation matrix, and K... SE K is the shear and tensile stiffness matrix of the constraint structure of the manipulator. SEjis the shear-tensile stiffness matrix of the j-th structural bone in the manipulator, and v is the linear velocity of the manipulator's pose changing along the arc length of a reference line. For example, the reference line could be the central axis of the manipulator. min This is the linear velocity of the manipulator in its natural state. For example, v min It can be the linear velocity when there is no external force or driving force, v min =[0 0 1] T .

[0138] In some embodiments, the manipulator includes multiple structural bones (j is the structural bone number, j = 1, 2, 3…m), and the internal moment constitutive relation of the manipulator can be determined based on the internal moment constitutive relation of the structural bones. The internal moment constitutive relation of the manipulator can be determined based on the bending and torsional stiffness matrix of the structural bones.

[0139] In some embodiments, the manipulator includes a constraint structure and a structural skeleton, and the internal moment constitutive relation of the manipulator can be determined based on the internal force constitutive relation of the constraint structure and the structural skeleton. For example, the internal moment constitutive relation of the manipulator is given by equation (21):

[0140]

[0141] In formula (21), m all R is the internal torque of the manipulator, R is the rotation matrix, and K is the internal torque of the manipulator. BT Let K be the bending and torsional stiffness matrix of the constraint structure of the manipulator. BTj U is the bending and torsional stiffness matrix of the j-th structural bone in the manipulator, and u is the angular velocity of the manipulator's pose along the arc length of the reference line. min It refers to the angular velocity under natural conditions, such as the angular velocity when there is no external force or driving force. min =0.

[0142] In some embodiments, the mechanical model includes mechanical equilibrium relationships related to the structural skeleton. These mechanical equilibrium relationships include the force equilibrium relationships of the structural skeleton, which include the force equilibrium at various locations along the axial direction of the structural skeleton. In some embodiments, the force equilibrium relationships include the equilibrium between external and internal forces acting on the structural skeleton along the axial direction at the points of force application.

[0143] In some embodiments, the mechanical balance relationship includes the force balance relationship of the operating arm, which includes the force balance of the structural bone at various points along the axial direction. In some embodiments, the force balance relationship of the operating arm includes the balance between the external and internal forces on the operating arm at the point of force application along the axial direction.

[0144] Figure 11 A force diagram of the manipulator 1100 according to some embodiments is shown. See also Figure 11In equation (a), for a structural unit [s, s + Δs] of the manipulator, the force balance relationship of the structural unit [s, s + Δs] is given by equation (22):

[0145]

[0146] The manipulator may include constraint structures (e.g., spacer plates, fixed plates, covering layers, etc.) and structural skeleton. In formula (22), n(s) represents the internal force of the manipulator skeleton at point s, n(s+Δs) represents the internal force of the manipulator constraint structure at point (s+Δs), Δs is a small increment, and f e (ξ) represents the distributed external force on the manipulator at point ξ. j (s) represents the internal force on the j-th structural bone at point s, n j (s+Δs) represents the internal force on the j-th structural bone at (s+Δs).

[0147] Based on formula (22), the force balance relationship within the operating arm is obtained, as shown in formula (23):

[0148] n′ all +f e =0 (23)

[0149] In formula (23), n all It represents the internal forces acting on the manipulator, where ()′ denotes differentiation, f e This refers to distributed external forces (such as the gravity of the manipulator). In some embodiments, distributed external forces can be ignored.

[0150] In some embodiments, the mechanical balance of the manipulator includes a torque balance relationship, which includes the axial torque of the structural bones. See also Figure 11 In (b), for a structural unit [s, s + Δs] of the manipulator, the torque balance relationship of the structural unit [s, s + Δs] is given by formula (24):

[0151]

[0152] In formula (24), m(s) represents the internal torque of the constraint structure of the manipulator at point s, m(s+Δs) represents the internal torque of the constraint structure of the manipulator at point s+Δs, and Δs is a small increment. p(s+Δs) represents the position of the constraint structure of the manipulator at point s+Δs, and n(s+Δs) represents the internal force on the constraint structure of the manipulator at point s+Δs. e(ξ) represents the distributed torque of the constraint structure of the manipulator at ξ (in some embodiments, the distributed torque can be ignored), p(ξ) is the position of the constraint structure of the manipulator at ξ, and f e (ξ) represents the distributed external force at ξ on the constraint structure of the manipulator. m j (s) represents the internal torque of the j-th structural bone at point s, m j (s+Δs) represents the internal torque of the j-th structural bone at s+Δs. j (s+Δs) represents the position of the j-th structural bone at s+Δs, where n j (s+Δs) represents the internal force on the j-th structural bone at s+Δs. j (s) represents the position of the j-th structural bone at s, n j (s) represents the internal force on the j-th structural bone at point s.

[0153] Based on formula (24), the torque balance relationship of the operating arm at position s is obtained, see formula (25):

[0154]

[0155] In formula (25), m all It is the internal torque of the manipulator, p is the position of the reference line of the manipulator, and n is the torque of the manipulator. all It is the internal force of the operating arm, l e This represents the distributed torque of the manipulator; in some embodiments, the distributed torque can be ignored. R is the rotation matrix of the manipulator's reference line, r j It is the distribution of the j-th structural bone on the cross section of the manipulator, for example, r. j It can be the coordinates of the structural bones on the cross-section of the manipulator, u is the angular velocity of the manipulator's pose changing along the arc length of the reference line, and K SEj The shear-tensile stiffness matrix of the j-th structural bone. It is linear strain, representing the difference in linear velocity of the pose of the j-th structural bone along the arc length of the reference line before and after deformation.

[0156] In some embodiments, a torque boundary condition is applied at the end of the operating arm, which includes the sum of the torques at the end of the operating arm being zero, as shown in equation (26):

[0157]

[0158] In formula (26), m e is the external torque of the operating arm at position L at the end of the operating arm, m(L) is the internal torque of the operating arm at position L at the end of the operating arm, and R(L) is the rotation matrix of the operating arm at position L at the end of the operating arm.

[0159] In some embodiments, a force boundary condition is applied at the end of the operating arm, which includes the sum of the forces acting on the operating arm at the end of the operating arm being zero, as shown in equation (27):

[0160] n e -n(L)=0 (27)

[0161] In formula (27), n e n(L) is the external force on the operating arm at position L at the end of the operating arm, and n(L) is the internal force on the operating arm at position L at the end of the operating arm.

[0162] In some embodiments, the mechanical model of the manipulator includes the relationship between the axial length variation of the structural bones and the distribution of the structural bones across the cross-section of the manipulator. For example, the axial length variation of each structural bone in the manipulator (j is the structural bone number, j = 1, 2, 3, ..., m) is given by formula (28):

[0163] q′ j =||v+u^r j ||-1 (28)

[0164] In formula (28), q j r is the change in length of the j-th structural bone along the axial direction. j It refers to the distribution of structural bones across the cross-section of the manipulator, for example, r. j It can be the coordinates of the distribution of the structural bones on the cross-section of the manipulator, v is the linear velocity of the manipulator's pose along the arc length of the reference line, and u is the angular velocity of the manipulator's pose along the arc length of the reference line.

[0165] In some embodiments, the length change q of the structural bone j (j is the structural bone number, j = 1, 2, 3, ..., m) can be related to drive information, for example, drive information can be information about the drive mechanism driving the movement of the structural bone. In some embodiments, the length change of the structural bone can also be related to the deformation of the manipulator. For example, the deformation of the manipulator can be the extension and retraction deformation of its structural bone. In some embodiments, a length boundary condition is applied to the structural bone, which includes the length change q of the structural bone at its end. j (L) equals the length driving quantity q aj With length deformation ε j The sum. The driving information of the structural bones includes the length driving amount. For example, the axial length changes of all structural bones in the manipulator are given by formula (29):

[0166] q(L)=q a +L all ε (29)

[0167] In formula (29), q(L)=[q1(L)q2(L)...q m (L)] T q represents the length of each structural bone at s = L. a =[q a1 q a2 ... q am ] T L represents the driving length of each bone structure. all Representing the total length of the structural bone, ε = [ε1 ε2 ... ε m ] T This represents the linear strain of expansion and contraction on each structural skeleton. For example, ε j It can be a percentage, L all ε j This represents the stretching deformation of the j-th structural bone.

[0168] Equations (20) to (29) constitute the mechanical model of the manipulator. In some embodiments, the driving length q of each structural bone is given. a Given the distributed external force f e Distributed torque I e External torque m e and external force n e Under these conditions, the end effector pose of the manipulator can be calculated. For example, in some embodiments, the current actual actuation amount of the manipulator, such as q, is obtained. a,bkb And end-effector virtual forces, such as f virtual Then let q a =q a,bkb n e =f virtual m e =0, distributed external force f e and distributed torque I e If it can be ignored, then f e =0, I e =0, solve the mechanical model of the manipulator to obtain the virtual end-effector pose (including virtual end-effector position and virtual end-effector orientation). For example, the virtual end-effector pose can be denoted as ( tc p tip (f virtual ), tc R tip (f virtual )),in, tc p tip (f virtual () represents the virtual location at the end. tc R tip (f virtual() represents the virtual posture of the end effector.

[0169] In some embodiments, the virtual position deformation of the end can be determined based on the virtual position of the end and the current actual position of the end, for example, such as Figure 12 As shown, the end-effector virtual deformation Δp of the manipulator is... virtual It can be determined by the following formula (30):

[0170]

[0171] Where, Δp virtual For the end virtual position deformation, This represents the current actual position of the terminal.

[0172] In some embodiments, the current end-effector stiffness of the manipulator can be determined based on the end-effector virtual force and the end-effector virtual position deformation, for example, by the following formula (31):

[0173] k stiffness =||f virtual || / ||Δp virtual || (31)

[0174] Where, k stiffness f is the current end stiffness. virtual For the virtual force at the end, Δp virtual This is a virtual position shape variable at the end.

[0175] In some embodiments, method 100 includes determining the forces acting on the end of the manipulator based on the current actual end position, the current ideal end position, and the current end stiffness. For example, the actual position deformation Δp of the end of the manipulator can be obtained based on the current actual end position and the current ideal end position using formula (18) as described in some of the embodiments above. load The current end-effector stiffness k can be obtained using formula (31) as described above. stiffness Based on Hooke's Law, the actual position deformation Δp at the end of the manipulator is utilized. load The current end stiffness can determine the force at the end, for example, the force at the end can be determined using the following formula (32):

[0176]

[0177] in, The force is at the end.

[0178] In some of the embodiments described above, formula (32) assumes the force direction at the end of the operating arm and the actual position deformation Δp at the end of the operating arm. loadThe parallel orientation avoids the uncertainty in force perception caused by the direction of force application, which stems from the anisotropic nature of the end-effector stiffness (the axial stiffness of the end-effector is infinite). However, for the perception of lateral forces at the end-effector, the resulting error in the perceived direction of the force is acceptable.

[0179] Continue reading Figure 9 In step 9011, based on the force on the target portion, the force conditions of the target portion of the operating arm are determined, including a first force condition and a second force condition. In some embodiments, the target portion is determined to satisfy the first force condition based on the fact that the force on the target portion is not greater than an external force threshold; the target portion is determined to satisfy the second force condition based on the fact that the force on the target portion is greater than an external force threshold.

[0180] In some embodiments, method 900 may further include determining the motion state of the operating arm, including a free motion state (e.g., a motion state without external load force) and a constrained state (e.g., a motion state under external load force).

[0181] Figure 10 A flowchart illustrating a method 1000 (hereinafter also referred to as "method 1000") for determining the motion state of a manipulator according to some embodiments of the present disclosure is shown. Method 1000 may be implemented or performed by hardware, software, or firmware. In some embodiments, method 1000 may be implemented by a surgical robot system (e.g., Figure 2 The surgical robot system shown Figure 5 The surgical robot system 500 shown is... Figure 15 The surgical robot system 1500 shown is executed. In some embodiments, method 1000 can be implemented as computer-readable instructions. These instructions can be executed by a general-purpose processor or a special-purpose processor (e.g., Figure 2 The control device 220 shown or Figure 15 The processor 1530 shown reads and executes data. For example, the control unit of a surgical robot system (e.g., Figure 2 The control device 220 shown may include a processor configured to execute method 1000. In some embodiments, these instructions may be stored on a computer-readable medium.

[0182] See Figure 10 In step 1001, the current actual drive amount of the manipulator is obtained. In some embodiments, this can be achieved using... Figure 9 The current actual drive amount of the manipulator is obtained in the same manner as in step 903. This has been described in detail in some of the above embodiments and will not be repeated here. In some embodiments, Figure 9 Step 903 in the process can be used directly. Figure 10 The current actual drive quantity of the operating arm is obtained in step 1001, so step 903 can be omitted in method 900.

[0183] See also Figure 10 In step 1003, the current theoretical drive amount of the manipulator is determined. In some embodiments, the current theoretical drive amount of the manipulator can be determined based on the current configuration parameters of the manipulator.

[0184] In some embodiments, method 1000 may include obtaining current shape information of the operating arm and determining current configuration parameters based on the current shape information.

[0185] In some embodiments, the manipulator includes a shape sensor, which can be used to obtain the current shape information of the manipulator, and the current configuration parameters of the manipulator can be determined based on the current shape information. For example, measurement signals can be received from the shape sensor, and the current shape information of the manipulator can be determined based on the measurement signals. In some embodiments, the measurement signals can include the curvature at multiple locations along the axial direction of the manipulator, and the current shape information of the manipulator can be determined based on the curvature. In some embodiments, the curvature at multiple locations can be continuously processed using a continuity algorithm to determine the current shape information of the manipulator. For example, the continuity algorithm can be an interpolation algorithm. In this way, the shape information of the manipulator determined based on the continuous curvature is more accurate.

[0186] In some embodiments, the number of shape sensors can be one or more. In some embodiments, the shape sensors can be uniformly or non-uniformly arranged on the manipulator. In some embodiments, the shape sensor includes a fiber Bragg grating sensor, which includes gratings located at multiple positions along the axial direction of the manipulator. Measurement signals can be obtained through the gratings, and the shape of the manipulator can be reconstructed based on the measurement signals. The current shape information of the manipulator can be obtained based on the reconstructed shape of the manipulator. Fiber Bragg grating sensors use optical measurement, have good electromagnetic compatibility, and can obtain accurate shape measurement data. The optical measurement method occupies less internal space in the deformable manipulator, making data acquisition more flexible. The overall shape of the manipulator can be obtained by setting shape sensors in the manipulator. Fiber Bragg grating sensors are existing medical imaging equipment in hospitals, can be reused, and have the advantage of low cost in data measurement. In some embodiments, the shape sensor can also be an electromagnetic sensor. The electromagnetic sensor can include an electromagnetic induction device and multiple electromagnetic reflectors discretely arranged at multiple positions on the manipulator. The electromagnetic induction device relies on the mutual inductance principle of electromagnetic fields to determine the pose information of the electromagnetic reflectors discretely arranged at multiple positions on the manipulator, reconstructs the shape of the manipulator based on the discrete pose information, and obtains the current shape information of the manipulator based on the reconstructed shape of the manipulator. In some embodiments, medical imaging equipment such as computed tomography (CT), magnetic resonance imaging (MRI), and stereoscopic vision can also be used to obtain the current shape information of the manipulator.

[0187] In some embodiments, method 1000 may include obtaining the current actual pose of the end effector of the manipulator and determining current position parameters based on the current actual pose of the end effector. In some embodiments, the current actual pose of the end effector of the manipulator may be obtained based on an electromagnetic sensor, as described in detail in some of the above embodiments and will not be repeated here. In some embodiments, the current actual pose of the end effector of the manipulator may also be obtained through a visual detection algorithm, as described in detail in some of the above embodiments and will not be repeated here.

[0188] In some embodiments, method 1000 may include determining current position parameters by minimizing the difference between the theoretical end-effector pose and the current actual end-effector pose based on the current actual pose of the end-effector and the kinematic model of the manipulator.

[0189] In some embodiments, the bending deformation of each component in the manipulator can be estimated based on the constant curvature assumption. Here, constant curvature means that the magnitude of the curvature κ(s) of the center curve of each component in the manipulator is a constant with respect to its arc length s. For example, as... Figure 3 As shown, in the i-th segment, the curvature change information of the segment undergoing bending deformation can be represented as θ. i θ i for about or Rotate to The required rotation angle. A single segment 300 can be described by a kinematic model. In some embodiments, the position of the end of the i-th segment (e.g., the fixed disk coordinate system {ie}) relative to the base disk coordinate system {ib} can be determined using formulas (1) to (5) as described in some of the embodiments above. ib p ie ,attitude ib R ie The segment parameter ψ of a single segment 300 i It can be determined based on formula (6) in some of the above embodiments. Therefore, the configuration parameters of the manipulator can be determined according to the working state of the manipulator (e.g., C1-C4 configuration). For example, the configuration parameter ψ of the manipulator can be determined based on formulas (9) to (12). For example, as... Figure 4 The configuration parameters of the shown manipulator 400 in configuration C3 can be expressed as follows:

[0190] In some embodiments, the bending deformation of the segments in the manipulator can be estimated based on the linear curvature assumption. Here, linear curvature means that the magnitude of the curvature κ(s) of the center curve of each segment in the manipulator varies linearly with its arc length variable s. For example, as... Figure 3 As shown, in the i-th segment, the curvature change information of the segment undergoing bending deformation can be represented as κ. i (s i ), κ i (s i It can be shown in the following formula (33):

[0191] κ i (s i ) = a i s i +b i (33)

[0192] Among them, κ i (s i ) represents the center curve of the i-th segment (e.g., the center curve corresponding to the virtual structural bone of the i-th segment) at an arc length s. i The magnitude of curvature at point a i and b i θ represents the slope and intercept of the curvature change of the i-th segment, respectively. i With κ i (s i The conversion relationship between them can be shown in the following formula (34):

[0193]

[0194] Among them, Li The virtual structural skeleton for the i-th segment (e.g., Figure 3 The length of the virtual structure bone 321 shown in the figure.

[0195] The segment parameter of a single segment 300 can be expressed as ψ. i It means that ψ i It can be shown in the following formula (35):

[0196] ψ i =[a i ,b i ,δ i ] T (35)

[0197] Where, δ i Let be the bending direction angle, representing the bending plane and in the i-th structural section. The included angle.

[0198] like Figure 3 The single segment 300 shown can be represented by a kinematic model. The position of the end of the i-th segment (fixed disk coordinate system {ie}) relative to the base disk coordinate system {ib}. ib p ie ,attitude ib R ie As shown in the following formulas (36) and (37):

[0199]

[0200] ib R ie = ib R i1 i1 R i2 i2 R ie (37)

[0201] Among them, L i The virtual structural skeleton for the i-th segment (e.g., Figure 3 The length of the virtual structural bone 221 shown in the figure, ib R i1 Let {i1} be the orientation of the bending plane coordinate system of the i-th segment relative to the base disk coordinate system {ib}. i1 R i2 Let be the orientation of the bending plane coordinate system 2{i2} of the i-th segment relative to the bending plane coordinate system 1{i1}. i2 R ie Let {i1} be the orientation of the fixed disk coordinate system {i2} of the i-th segment relative to the curved plane coordinate system {i3}. Similar to some embodiments described above, ib Ri1 , i1 R i2 and i2 R ie It can be determined based on formulas (3), (4) and (5) in some of the embodiments above.

[0202] Therefore, the configuration parameters of the manipulator can be determined based on its working state (e.g., C1-C4 configuration). For example, the configuration parameter ψ of the manipulator can be determined based on formulas (9) to (12). Figure 4 The configuration parameters of the shown manipulator 400 in configuration C3 can be expressed as follows:

[0203] In some embodiments, such as Figure 4 In the illustrated manipulator 400, the length of the first segment 4201 is significantly greater than the length of the second segment 4202; for example, L1 is 2 to 3 times the length of L2. Here, L1 represents the length of the virtual structural bone of the first segment 4201, and L2 represents the length of the virtual structural bone of the second segment 4202. Those skilled in the art should understand that the lengths of the first segment 4201 and the second segment 4202 can be configured according to the requirements of the actual application scenario. Under load, the first segment 4201 is closer to a linear curvature shape, and the shape error caused by estimating the bending deformation of the first segment 4201 based on the linear curvature assumption is also smaller. Similarly, under load, the second segment 4202 is closer to a constant curvature shape, and the shape error caused by estimating the bending deformation of the second segment 4202 based on the constant curvature assumption is also smaller. Therefore, in some embodiments, the bending deformation of the first segment 4201 can be estimated based on the linear curvature assumption, and the bending deformation of the second segment 4202 can be estimated based on the constant curvature assumption. The estimation of bending deformation of the segment based on the linear curvature assumption and the estimation of bending deformation of the segment based on the constant curvature assumption have been described in detail in the above embodiments, and will not be repeated here. Based on formula (35) in the above embodiments, the segment parameter of the first segment 4201 can be obtained as ψ1, and based on formula (6) in the above embodiments, the segment parameter of the second segment 4202 can be obtained as ψ2. ψ1 and ψ2 can be shown as shown in the following formulas (38) and (39), respectively:

[0204] ψ1=[a1,b1,δ1] T (38)

[0205] ψ2=[θ2,δ2] T (39)

[0206] Where a1 and b1 represent the slope and intercept of the curvature change of the first segment 4201, respectively, and δ1 represents the bending plane and the curve in the first segment 4201. The included angle, θ2, represents the angle in the second segment 4202. about or Rotate to The required rotation angle, δ2, represents the bending plane in the second component 4202 and The included angle.

[0207] like Figure 3 The single component 300 shown can be represented by a kinematic model. Based on formulas (36) and (37) in some of the above embodiments, the position of the end of the first component 4201 (fixed disk coordinate system {1e}) relative to the base disk coordinate system {1b} 1b P 1e ,attitude 1b R 1e The results can be shown in formulas (40) and (41) below, respectively.

[0208]

[0209] 1b R 1e = 1b R 11 11 R 12 12 R 1e (41)

[0210] Based on formulas (1) and (2) in some of the above embodiments, the position of the end of the second component 4201 (fixed disk coordinate system {2e}) relative to the base disk coordinate system {2b} 2b P 2e ,attitude 2b R 2e As shown in the following formulas (42) and (43):

[0211]

[0212] 2b R 2e = 2b R 21 21 R 22 22 R 2e (43)

[0213] in, 2b R 21 , 21 R 22 , 22 R 2e It can be determined based on formulas (3), (4) and (5) in some of the above embodiments.

[0214] Therefore, the configuration parameters of the manipulator can be determined based on its working state (e.g., C1-C4 configuration). For example, the configuration parameter ψ of the manipulator can be determined based on formulas (9) to (12). Figure 4 The configuration parameters of the shown manipulator 400 in configuration C3 can be expressed as follows:

[0215] In some embodiments, the shape of the manipulator is reconstructed based on the shape assumptions of the manipulator and the current actual pose of the end effector to determine the current configuration parameters of the manipulator. In some embodiments, the shape reconstruction problem of the manipulator is expressed as the following nonlinear optimization problem, as shown in Equation (44):

[0216]

[0217] in, For the configuration parameters of the manipulator that need to be optimized, This represents the current theoretical end-effector pose of the manipulator. This represents the current theoretical end-effector position of the manipulator. This indicates the current theoretical end-effector posture of the manipulator. This is represented as the current actual pose of the end effector of the manipulator. In some embodiments, This can be the current actual pose of the end effector of the manipulator, obtained based on a vision detection algorithm. In some embodiments, Based on the shape assumption of the operating arm, the shape assumption method in some of the above embodiments can be used to determine the shape.

[0218] In some embodiments, the current theoretical end-effector pose of the manipulator can be described based on shape assumptions about the manipulator and through a kinematic model. For example, as... Figure 4 The pose of a single segment (e.g., the first segment 4201 or the second segment 4202) in the illustrated manipulator 400 can be determined by combining a kinematic model with an estimate of the bending deformation of the segments in the manipulator. For example, the position of the end point of the i-th segment (e.g., the fixed disk coordinate system {ie}) relative to the base disk coordinate system {ib} can be obtained based on the segment parameters of a single segment. ib p ie ,attitude ib R ie The specific details have been described in detail in some of the above embodiments, and will not be repeated here. Furthermore, as... Figure 4 The end-effector pose of the entire manipulator 400 shown can be described by a kinematic model, thus allowing the theoretical end-effector pose to be obtained given the manipulator's configuration parameters. For example, as Figure 4The homogeneous transformation matrix of the surgical actuator coordinates {tip} at the end of the manipulator 400 relative to its sheath coordinate system {tc} can be expressed as shown in formula (45):

[0219]

[0220] in, Let represent the homogeneous transformation matrix of the surgical actuator coordinates {tip} obtained based on the kinematic model relative to its sheath coordinate system {tc}. This represents the theoretical end-effector pose of the manipulator 400 (e.g., the pose of the surgical actuator relative to its sheath coordinate system obtained through a kinematic model). This indicates the theoretical end-effector position of the manipulator 400 (e.g., the position of the surgical actuator coordinates relative to its sheath coordinate system obtained through a kinematic model). tc T 1b This represents the homogeneous transformation matrix of the base disk of the first component 4201 relative to the sheath coordinate system. 1b T 1e This represents the homogeneous transformation matrix of the fixed disk of the first component 4201 relative to the base disk of the first component 4201. 1e T 2b This represents the homogeneous transformation matrix of the base disk of the second component 4202 relative to the fixed disk of the first component 4201. 2b T 2e This represents the homogeneous transformation matrix of the fixed disk of the second component 3202 relative to the base disk of the second component 4202. 2e T tip This represents the homogeneous transformation matrix of the surgical actuator relative to the fixed disk of the second component 4202. In some embodiments, the surgical actuator is fixedly disposed at the distal end of the fixed disk of the second component 4202, therefore, 2e T tip It is known or predetermined. In some embodiments, the base plate of the second component 4202 is connected to the fixed plate of the first component 4201 via a first straight rod segment 4203, therefore, 1e T 2b It is known or predetermined, for example, in some embodiments, such as Figure 4 The first structural bone of the first segment 4201 and the second segment 4202 of the operating arm 400 shown are γ2 apart on the cross section of the operating arm 400, and the length of the first straight rod segment 4202 is L. r (That is, the distance between the base plate of the second component 4202 and the fixed plate of the first component 4201 can be approximated as L) r ),thus, 1e T 2b =Rot z (γ2)·Transz (L r In some embodiments, the base plate of the first component 4201 is connected to the second straight rod 4204, and the position of the sheath relative to the base plate of the first component 4201 is determined by the overall rotation angle of the operating arm. Sure, It can be obtained directly from the drive unit, therefore, tc T 1b It can be determined in advance.

[0221] In some embodiments, the configuration parameters of the manipulator to be optimized This can be obtained based on assumptions about the shape of the manipulator, such as the constant curvature assumption and / or linear curvature assumption described in the above embodiments. For example, as... Figure 4 The bending deformation of the first segment 4201 of the manipulator 400 shown is estimated based on the linear curvature assumption, and the bending deformation of the second segment 4202 is estimated based on the constant curvature assumption. The configuration parameters of the manipulator 400 in the C3 configuration can be expressed as follows: The required configuration parameter vector of the operator arm to be optimized. It can be represented as: The parameter to be estimated. Those skilled in the art will understand that, in cases such as... Figure 4 In the manipulator 400 shown, when estimating the bending deformation of each component based on the constant curvature assumption, the configuration parameter vector of the manipulator 400 to be optimized under the C3 configuration can be expressed as: Here are the parameters to be estimated. When estimating the bending deformation of each component based on the linear curvature assumption, the configuration parameter vector of the manipulator 400 in configuration C3 can be expressed as: These are the parameters to be estimated.

[0222] In some embodiments, the feed amount of the manipulator (e.g., the overall feed length of the manipulator, the feed length of a segment in the manipulator, or the feed length of a straight section in the manipulator) is provided by a drive unit (e.g., a linear drive mechanism that drives the linear feed of the manipulator). For example, such as Figure 4 In the C3 configuration of the illustrated manipulator 400, the feed length L1 of the first segment 4201 is provided by a drive unit (e.g., a linear drive mechanism for linear feed of the manipulator). In some embodiments, the manipulator has an elongated section (e.g., longer than 200 mm) between the drive unit and the sheath. This elongated section will twist under external load, thus affecting the overall rotation angle of the manipulator. The required driving rotation angle is provided by the drive unit (e.g., a rotary drive mechanism that drives the manipulator to rotate about its own central axis). Add an extra offset angle to the base Right now Therefore, exemplarily, such as Figure 4 The manipulator 400 shown, in configuration C3, when estimating the bending deformation of each component based on the constant curvature assumption, the required optimized configuration parameter vector of the manipulator can be expressed as: Let be the parameters to be estimated. When estimating the bending deformation of each component based on the linear curvature assumption, the configuration parameter vector of the manipulator to be optimized can be expressed as: Let be the parameters to be estimated. When estimating the bending deformation of the first component 4201 based on the linear curvature assumption, and estimating the bending deformation of the second component 4202 based on the constant curvature assumption, the required optimized manipulator configuration parameter vector can be expressed as: These are the parameters to be estimated.

[0223] In some embodiments, the formula (44) is used to obtain the following: This is determined as the current bit type parameter.

[0224] In some embodiments, method 1000 includes determining a current theoretical drive amount based on current configuration parameters. In some embodiments, method 1000 may determine the current theoretical drive amount based on current configuration parameters and the relationship between current configuration parameters and the drive amount of the manipulator.

[0225] In some embodiments, in at least one segment of the manipulator, the actuation amount of multiple structural bones has a known mapping relationship with the segment parameters. In some embodiments, the mapping relationship between the actuation amount of multiple structural bones and the segment parameters can be determined based on formula (7). In some embodiments, similar to a single segment, the actuation amount of each structural bone in each segment of the manipulator can be determined based on formula (7), and then the actuation signal of the actuation unit can be determined based on the actuation amount.

[0226] In some embodiments, for example Figure 4 The current state characteristic parameters of the manipulator 400 shown are estimated based on formula (44). The theoretical driving amount of at least one structural bone caused by changes in the shape of the manipulator can be calculated, such as the theoretical driving length of at least one structural bone.

[0227] For example, such as Figure 4In the illustrated manipulator 400, the deformation of the i-th segment is achieved by driving two sets of symmetrically distributed structural bones through a drive unit. In some embodiments, each set of symmetrically distributed structural bones may include two symmetrically distributed structural bones (e.g., the included angle between the two structural bones is π). In some embodiments, the deformation of the i-th segment can be achieved by driving the two sets of symmetrically distributed structural bones through two pairs of double-ended screw drive mechanisms. In some embodiments, the double-ended screw drive mechanism may include a double-ended screw and two threaded sliders located on the double-ended screw. When the double-ended screw rotates, the two threaded sliders located on the double-ended screw move in opposite directions at the same speed. The two threaded sliders drive the two symmetrically distributed structural bones to move in opposite directions at the same speed, causing the two symmetrically distributed structural bones to be pushed or pulled, thereby achieving the bending deformation of the segment. In some embodiments, the two sets of structural bones driven by the i-th segment are represented by structural bones numbered i1 and i2, respectively, and the theoretical driving amount of the structural bones can be denoted as: q shape,bkb =[q shape,11 ,q shape,12 ,q shape,21 ,q shape,22 In some embodiments, such as Figure 4 In the illustrated manipulator 400, the drive of the second component 4202 takes into account the coupling effect of the structural skeleton of the second component 4202 through the channel of the first component 4201, and the theoretical drive amount q of the structural skeleton. shape,bkb =[q shape,11 ,q shape,12 ,q shape,21 ,q shape,22 It can be shown in the following formula (46):

[0228]

[0229] Where, r i1 Let r be the distance from structural bone i1 in the i-th segment to the virtual structural bone. i2 Let β be the distance from structural bone i2 in the i-th segment to the virtual structural bone. i1 Let β be the angle between structural bone i1 and the first structural bone in the i-th segment. i2 γ is the angle between structural bone i2 and the first structural bone in the i-th segment, and γ is the offset angle of the second segment 4202 relative to the first segment 4201. To estimate the current configuration parameters of the manipulator based on formula (44) The estimated values ​​of the parameters to be estimated in the model are determined.

[0230] In some embodiments, in the i-th structural segment, the included angle between structural bones numbered i1 and i2 can be π / 2, and structural bone i1 can be taken as the first structural bone. Therefore, β 11 =0,β 21=0,β 12 =π / 2, β 22 =π / 2. In some embodiments, the offset angle γ of the second segment 4202 relative to the first segment 4201 is π / 4.

[0231] In some embodiments, the bending deformation of the first segment 4201 is estimated based on the linear curvature assumption, and the bending deformation of the second segment 4202 is estimated based on the constant curvature assumption. The parameters to be estimated can be obtained through formula (44). The estimated value can be obtained from The estimated value was calculated. The estimated value can be expressed as: Those skilled in the art should understand that, based on the assumption of constant curvature, the bending deformation of each component can be estimated directly using formula (44), and the parameters to be estimated can be obtained directly. The estimated value; based on the linear curvature assumption, the bending deformation of each component is estimated, and the parameters to be estimated can be obtained through formula (44). The estimated value can be obtained from The estimated value was calculated. The estimated value, The estimated value was calculated. The estimated value can be expressed as:

[0232] See also Figure 10 In step 1005, based on the current actual drive quantity and the drive quantity residual calculation model, the first drive quantity residual of the operating arm in the free movement state is determined. Step 1005 and Figure 9 The steps in step 905 correspond to the steps in step 905. In some embodiments, a method similar to step 905 can be used to determine the first driving amount residual of the operating arm in a free-movement state. The specific details have been described in detail in the above embodiments and will not be repeated here. In some embodiments, Figure 9 Step 905 in the middle can be used Figure 10 The first driving quantity residual of the operating arm in free motion state determined in step 1005 in method 900 can be omitted from step 905.

[0233] Continue reading Figure 10 In step 1007, a second drive quantity residual of the manipulator is determined based on the current actual drive quantity and the current theoretical drive quantity. In some embodiments, the second drive quantity residual of the manipulator may be the second drive quantity residual of the structural bone of the manipulator.

[0234] In some embodiments, the current theoretical actuation amount of the manipulator can be obtained through shape reconstruction of the manipulator. In some embodiments, the manipulator can be a continuum manipulator (e.g., Figure 4The illustrated manipulator arm 400 includes at least one segment, which comprises a fixing disc and multiple structural bones. The distal ends of the multiple structural bones are fixedly connected to the fixing disc, and the proximal ends of the multiple structural bones are connected to a drive unit. The current theoretical drive amount of the manipulator arm can be based on the current shape of the manipulator arm and the theoretical drive amount of the structural bones of the manipulator arm. For example, as shown... Figure 4 In the illustrated manipulator arm 400, the deformation of the i-th segment is achieved by a drive unit driving two sets of symmetrically distributed structural bones. The specific method by which the drive unit drives the bending deformation of the segment is consistent with some of the embodiments described above, and will not be repeated here. Figure 4 In the manipulator 400 shown, the two sets of structural bones in the i-th segment are represented by structural bones numbered i1 and i2, respectively. The theoretical driving amount of the representative structural bones can be denoted as q. shape,ij ,ij=11,12,21,22, the theoretical driving forces of multiple structural bones constitute the theoretical driving force vector of the structural bone, which can be denoted as: q shape,bkb =[q shape,11 ,q shape,12 ,q shape,21 ,q shape,22 In some embodiments, the manipulator's shape is reconstructed based on its shape information and / or end-effector pose to obtain its configuration parameters. These reconstructed configuration parameters can then be used to calculate the theoretical driving force (e.g., q) of multiple structural bones within the manipulator caused by changes in its shape. shape,ij ), the specific details of which will be described later. In some embodiments, the current theoretical driving vector of the structural bone (e.g., q) can be obtained. shape,bkb The current theoretical driving vector of the structural bone is used as the current theoretical driving vector of the manipulator; that is, the current theoretical driving vector of the manipulator can be expressed as q. shape,bkb .

[0235] In some embodiments, the current theoretical driving quantity q of the manipulator is obtained through shape reconstruction of the manipulator. shape,bkb At that time, the influence of factors such as the driving hysteresis and / or elastic tensile and compressive deformation of each structural bone was not taken into account. Therefore, there is a driving residual in each structural bone. In this disclosure, the driving residual calculated by the actual driving amount of the operating arm and the theoretical driving amount of the operating arm is denoted as the second driving residual. The second driving residual can be expressed as shown in the following formula (47):

[0236] q rsd,bkb =q a,bkb -q shape,bkb ,q rsd,bkb ≠0 (47)

[0237] For example, taking the manipulator that drives four symmetrically distributed structural bones as an example, q rsd,bkb =[q rsd,11 ,qrsd,12 ,q rsd,21 ,q rsd,22 ] T , where q rsd,ij =q ij -q shape,ij ,ij=11,12,21,22.

[0238] Continue reading Figure 10 In step 1009, the motion state of the operating arm is determined based on the first drive quantity residual and the second drive quantity residual. In some embodiments, the motion state includes a free motion state (e.g., a motion state without external load force) and / or a constrained state (e.g., a motion state under external load force).

[0239] In some embodiments, method 1000 includes obtaining the difference between the drive residual amount and the second drive residual, for example, as shown in the following formula (48):

[0240]

[0241] in, For the first driving quantity residual, q rsd,bkb For the second driving quantity residual, The difference that drives the residual.

[0242] For example, consider a manipulator that drives four symmetrically distributed structural bones:

[0243] q rsd,bkb =[q rsd,11 ,q rsd,12 ,q rsd,21 ,q rsd,22 ] T , in, Let be the difference in the driving residual of the ij-th structural bone. q rsd,ij =q ij -q shape,ij , The values ​​ij are calculated using a driving quantity residual calculation model (e.g., the SVR model), where ij = 11, 12, 21, 22.

[0244] In some embodiments, the difference between the first driving quantity residual and the second driving quantity residual is greater than the driving quantity residual threshold, which determines that the operating arm is in a constrained state at the current moment (e.g., the operating arm is moving under the drive of the driving unit and is subjected to an external load force at the current moment).

[0245] In some embodiments, the difference between the first drive quantity residual and the second drive quantity residual is not greater than the drive quantity residual threshold, and it is determined that the operating arm is in a free motion state at the current moment (e.g., the operating arm moves only under the drive of the drive unit and is not subjected to any external load force at the current moment).

[0246] In some embodiments, the difference in driving residuals is obtained based on formula (48), and the current manipulator is determined to be in a constrained state based on the difference in driving residuals of at least one structural bone being greater than the driving residual threshold. For example, taking a manipulator that drives four sets of symmetrically distributed structural bones as an example, when At that time, it is determined that the operating arm is in a constrained state. Let ξ be the difference in the driving residual of the ij-th structural bone. ij To drive the residual threshold.

[0247] In some embodiments, the difference in driving residuals is obtained based on formula (48), and it is determined that the manipulator is in a free motion state at the current moment, based on the fact that the difference in driving residuals of all structural bones is not greater than the driving amount residual threshold. For example, taking the arm body that drives four symmetrically distributed structural bones as an example, when At that time, it is determined that the operating arm is in a free movement state.

[0248] In some embodiments, each structural bone corresponds to a driving quantity residual threshold ξ. ij ξ ij The minimum value ξ ij，min The magnitude of this is related to the average prediction error of the driving quantity residual calculation model (e.g., the SVR model).

[0249] In some embodiments, method 900 may include determining that the target portion satisfies a first force condition in response to the manipulator being in a free-moving state.

[0250] In some embodiments, method 900 may include, in response to the operator arm being in a constrained state, performing steps 909 and 9011, determining the force on a target portion of the operator arm based on the current actual position and the current ideal position, and determining a force condition for the target portion based on the force on the target portion. In some embodiments, when the operator arm is in a constrained state, the target portion is determined to satisfy a first force condition based on the fact that the force on the target portion is not greater than an external force threshold; and the target portion is determined to satisfy a second force condition based on the fact that the force on the target portion is greater than the external force threshold.

[0251] Those skilled in the art should understand that when the manipulator is in a free-moving state, the target part of the manipulator can be considered to be unaffected by external load forces (i.e., the force on the target part is zero), satisfying the condition that the force on the target part is not greater than the external force threshold, thus meeting the first force condition. By determining the motion state of the manipulator, the force condition of the target part can be preliminarily determined. For example, when the manipulator is in a free-moving state, it can be quickly determined that the target part meets the first force condition, thus eliminating the need for complex force calculations on the target part and reducing the computational load. Those skilled in the art should also understand that determining the motion state of the manipulator by the difference in the drive residual of the manipulator can more accurately determine the force condition of the target part of the manipulator, thereby improving the control effect of the manipulator.

[0252] Continue reading Figure 1 In step 105, in response to the first force condition of the target part of the manipulator, in the first control mode of the manipulator, the drive signal of the manipulator is determined based on the current actual pose and the target pose.

[0253] In some embodiments, method 100 includes determining drive signals for the manipulator at predetermined intervals to achieve real-time control through multiple motion control cycles. In some embodiments, a current actual pose difference is determined based on the current actual pose and target pose of the target portion of the manipulator; and a drive signal for the manipulator is determined based on the current actual pose difference and the kinematic model of the manipulator. In some embodiments, the current actual pose and target pose of the target portion of the manipulator can be transformed to the same reference coordinate system, such as the world coordinate system or the manipulator base coordinate system (e.g., the sheath coordinate system). For example, based on the difference between the end-effector target pose and the end-effector current actual pose in the world coordinate system or the sheath coordinate system, the drive values ​​of multiple joints included in the manipulator in the current motion control cycle (or the drive values ​​of multiple corresponding motors controlling the movement of the manipulator) can be determined using an inverse kinematic numerical iterative algorithm of the manipulator kinematic model.

[0254] In some embodiments, the current actual pose difference includes position difference and attitude difference. For example, in the k-th motion control cycle, the current actual pose difference can be determined by the following formula (49):

[0255]

[0256] in, Let be the position difference of the target part of the manipulator during the k-th motion control cycle. Let be the attitude difference of the target part of the manipulator during the k-th motion control cycle. This represents the target position of the target portion of the manipulator during the k-th motion control cycle. The target posture of the target part of the manipulator during the k-th motion control cycle. This represents the current actual position of the target portion of the manipulator during the k-th motion control cycle. This represents the current actual posture of the target portion of the manipulator during the k-th motion control cycle. for and The angular error vector between them.

[0257] In some embodiments, the target portion includes the end effector of the manipulator, and the current actual pose includes the current actual pose of the end effector, which can be obtained, for example, through a visual detection algorithm. This represents the current actual position of the terminal. Given the current actual attitude of the terminal, we can therefore set... The end-effector pose can be determined based on control commands, for example... The current actual pose difference can be obtained based on formula (49).

[0258] In some embodiments, method 100 further includes determining a Cartesian space velocity based on the current actual pose difference, the Cartesian space velocity including Cartesian space linear velocity and Cartesian space angular velocity. In some embodiments, method 100 further includes: determining a Cartesian space linear velocity based on the position difference; and determining a Cartesian space angular velocity based on the pose difference. In some embodiments, the Cartesian space velocity can be determined using a PD (proportional and derivative) control law based on the position difference and the pose difference. For example, in the k-th motion control cycle, the Cartesian space velocity can be determined by the following formula (50):

[0259]

[0260] in, The Cartesian space velocity, v, of the k-th motion control cycle k Let ω be the Cartesian linear velocity of the k-th motion control cycle. k Let be the Cartesian space angular velocity of the k-th motion control cycle. Let be the position difference of the target part of the manipulator during the k-th motion control cycle. Let be the attitude difference of the target part of the manipulator during the k-th motion control cycle. The position difference of the target part of the manipulator during the (k-1)th motion control cycle. Let P be the attitude difference of the target part of the manipulator during the (k-1)th motion control cycle. v D is the linear velocity proportionality coefficient. v P is the linear velocity differential coefficient. ω D is the angular velocity proportionality coefficient. ωThe coefficients are the differential coefficients of the angular velocity. Those skilled in the art should understand that other control laws can be used to determine the Cartesian space velocity based on the position difference and attitude difference, such as proportional control laws, PID (proportional, integral, and derivative) control laws, etc.

[0261] In some embodiments, the configuration parameter space velocity of the manipulator can be determined based on the Cartesian space velocity. For example, in the k-th motion control cycle, the configuration parameter space velocity can be determined by the following formula (51):

[0262]

[0263] in, The positional parameter space velocity of the k-th motion control cycle. J is the Jacobian matrix J of the manipulator from configuration space to task space. xψ The inverse matrix, the Jacobian matrix J of the manipulator from the configuration space to the task space. xψ It can be determined based on the structure of the manipulator.

[0264] In some embodiments, the drive signal can be determined based on the spatial velocity of the configuration parameter space. For example, based on the mapping relationship between the spatial velocity of the configuration parameter space and the drive amount, the drive amount of the manipulator can be determined, and then the drive signal of the drive unit (e.g., motor) can be determined based on the drive amount. For example, in the k-th motion control cycle, the drive amount of the manipulator can be determined by the following formula (52):

[0265]

[0266] Where, Δq a,k J is the drive quantity of the operator arm in the k-th motion control cycle. qψ Let J be the Jacobian matrix of the manipulator from the configuration parameter space to the joint space, where Δt is the period of the motion control cycle. qψ It can be determined based on the structure of the manipulator.

[0267] Continue reading Figure 1 In step 106, in response to the second force condition of the target part of the manipulator, in the second control mode of the manipulator, the current theoretical pose of the target part of the manipulator is obtained, and the drive signal of the manipulator is determined based on the current theoretical pose and the target pose.

[0268] In some embodiments, method 100 includes determining drive signals for the manipulator at predetermined intervals to achieve real-time control through multiple motion control cycles. In some embodiments, method 100 further includes determining the current theoretical pose based on the previous pose of the target portion of the manipulator in a previous motion control cycle, the previous joint parameter space velocity, and the kinematic model of the manipulator.

[0269] In some embodiments, the previous pose includes the actual or theoretical pose of the target portion of the manipulator in the previous motion control loop. In some embodiments, incremental prediction can be performed based on the previous pose, the previous joint parameter space velocity, and the kinematic model of the manipulator to obtain the current theoretical pose of the target portion of the manipulator. For example, in the k-th motion control loop, the current theoretical pose of the end effector of the manipulator can be incrementally predicted from the end effector pose of the (k-1)-th motion control loop, for example, the current theoretical pose of the end effector can be determined by the following formula (53):

[0270]

[0271] Where Δt is the period of the motion control cycle, J xψ The Jacobian matrix for manipulating the robotic arm from configuration space to task space. Let be the spatial velocity of the configuration parameter in the (k-1)th motion control cycle, Δp be the position increment, and ΔR be the attitude increment. This represents the end position of the (k-1)th motion control cycle. This represents the final pose of the (k-1)th motion control cycle. This represents the current theoretical position of the end effector of the manipulator. This represents the current theoretical orientation of the end effector of the manipulator.

[0272] In some embodiments, method 100 may include determining a current theoretical pose difference based on the current theoretical pose and target pose of the target portion of the manipulator; and determining a drive signal for the manipulator based on the current theoretical pose difference and the kinematic model of the manipulator. In some embodiments, similar to some of the embodiments described above, method 100 may further include determining a Cartesian space velocity based on the current theoretical pose difference, whereby the Cartesian space velocity includes Cartesian space linear velocity and Cartesian space angular velocity. Specifically, the Cartesian space linear velocity is determined based on the position difference; and the Cartesian space angular velocity is determined based on the attitude difference. The configuration parameter space velocity of the manipulator may be determined based on the Cartesian space velocity. The drive signal may be determined based on the configuration parameter space velocity. For example, based on the mapping relationship between the configuration parameter space velocity and the drive quantity, the drive quantity of the manipulator may be determined, and then the drive signal of the drive unit (e.g., a motor) may be determined based on the drive quantity.

[0273] Figure 13A logic block diagram of a control method 1300 for a manipulator according to some embodiments of the present disclosure is shown. Figure 13 As shown, in some embodiments, the manipulator may employ, for example... Figure 4 The continuous robotic arm is shown. In some embodiments, the control method for the manipulator includes steps 1301 to 1329.

[0274] In some embodiments, in step 1301, during the k-th motion control cycle, the current actual pose of the target portion of the manipulator can be obtained based on a method similar to that described in some of the above embodiments. For example, a visual detection algorithm can be used to obtain the current actual pose of the end effector of the manipulator.

[0275] In some embodiments, in step 1303, during the k-th motion control cycle, the target pose of the manipulator, such as the end effector target pose, can be obtained based on a method similar to that described in some of the above embodiments.

[0276] In some embodiments, in step 1305, during the k-th motion control loop, the shape of the manipulator can be reconstructed based on the current actual pose of the end effector of the manipulator, using a method similar to that described in some of the above embodiments, to obtain the manipulator's configuration parameters, such as the current configuration parameters.

[0277] In some embodiments, in step 1307, during the k-th motion control loop, the current configuration parameters obtained from shape reconstruction can be used based on a method similar to that described in some of the above embodiments. The current theoretical drive quantity q of the manipulator can be obtained. shape,bkb .

[0278] In some embodiments, in step 1309, during the k-th motion control cycle, the current actual drive amount of the manipulator can be obtained based on a method similar to that described in some of the above embodiments. For example, the current actual drive amount q a,bkb .

[0279] In some embodiments, in step 1311, in the k-th motion control cycle, a method similar to that described in some of the above embodiments can be used to determine the first drive quantity residual of the manipulator in its free motion state based on the current actual drive quantity and the drive quantity residual calculation model of the manipulator. Based on the current actual drive quantity of the manipulator and the first drive quantity residual of the manipulator in its free motion state, the ideal drive quantity corresponding to the current free motion state of the manipulator can be determined, for example, the ideal drive quantity.

[0280] In some embodiments, in step 1313, during the k-th motion control cycle, the ideal pose of the target portion, such as the current ideal pose of the end effector, can be determined based on a method similar to that described in some of the above embodiments, using the ideal actuation amount of the manipulator. Based on the current actual position and the current ideal position of the target portion, the actual position deformation of the target portion, such as the actual position deformation Δp of the end effector, can be determined. load .

[0281] In some embodiments, in step 1315, during the k-th motion control cycle, the current end-effector stiffness of the manipulator can be determined based on a method similar to that described in some of the above embodiments. For example, the current end-effector stiffness k... stiffness .

[0282] In some embodiments, in step 1317, during the k-th motion control cycle, the forces acting on the target portion of the manipulator, such as the end effector, can be determined based on methods similar to those described in some of the above embodiments.

[0283] In some embodiments, in step 1319, during the k-th motion control cycle, the force conditions of the target part can be determined based on the force acting on the target part. For example, if the force at the end point is not greater than an external force threshold, it is determined that the end point of the manipulator meets the first force condition; if the force at the end point is greater than the external force threshold, it is determined that the end point of the manipulator meets the second force condition. In some embodiments, the motion state of the manipulator can also be determined. It can be determined that the target part meets the first force condition based on the manipulator being in a free motion state, or that the force on the target part is determined based on the manipulator being in a constrained state. Based on the force acting on the target part, the force conditions of the target part can be determined. These details have been described in detail in some of the above embodiments and will not be repeated here.

[0284] In some embodiments, in step 1321, during the k-th motion control cycle, in response to the first force condition of the target portion of the manipulator, in the first control mode of the manipulator, closed-loop control of the manipulator is performed based on the current actual pose of the target portion obtained in step 1301. For example, it can be set as follows: It also performs closed-loop control of the manipulator.

[0285] In some embodiments, in step 1323, in the k-th motion control cycle, in response to the second force condition of the target portion of the manipulator, under the second control mode of the manipulator, incremental prediction is performed based on the previous pose of the target portion of the manipulator, the previous joint parameter space velocity, and the kinematic model of the manipulator in the (k-1)-th motion control cycle to obtain the current theoretical pose of the target portion of the manipulator, so as to perform open-loop control of the manipulator based on the current theoretical pose of the target portion. For example, based on the end-effector pose of the (k-1)-th motion control cycle... and the spatial velocity of the configuration parameter in the (k-1)th motion control cycle. Incremental prediction is performed to obtain the current theoretical pose at the end of the k-th motion control cycle. It can make And the operating arm is controlled in an open-loop manner.

[0286] In some embodiments, in step 1325, in the k-th motion control loop, the pose difference (e.g., the current actual pose difference or the current theoretical pose difference) can be obtained based on a method similar to that in some of the embodiments described above, and the Cartesian space velocity can be determined based on the pose difference.

[0287] In some embodiments, in step 1327, during the k-th motion control cycle, the configuration parameter space velocity of the manipulator can be determined based on the Cartesian space velocity, using a method similar to that described in some of the above embodiments.

[0288] In some embodiments, in step 1329, in the k-th motion control cycle, the drive signal can be determined based on the configuration parameter space velocity, using a method similar to that described in some of the above embodiments.

[0289] In some embodiments, the target pose of the target portion of the manipulator is updated at predetermined intervals before or during each motion control cycle. In some embodiments, multiple motion control cycles are executed iteratively, and in each motion control cycle, some or all of the steps in the method according to some embodiments of this disclosure may be performed, such as... Figure 1 Some or all of the steps in the disclosed method 100, or Figure 9 Some or all of the steps in the method 900 disclosed in the paper, or Figure 10 Some or all of the steps in the method 1000 disclosed herein, or Figure 13Method 1300 disclosed herein includes some or all of the steps to control the movement of a manipulator to a target pose. In some embodiments, by iteratively executing multiple motion control loops, open-loop / closed-loop switching control of the target portion of the manipulator can be achieved. When the target portion of the manipulator meets a first force condition (e.g., the force on the target portion of the manipulator is not greater than an external force threshold), real-time closed-loop control of the manipulator is performed, which can improve the pose control accuracy of the manipulator. When the target portion of the manipulator meets a second force condition (e.g., the force on the target portion of the manipulator is greater than an external force threshold), open-loop control of the manipulator is performed, avoiding a large instantaneous bouncing motion of the manipulator when the force on the target portion of the manipulator suddenly disappears, thus improving the safety of the surgical operation. Those skilled in the art should understand that manipulator pose control achieved by the method of this disclosure can improve the trajectory tracking error of the manipulator and improve the safety of the surgical operation.

[0290] In some embodiments of this disclosure, a computer device is also provided, including a memory and a processor. The memory may be used to store at least one instruction, and the processor is coupled to the memory for executing the at least one instruction to perform some or all of the steps in the method of this disclosure, such as... Figure 1 Some or all of the steps in the disclosed method 100, or Figure 9 Some or all of the steps in the method 900 disclosed in the paper, or Figure 10 Some or all of the steps in the method 1000 disclosed herein, or Figure 13 Some or all of the steps in the method 1300 disclosed in the paper.

[0291] Figure 14 A schematic block diagram of a computer device 1400 according to some embodiments of the present disclosure is shown. See also Figure 14 The computer device 1400 may include a central processing unit (CPU) 1401, a system memory 1404 including random access memory (RAM) 1402 and read-only memory (ROM) 1403, and a system bus 1405 connecting the various components. The computer device 1400 may also include an input / output system 1406 and a mass storage device 1407 for storing an operating system 1413, application programs 1414, and other program modules 1415. The input / output system includes an input / output controller 1406 primarily consisting of a display 1408 and input devices 1409.

[0292] Mass storage device 1407 is connected to central processing unit 1401 via a mass storage controller (not shown) connected to system bus 1405. Mass storage device 1407 or computer-readable media provides non-volatile storage for computer devices. Mass storage device 1407 may include computer-readable media (not shown) such as hard disk or compact disc read-only memory (CD-ROM) drives.

[0293] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, flash memory or other solid-state storage technologies, CD-ROM, or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The aforementioned system memories and mass storage devices can be collectively referred to as memory.

[0294] Computer device 1400 can be connected to network 1412 via network interface unit 1411 connected to system bus 1405. System memory 1404 or mass storage device 1407 is also used to store one or more instructions. Central processing unit 1401 implements all or part of the steps of the methods in some embodiments of this disclosure by executing the one or more instructions, for example, Figure 1 Some or all of the steps in the disclosed method 100, or Figure 9 Some or all of the steps in the method 900 disclosed in the paper, or Figure 10 Some or all of the steps in the method 1000 disclosed herein, or Figure 13 Some or all of the steps in the method 1300 disclosed in the paper.

[0295] In some embodiments of this disclosure, a computer-readable storage medium is also provided, storing at least one instruction that is executed by a processor to cause a computer to perform some or all of the steps in the methods of some embodiments of this disclosure, such as... Figure 1 Some or all of the steps in the disclosed method 100, or Figure 9 Some or all of the steps in the method 900 disclosed in the paper, or Figure 10 Some or all of the steps in the method 1000 disclosed herein, or Figure 13Some or all of the steps in the method 1300 disclosed herein. Examples of computer-readable storage media include memory for computer programs (instructions), such as read-only memory (ROM), random access memory (RAM), compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage devices.

[0296] Figure 15 A schematic diagram of a surgical robot system 1500 according to some embodiments of the present disclosure is shown. In some embodiments of the present disclosure, see [link to relevant documentation]. Figure 15 The surgical robot system 1500 may include a surgical tool 1510 and a processor 1530. The surgical tool 1510 includes a manipulator arm 1511 and a surgical actuator 1515 disposed at the distal end of the arm body of the manipulator arm 1511. The processor 1530 is used to perform some or all of the steps in the methods of some embodiments of this disclosure, such as... Figure 1 Some or all of the steps in the disclosed method 100, or Figure 9 Some or all of the steps in the method 900 disclosed in the paper, or Figure 10 Some or all of the steps in the method 1000 disclosed herein, or Figure 13 Some or all of the steps in the method 1300 disclosed in the paper.

[0297] Note that the above are merely exemplary embodiments and technical principles of this disclosure. Those skilled in the art should understand that this disclosure is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this disclosure. Therefore, although this disclosure has been described in detail through the above embodiments, this disclosure is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this disclosure, the scope of which is determined by the scope of the appended claims.

Claims

1. A control method for a manipulator, characterized in that, include: Obtain the current actual pose of the target portion of the manipulator; Obtain the target pose of the target portion of the manipulator; Based on the fact that the force on the target part of the operating arm is not greater than the external force threshold, it is determined that the target part of the operating arm satisfies the first force condition. Based on the fact that the force on the target part of the operating arm is greater than the external force threshold, it is determined that the target part of the operating arm satisfies the second force condition; In response to the first force condition of the target portion of the manipulator, in the first control mode of the manipulator, the drive signal of the manipulator is determined based on the current actual pose and the target pose. as well as In response to the second force condition of the target portion of the manipulator, in the second control mode of the manipulator, Obtain the current theoretical pose of the target portion of the manipulator; Based on the current theoretical pose and the target pose, the drive signal of the manipulator is determined.

2. The method according to claim 1, characterized in that, The method further includes: Obtain the localization image; In the positioning image, multiple markers located on the manipulator are identified; and Based on the multiple identifiers, the current actual pose is determined.

3. The method according to claim 1, characterized in that, The operating arm is equipped with an electromagnetic sensor, and the method further includes: Obtain information from the electromagnetic sensor; and Based on the information from the electromagnetic sensor, the current actual pose is determined.

4. The method according to claim 1, characterized in that, The method further includes: The drive signal of the operating arm is determined at a predetermined period to achieve real-time control through multiple motion control cycles.

5. The method according to claim 4, characterized in that, The method further includes: The current theoretical pose is determined based on the previous pose of the target portion of the manipulator in the previous motion control cycle, the previous joint parameter space velocity, and the kinematic model of the manipulator. The previous pose includes the actual or theoretical pose of the target portion of the manipulator in the previous motion control cycle.

6. The method according to claim 1, characterized in that, Determining the drive signal for the manipulator based on the current actual pose and the target pose includes: Based on the current actual pose and the target pose, determine the current actual pose difference; and Based on the current actual pose difference and the kinematic model of the manipulator, the drive signal of the manipulator is determined.

7. The method according to claim 1, characterized in that, Determining the drive signal for the manipulator based on the current theoretical pose and the target pose includes: Based on the current theoretical pose and the target pose, determine the current theoretical pose difference; and Based on the current theoretical pose difference and the kinematic model of the manipulator, the drive signal of the manipulator is determined.

8. The method according to claim 6 or 7, characterized in that, The method includes: Determine the Cartesian space velocity based on the current actual pose difference or the current theoretical pose difference; Based on the Cartesian space velocity, determine the joint parameter space velocity of the manipulator; and The drive signal is determined based on the joint parameter spatial velocity.

9. The method according to claim 8, characterized in that, The current actual pose difference or the current theoretical pose difference includes position difference and attitude difference; the Cartesian space velocity includes Cartesian space linear velocity and Cartesian space angular velocity; the method further includes: Based on the position difference, determine the linear velocity in Cartesian space; and Based on the attitude difference, the Cartesian space angular velocity is determined.

10. The method according to claim 1, characterized in that, The method further includes: Obtain the current actual position of the target portion of the manipulator; Obtain the current actual drive quantity of the manipulator; Based on the current actual driving quantity and the driving quantity residual calculation model, the first driving quantity residual of the operating arm in the free motion state is determined; Based on the current actual driving quantity and the residual of the first driving quantity, the current ideal position of the operating arm is determined; Based on the current actual position and the current ideal position, determine the forces acting on the target portion; and Based on the forces acting on the target portion, the force conditions of the target portion are determined, and the force conditions include the first force condition and the second force condition.

11. The method according to claim 10, characterized in that, The method further includes: Based on the current actual driving quantity and the residual of the first driving quantity, determine the ideal driving quantity corresponding to the current free movement state of the operating arm; and Based on the ideal driving force corresponding to the current free movement state of the manipulator and the kinematic model of the manipulator, the current ideal position of the target part of the manipulator is determined.

12. The method according to claim 10, characterized in that, The method further includes: Determine the motion state of the operating arm, which includes a free motion state and a constrained state; In response to the operating arm being in the constrained state: Based on the current actual position and the current ideal position, determine the force on the target part; Based on the forces acting on the target portion, the force conditions of the target portion are determined, and the force conditions include the first force condition and the second force condition; as well as In response to the operating arm being in the free movement state, it is determined that the target portion satisfies the first force condition.

13. The method according to claim 12, characterized in that, The method further includes: Determine the current theoretical drive quantity of the manipulator; Based on the current actual drive quantity and the current theoretical drive quantity, determine the second drive quantity residual of the operating arm; and The motion state of the operating arm is determined based on the first drive quantity residual and the second drive quantity residual.

14. The method according to claim 13, characterized in that, Determining the motion state of the operating arm includes: Based on the fact that the difference between the first drive quantity residual and the second drive quantity residual is greater than the drive quantity residual threshold, it is determined that the operating arm is in the constrained state; and Based on the fact that the difference between the first drive quantity residual and the second drive quantity residual is not greater than the drive quantity residual threshold, it is determined that the operating arm is in the free motion state.

15. The method according to claim 10, characterized in that, The method further includes: Obtain the current stiffness of the target portion; and Based on the current actual position, the current ideal position, and the current stiffness, the force on the target part is determined.

16. The method according to claim 15, characterized in that, The method further includes: Determine the actual position deformation of the target part based on the current actual position and the current ideal position; Obtain the current stiffness of the target portion corresponding to the deformation direction at the actual position; and Based on the actual position deformation and the current stiffness, the force on the target part is determined.

17. The method according to claim 13 or 14, characterized in that, The manipulator includes at least one segment, the segment including a fixing plate and multiple structural bones, the distal ends of the multiple structural bones being fixedly connected to the fixing plate, and the proximal ends of the multiple structural bones being connected to a drive unit; the target portion includes the end of the manipulator; the current actual position includes the current actual position of the end, and the current ideal position includes the current ideal position of the end; The current actual driving amount includes the current actual driving amount of the multiple structural bones; the current theoretical driving amount includes the theoretical driving amount of the structural bones of the manipulator based on the current shape of the manipulator.

18. The method according to claim 17, characterized in that, The method further includes: Obtain the average end stiffness of the end of the operating arm; Based on the current actual position of the end effector, the current ideal position of the end effector, and the average stiffness of the end effector, determine the virtual end force of the end effector of the manipulator; and Based on the current actual driving amount of the manipulator, the virtual force at the end of the manipulator, and the mechanical model of the manipulator, the virtual position deformation at the end of the manipulator is determined, wherein the mechanical model is determined based on the distribution of the multiple structural bones of the manipulator on the cross-section of the manipulator and the physical properties of the multiple structural bones.

19. The method according to claim 18, characterized in that, The method further includes: Based on the virtual end force and the virtual end position deformation, determine the current end stiffness of the manipulator's end; and The force on the end of the operating arm is determined based on the current actual position of the end, the current ideal position of the end, and the current stiffness of the end.

20. The method according to claim 1, characterized in that, The target portion includes the end effector of the manipulator; the current actual pose includes the current actual pose of the end effector; the target pose includes the target pose of the end effector; and the current theoretical pose includes the current theoretical pose of the end effector.

21. The method according to claim 1, characterized in that, The manipulator arm includes at least one segment, the segment including a fixing plate and multiple structural bones, the distal ends of the multiple structural bones being fixedly connected to the fixing plate, and the proximal ends of the multiple structural bones being connected to the drive unit. Determining the drive signal for the manipulator includes: Determine the driving amount of the multiple structural bones; and Based on the driving amount of the multiple structural bones, the driving signal for the driving unit is determined.

22. A surgical robot system, characterized in that, include: Surgical instruments, the surgical instruments including an operating arm and an end effector disposed at the distal end of the arm body of the operating arm; as well as A processor for performing the method as described in any one of claims 1-21.

23. A computer device, characterized in that, The computer device includes: Memory for storing at least one instruction; and A processor, coupled to the memory and configured to execute the at least one instruction to perform the method as described in any one of claims 1-21.

24. A computer-readable storage medium for storing at least one instruction, characterized in that, When the at least one instruction is executed by the computer, it causes the robot system to perform the method as described in any one of claims 1-21.

Images