Control method for an arm body and surgical robot system

Abstract

The present disclosure relates to the field of robot control, and discloses a control method for an arm body and a surgical robot system. The control method for the arm body comprises: obtaining a current actual driving amount of the arm body; determining a current theoretical driving amount of the arm body; determining a first driving amount residual of the arm body in a free motion state based on the current actual driving amount and a driving amount residual calculation model; determining a second driving amount residual of the arm body based on the current actual driving amount and the current theoretical driving amount; and performing a control operation based on the first driving amount residual and the second driving amount residual.

Description

Technical Field

[0001] This disclosure relates to the field of medical devices, and more particularly to a control method for an arm and a surgical robot system. Background Technology

[0002] In recent years, many surgical robot systems using robotic arms have been developed for cardiovascular surgery, neurosurgery, and endoscopic surgery. During surgical procedures, the arm may interact with the patient's tissues, and active compliance control of the arm is an important task for improving surgical safety. Summary of the Invention

[0003] In some embodiments, this disclosure provides a control method for an arm body, comprising: obtaining the current actual driving quantity of the arm body; determining the current theoretical driving quantity of the arm body; determining a first driving quantity residual of the arm body in a free motion state based on the current actual driving quantity and a driving quantity residual calculation model; determining a second driving quantity residual of the arm body based on the current actual driving quantity and the current theoretical driving quantity; and performing a control operation based on the first driving quantity residual and the second driving quantity residual.

[0004] In some embodiments, this disclosure provides a surgical robot system, including: a surgical instrument including an arm and a surgical actuator disposed at the end of the arm; and a processor for performing a method according to any one of the embodiments of this disclosure.

[0005] In some embodiments, this disclosure provides a computer device including: a memory for storing at least one instruction; and a processor coupled to the memory and configured to execute at least one instruction to perform a method according to any of some embodiments of this disclosure.

[0006] In some embodiments, this disclosure provides a computer-readable storage medium for storing at least one instruction, which, when executed by a computer, causes a robot system to perform a method as described in any of some embodiments of this disclosure. Attached Figure Description

[0007] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments of this disclosure will be briefly introduced below. The accompanying drawings described below only show some embodiments of this disclosure. For those skilled in the art, other embodiments can be obtained based on the content of the embodiments of this disclosure and these drawings without creative effort.

[0008] Figure 1 A schematic diagram of an arm control system according to some embodiments of the present disclosure is shown;

[0009] Figure 2 A schematic diagram of the arm body's structure according to some embodiments of the present disclosure is shown;

[0010] Figure 3 This diagram shows a structural schematic of an arm body according to some embodiments of the present disclosure;

[0011] Figure 4 A flowchart illustrating a control method for an arm body according to some embodiments of the present disclosure is shown;

[0012] Figure 5 A schematic diagram of an arm equipped with a shape sensor according to some embodiments of the present disclosure is shown;

[0013] Figure 6 A schematic diagram showing the acquisition of the end-effector pose in a surgical robot system according to some embodiments of the present disclosure;

[0014] Figure 7 A schematic diagram is shown of a positioning tag including multiple pose markers and multiple angle markers according to some embodiments of the present disclosure;

[0015] Figure 8 A schematic diagram showing a positioning tag disposed on the peripheral side of the distal end of the arm body and formed into a cylindrical shape according to some embodiments of the present disclosure;

[0016] Figure 9 A schematic diagram showing the end-effector pose of the arm body obtained by a vision detection algorithm based on a binocular endoscope according to some embodiments of the present disclosure;

[0017] Figure 10 A logic block diagram of active compliance control of an arm body according to some embodiments of the present disclosure is shown;

[0018] Figure 11A This diagram shows the difference curves of the drive residual during the active compliance control process of the arm body when a human hand applies a load according to some embodiments of the present disclosure;

[0019] Figure 11B This shows the difference curves of the drive residuals during the active compliant control process of the arm body under straight channel constraints according to some embodiments of the present disclosure;

[0020] Figure 12 A schematic block diagram of a computer device according to some embodiments of the present disclosure is shown;

[0021] Figure 13 A schematic diagram of a surgical robot system according to some embodiments of the present disclosure is shown. Detailed Implementation

[0022] To make the technical problems solved by this disclosure, the technical solutions adopted, and the technical effects achieved clearer, the technical solutions of the embodiments of this disclosure will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely exemplary embodiments of this disclosure, and not all embodiments.

[0023] In the description of this disclosure, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings, are only for the convenience of describing this disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this disclosure. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this disclosure, it should be noted that unless otherwise expressly specified and limited, the terms "installed," "connected," "coupled," and "coupled" should be interpreted broadly. For example, they can refer to fixed connections or detachable connections; mechanical connections or electrical connections; direct connections or indirect connections through an intermediate medium; and internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this disclosure according to the specific circumstances.

[0024] Those skilled in the art will understand that the embodiments of this disclosure can be applied to deformable arms mounted on mechanical devices (e.g., surgical robots) operating in various environments, including but not limited to the surface, underground, underwater, space, and within living organisms. In this disclosure, the end closer to the operator (e.g., a doctor) is defined as the proximal end, proximal or rear end, or rear portion, and the end closer to the object of work (e.g., a surgical patient) is defined as the distal end, distal or front end, or front portion. In this disclosure, the arm body may include a distal end portion at the distal end, and an end-effector (e.g., an imaging device, a surgical actuator, etc.) may be mounted on the distal end portion. For clarity, in this disclosure, the arm body end refers to the most distal portion of the arm body and its end-effector. For example, when an end-effector (e.g., an imaging device, a surgical actuator, etc.) is mounted on the distal end portion of the arm body, the arm body end can refer to the end-effector; when no end-effector (e.g., an imaging device, a surgical actuator, etc.) is mounted on the distal end portion of the arm body, the arm body end refers to the distal end portion of the arm body.

[0025] In this disclosure, the term "position" refers to the location of an object or a portion of an object in three-dimensional space (e.g., three translational degrees of freedom can be described using variations in Cartesian X, Y, and Z coordinates, such as three translational degrees of freedom along the Cartesian X, Y, and Z axes, respectively). In this disclosure, the term "pose" refers to the rotational setting of an object or a portion of an object (e.g., three rotational degrees of freedom, which can be described using roll, pitch, and yaw). In this disclosure, the term "pose" refers to a combination of the position and pose of an object or a portion of an object, which can be described, for example, using six parameters from the six degrees of freedom mentioned above.

[0026] Figure 1 A schematic diagram of an arm control system 100 according to some embodiments of the present disclosure is shown. For example... Figure 1 As shown, the arm control system 100 may include at least one arm 110, an arm shape sensing device 120, and a control device 130. At least one arm 110 and the arm shape sensing device 120 are communicatively connected to the control device 130. In some embodiments, such as Figure 1 As shown, the control device 130 can be used to control the movement of at least one arm 110 to adjust the pose of at least one arm 110, coordinate with each other, etc. In some embodiments, at least one arm 110 may include a distal arm end portion 140 at its distal end. The control device 130 can control the movement of at least one arm 110 so that the distal arm end portion 140 moves to a desired target pose (including target position and target orientation). Those skilled in the art will understand that the arm control system 100 can be applied to surgical robot systems, such as laparoscopic surgical robot systems. For example, an end effector 150 may be disposed at the distal end of the distal arm end portion 140, and the end effector 150 may be, for example, a surgical actuator, such as... Figure 1 As shown. It should be understood that the arm control system 100 can also be applied to dedicated or general-purpose robot systems in other fields (e.g., logistics, industrial manufacturing, etc.).

[0027] In this disclosure, the control device 130 can be communicatively connected to at least one drive unit 160 (e.g., a motor) of the arm 110 and send drive signals to the drive unit 160, thereby enabling the drive unit 160 to control at least one arm 110 to move to a corresponding target pose based on the drive signals. For example, the drive unit 160 controlling the movement of the arm 110 can be a servo motor, which can receive instructions from the control device 130 to control the movement of the arm 110. The control device 130 can also be communicatively connected, for example, to a sensor coupled to the drive unit 160 via a communication interface to receive motion data of the arm 110 and monitor the motion state of the arm 110. In one example of this disclosure, the communication interface can be a CAN (Controller Area Network) bus communication interface, which enables the control device 130 to communicate with the drive unit 160 and the sensor via the CAN bus.

[0028] In some embodiments, at least one arm body 110 may include a deformable robotic arm, such as a multi-degree-of-freedom robotic arm composed of multiple joints, such as a robotic arm that can achieve 6 degrees of freedom of movement, or, for example, a deformable continuum robotic arm.

[0029] Figure 2 A schematic diagram of a segment 200 of an arm body according to some embodiments of the present disclosure is shown. The arm body (e.g., Figure 1 Arm 110 shown Figure 3 The arm shown is 300mm long. Figure 6 The surgical execution arm 610 and visual guidance arm 620 shown are either Figure 13 The arm body 1311 shown may include at least one deformable segment 200. For example... Figure 2 As shown, the deformable segment 200 includes a fixed disk 210 and multiple structural bones 220. A first end of each structural bone 220 is fixedly connected to the fixed disk 210, and a second end is connected to a drive unit (not shown). In some embodiments, the fixed disk 210 may be, but is not limited to, a ring-shaped structure, a disc-shaped structure, etc., and its cross-section may be circular, rectangular, polygonal, or various other shapes.

[0030] In some embodiments, the structural skeleton 220 in the deformable segment 200 can be made of an elastic material, possessing a certain degree of flexibility. For example, the material of the structural skeleton 220 can be a hyperelastic alloy, a gas / liquid cavity, a shape memory alloy, a polymer structural material, such as a nickel-titanium alloy. Based on the elastic properties of the structural skeleton 220, the deformable segment 200 can undergo shape changes when subjected to external forces and / or the driving action of the driving unit (e.g., push-pull action). The shape change of the deformable segment 200 can manifest as bending deformation, stretching deformation, or torsional deformation, etc. For example, the driving unit drives the structural skeleton 220 to bring the segment 200 into a position such as... Figure 2 The bending state is shown. In some embodiments, the second end of the multiple structural bones 220 passes through the base plate 230 and is connected to the driving unit, so that the driving unit drives the structural bones 220 to change the shape of the deformable segment 200. In some embodiments, similar to the fixed plate 210, the base plate 230 may be, but is not limited to, a ring structure, a disc structure, etc., and the cross-section may be a circle, a rectangle, a polygon, etc. In some embodiments, the driving unit may include a linear motion mechanism, a driving segment, or a combination of both. The linear motion mechanism may be connected to the multiple structural bones 220 to push or pull the multiple structural bones 220, thereby driving the segment 200 to bend. The driving segment may include a fixed plate and multiple structural bones, wherein one end of the multiple structural bones is fixedly connected to the fixed plate. The other end of the multiple structural bones of the driving segment is connected to or integrally formed with the multiple structural bones 220, so as to drive the bending of the deformable segment 200 by bending the driving segment.

[0031] In some embodiments, a spacer disk 240 is further included between the fixed disk 210 and the base disk 230, through which multiple structural bones 220 pass. Similarly, the drive segment may also include a spacer disk.

[0032] Figure 3 A schematic diagram of the structure of an arm 300 according to some embodiments of the present disclosure is shown. For example... Figure 3 As shown, the arm body 300 is a continuous robotic arm, which may include a distal end 310 and a main body 320. The main body 320 may include one or more segments, such as a first segment 3201 and a second segment 3202. In some embodiments, the structures of the first segment 3201 and the second segment 3202 may be consistent with... Figure 2 The shown component 200 is similar. In some implementations, such as... Figure 3 As shown, the main body 320 of the arm also includes a first straight rod segment 3203 located between the first segment 3201 and the second segment 3202. The first end of the first straight rod segment 3203 is connected to the base plate of the second segment 3202, and the second end is connected to the fixing plate of the first segment 3201. In some embodiments, such as... Figure 3 As shown, the arm body 320 also includes a second straight rod segment 3204, the first end of which is connected to the base plate of the first component 3201. In some embodiments, the distal end portion 310 of the arm body is located at the distal end of the arm body 320. In some embodiments, the distal end portion 310 of the arm body may include a columnar portion located at the distal end of the arm body 320. In some embodiments, the distal end of the distal end portion 310 of the arm body may be provided with an end effector (e.g., a surgical actuator, etc.) 330.

[0033] In some embodiments, the arm body 300 and its constituent segments can be described by a kinematic model. In some embodiments, the structure of each segment can be specifically as follows: Figure 2 The shown component is 200. (As shown in the image) Figure 2 As shown, the base coordinate system The base plate 230, which is attached to the i-th (i = 1, 2, 3...) segment, has its origin located at the center of the base plate 230, and its XY plane coincides with the plane of the base plate 230. Pointing from the center of base plate 230 to the first structural bone (the first structural bone can be understood as any one of the multiple structural bones 220 designated as a reference). Curved plane coordinate system. Its origin coincides with the origin of the base coordinate system, and the XY plane coincides with the bending plane. and Coincident. Fixed disk coordinate system. The origin of the fixed disk 210 attached to the i-th segment is located at the center of the fixed disk 210, and the XY plane coincides with the plane of the fixed disk 210. Pointing from the center of fixed plate 210 to the first structural bone. Curved plane coordinate system. Its origin is located at the center of the fixed disk 210, and the XY plane coincides with the bending plane. and coincide.

[0034] In some embodiments, such as Figure 2 The single segment 200 shown can be described by a kinematic model. The position of the i-th segment end (e.g., in the fixed disk coordinate system {ie}) relative to the base disk coordinate system {ib}. ib p ie ,attitude ib R ie As shown in formulas (1) and (2) below:

[0035]

[0036] ib R ie = ib R i1 i1 R i2 i2 R ie (2)

[0037] Among them, L i The virtual structural skeleton for the i-th segment (e.g., Figure 2 The length of the virtual structure bone 221 shown in the figure; θ i Let be the bending angle of the i-th structural segment, indicating that in the i-th structural segment, about or Rotate to Required rotation angle; δ i Let be the bending direction angle of the i-th segment, and let represent the bending plane and in the i-th segment. The included angle; ib R i1 Let {i1} be the orientation of the bending plane coordinate system 1{i1} of the i-th segment relative to the base disk coordinate system {ib}; i1 R i2 Let 2{i2} be the orientation of the bending plane coordinate system 2{i2} of the i-th segment relative to the bending plane coordinate system 1{i1}; i2 R ie Let {ie} be the orientation of the fixed disk coordinate system {i2} of the i-th segment relative to the curved plane coordinate system {i2}.

[0038] ib R i1 , i1 R i2 and i2 R ie It can be determined based on the following formulas (3), (4) and (5):

[0039]

[0040]

[0041]

[0042] like Figure 2 The segment parameter ψ of the single segment 200 shown i It can be determined based on the following formula (6):

[0043] ψ i =[θ i ,δ i ] T (6)

[0044] In some embodiments, the driving amount of multiple structural bones has a known mapping relationship with the segmental parameters. Based on the target segmental parameters and the mapping relationship, the driving amount of the multiple structural bones can be determined. The driving amount of the multiple structural bones can be understood as moving a single segment from its initial state (e.g., θ) i =0) The length of the structural bone subjected to push or tension when bent to the target bending angle. In some embodiments, the mapping relationship between the driving amount of multiple structural bones and the joint parameters can be determined based on the following formula (7):

[0045] q ij =-r ij θ i cos(δ i -βij (7)

[0046] Where, q ij r is the driving force of the j-th structural bone in the i-th segment. ij β is the distance from the j-th structural bone in the i-th segment to the virtual structural bone. ij Let be the angle between the j-th structural bone and the first structural bone in the i-th segment. The driving signal of the driving unit can be determined based on the driving amount of multiple structural bones.

[0047] In some embodiments, based on the overall rotation angle of the arm body 300, the feed amount of the arm body (e.g., the overall feed length of the arm body, the feed length of the segments in the arm body, or the feed length of the straight rod segment in the arm body) and the individual segments that make up the arm body (e.g., Figure 3 The segmental parameters of the first segment 3201 and the second segment 3202 shown can determine the shape characteristic parameters of the arm body. For example, the shape characteristic parameters of the arm body can be determined based on the following formula (8):

[0048]

[0049] in, Let d be the overall rotation angle of the arm body, and d be the overall feed length of the arm body. In some embodiments, the overall rotation angle of the arm body... The overall feed length d of the arm body is provided by the drive unit; for example, the overall feed length d of the arm body is provided by a linear drive mechanism that drives the linear feed of the arm body, and the overall rotation angle of the arm body... Provided by a rotary drive mechanism that drives the arm to rotate around its own central axis.

[0050] Those skilled in the art should understand that the arm body has different shape characteristic parameters in different working states. For example, Figure 3 The arm 300 shown includes at least four working states, which correspond to four configurations of the arm 300, and can be denoted as configurations C1-C4. The four working states of the arm 300 are described below:

[0051] First working state (C1 position): Only the second component 3202 participates in the pose control of the control device (for example, only the second component 3202 enters the workspace), and the shape characteristic parameters of the arm body 300 at this time are as shown in the following formula (9):

[0052]

[0053] Where, ψ c1 These are the shape characteristic parameters of the arm body 300 in the first working state. L1 is the overall rotation angle of the arm body 300, L2 is the feed length of the second component 3202, and L2 is related to... Figure 2 In the structure shown in section 200, L t The physical meaning is the same. ψ2 is the segment parameter of the second segment 3202. ψ2 can be determined by the above formula (6).

[0054] Second working state (C2 position): The second component 3202 and the first straight segment 3203 participate in the pose control of the control device (for example, the second component 3202 is fully in the working space, and the first straight segment 3203 is partially in the working space). At this time, the shape characteristic parameters of the arm body 300 are as shown in the following formula (10):

[0055]

[0056] Where, ψ c2 L represents the shape characteristic parameters of the arm body 300 in the second working state. r This is the feed length of the first straight segment 3203.

[0057] The third working state (C3 position): the second component 3202, the first straight segment 3203 and the first component 3201 participate in the position control of the actuator (for example, the second component 3202 is fully in the workspace, the first straight segment 3203 is fully in the workspace, and the first component 3201 is partially in the workspace). At this time, the shape characteristic parameters of the arm body 300 are as shown in the following formula (11):

[0058]

[0059] Where, ψ c3 For the shape characteristic parameters of the arm body 300 in the third working state, L1 is the feed length of the first component 3201, and L1 is related to... Figure 2 In the structure shown in section 200, L t The physical meanings are the same. ψ1 is the segment parameter of the first segment 3201, and ψ2 is the segment parameter of the second segment 3202. ψ1 and ψ2 can be determined by the above formula (6).

[0060] Fourth working state (C4 position): The second component 3202, the first linear segment 3203, the first component 3201, and the second linear segment 3204 participate in the position control of the actuator (for example, the second component 3202 is fully in the working space, the first linear segment 3203 is fully in the working space, the first component 3201 is fully in the working space, and the second linear segment 3204 is partially in the working space). At this time, the shape characteristic parameters of the arm body 300 are as shown in the following formula (12):

[0061]

[0062] Where, ψ c4L represents the shape characteristic parameters of the arm body 300 in the fourth working state. s This is the feed length for the second straight segment 3204.

[0063] In some embodiments, similar to a single segment, the driving amount of each structural bone of each segment of the arm body can be determined based on formula (7), and then the driving signal of the driving unit can be determined based on the driving amount.

[0064] Figure 1 In the arm control system 100, there is an arm shape sensing device 120, which is used to sense feature information related to the arm shape features to obtain the shape feature parameters of the arm.

[0065] In some embodiments, the arm shape sensing device may include a shape sensor that collects arm shape information, and the shape of the arm can be obtained based on the shape sensor to obtain the shape feature parameters of the arm. For example, a measurement signal can be received from the shape sensor, and the arm shape can be reconstructed based on the measurement signal to obtain the shape feature parameters of the arm. Details will be described later.

[0066] In some embodiments, the arm shape sensing device may include one or more shape sensors, such as fiber Bragg grating sensors extending along the length of the arm, for acquiring the shape of the arm. Shape characteristic parameters of the arm can be obtained based on the sensing signals from the fiber Bragg sensors.

[0067] In some embodiments, the arm shape sensing device may include an end-effector pose sensing device for acquiring the end-effector pose of the arm. The end-effector pose of the arm can be obtained based on the end-effector pose, and arm shape reconstruction can be performed based on the end-effector pose to obtain shape feature parameters of the arm. In this disclosure, the end-effector pose refers to the pose of the end of the arm. Those skilled in the art will understand that when an end-effector instrument (e.g., a surgical actuator) is mounted on the distal end of the arm, the end-effector pose can be the pose of the end-effector instrument, such as the pose of the surgical actuator; when no end-effector instrument (e.g., a surgical actuator) is mounted on the distal end of the arm, the end-effector pose can be the pose of the distal end of the arm.

[0068] In some embodiments, the end-effector pose sensing device includes an electromagnetic sensor disposed at the end of the arm and used to sense the end-effector pose. In some embodiments, the control device can receive information from the electromagnetic sensor and determine the current end-effector pose of the arm based on the information from the electromagnetic sensor. In some embodiments, the end-effector pose sensing device may further include an image acquisition device, which may include, but is not limited to, a dual-lens image acquisition device or a single-lens image acquisition device, such as a binocular or monocular camera. In some embodiments, the image acquisition device can be used to acquire positioning images to obtain the current end-effector pose of the arm based on the positioning images. Details will be described later.

[0069] Figure 1 In this system, the arm control system 100 includes a control device 130. In some embodiments, the control device 130 can reconstruct the shape of the arm (e.g., a continuous robotic arm) based on sensing information from a shape sensing device 120 (e.g., a shape sensor, an end-effector pose sensing device). In some embodiments, shape reconstruction aims to reproduce the shape of the arm (e.g., the 3D shape of a continuous robotic arm) as accurately as possible based on the sensing information from the arm shape sensing device, regardless of whether the arm is under no external load or under external load. In some embodiments, the control device 130 can determine the motion state of the arm based on the reconstructed shape and its kinematic model. In some embodiments, the motion state includes a free motion state (e.g., a motion state without external load) and / or a constrained state (e.g., a motion state under external load). In some embodiments, the control device 130 can adaptively control the motion of the arm based on its motion state. For example, the control device 130 can actively adjust the movement of the arm body when the arm body is in a constrained state to achieve an active compliant movement effect. For example, in a surgical robot system (such as a laparoscopic surgical robot system), the control device 130 can actively adjust the movement of the arm body when the arm body is in a constrained state to reduce the interaction force between the arm body and the patient's tissues and improve surgical safety.

[0070] Some embodiments of this disclosure provide a control method for an arm body. Figure 4 A flowchart is shown of a control method 400 for an arm body (hereinafter also referred to as "method 400") according to some embodiments of the present disclosure. Method 400 may be implemented or performed by hardware, software, or firmware. In some embodiments, method 400 may be performed by a surgical robot system (e.g., Figure 13 The surgical robot system 1300 shown is executed. In some embodiments, method 400 can be implemented as computer-readable instructions. These instructions can be executed by a general-purpose processor or a special-purpose processor (e.g., Figure 1 The control device 130 shown or Figure 13 The processor 1330 shown reads and executes the instructions. For example, the control unit of a surgical robot system may include a processor configured to execute method 400. In some embodiments, these instructions may be stored on a computer-readable medium.

[0071] This disclosure discloses embodiments of reconstructing the shape of an arm body (e.g., a continuum manipulator) based on the end-effector pose to obtain shape feature parameters. In some embodiments, the theoretical driving amount caused by changes in the arm body shape to multiple structural bones within the arm body can be calculated using the reconstructed shape feature parameters. Due to the presence of factors such as difficult-to-model and nonlinear drive hysteresis and / or tensile / compressive deformation of structural bones within the arm body, a driving amount residual always exists for the structural bones within the arm body. The driving amount residual is the difference between the actual driving amount output by the drive mechanism (e.g., a drive motor) in the drive unit and the theoretical driving amount calculated from the reconstructed shape feature parameters. This disclosure denotes the difference between the actual driving amount and the theoretical driving amount as the second driving amount residual. To compensate for the driving amount residual that is difficult to model when the arm body is without external load force, in some embodiments of this disclosure, a driving amount residual calculation model without external load force is trained by designing a support vector regression model and a series of random trajectory motion data. The trained driving amount residual calculation model can predict the driving amount residual under actual driving amount conditions and without external load force. In this disclosure, the driving quantity residual predicted by the driving quantity residual calculation model under actual driving quantity conditions and without external load force is denoted as the first driving quantity residual. However, under the same actual driving quantity conditions, external load force will change the shape of the arm body, thereby changing the second driving quantity residual of each corresponding structural bone. Based on this, some embodiments of this disclosure use the difference between the driving quantity residual calculated by the actual driving quantity and theoretical driving quantity (second driving quantity residual) and the driving quantity residual predicted by the driving quantity residual calculation model (first driving quantity residual) to determine the motion state of the arm body (e.g., free motion state or constrained state). Some embodiments of this disclosure perform adaptive motion control of the arm body based on the motion state of the arm body (e.g., active compliant control of the arm body).

[0072] The following, combined with Figure 4 The method 400 of some embodiments of this disclosure will be described in detail.

[0073] See Figure 4 In step 401, the current actual drive amount of the arm body is obtained. In some embodiments, the arm body may be a continuous robotic arm (e.g., Figure 3 The arm body 300 shown includes at least one segment, which includes a fixing plate and multiple structural bones. A first end of each structural bone is fixedly connected to the fixing plate, and a second end of each structural bone is connected to a drive unit. The current actual drive quantity of the arm body may include the current drive quantity output by the drive unit to the structural bones of the arm body.

[0074] In some embodiments, the operator issues control commands to control the movement of the arm (e.g., the control commands include drive information). The drive unit responds to the control commands by pushing and / or pulling the structural bone to move the structural bone, thus meeting the operator's operational needs for the arm. In some embodiments, the current control commands for the arm can be obtained, and the drive information (e.g., drive vector) of the arm can be obtained from the current control commands. Based on the drive information of the arm, the current drive amount of the drive unit on the structural bone of the arm is obtained as the current actual drive amount of the arm. In some embodiments, the state (e.g., rotation angle or stroke) of the drive unit (e.g., a drive motor) can be sensed by a sensor (e.g., a potentiometer) to obtain the current actual drive amount of the structural bone of the arm.

[0075] In some embodiments, the drive vector of the arm body may include the drive vector q of multiple structural bones of the arm body. ij , ij is the number of the structural bone, and the driving forces of multiple structural bones constitute the driving force vector of the structural bone, which can be denoted as q. a,bkb For example, such as Figure 3 In the arm body 300 shown, the deformation of the i-th segment is achieved by driving two sets of symmetrically distributed structural bones through a driving unit. In some embodiments, each set of symmetrically distributed structural bones may include two symmetrically distributed structural bones (e.g., the included angle between the two structural bones is π). In some embodiments, the deformation of the i-th segment can be achieved by driving (e.g., simultaneously pushing and pulling) the two sets of symmetrically distributed structural bones, for example, by two pairs of double-ended screw driving mechanisms. In some embodiments, the double-ended screw driving mechanism may include a double-ended screw and two threaded sliders located on the double-ended screw. When the double-ended screw rotates, the two threaded sliders located on the double-ended screw move in opposite directions at the same speed. The two threaded sliders drive the two symmetrically distributed structural bones to move in opposite directions at the same speed, so that the two symmetrically distributed structural bones are pushed or pulled to achieve the bending deformation of the segment. In some embodiments, the two sets of structural bones driven by the i-th segment are represented by structural bones numbered i1 and i2, respectively. Therefore, the driving vector of the structural bones can be expressed as: q a,bkb =[q 11 ,q 12 ,q 21 ,q 22 ] T In some embodiments, the current driving vector of the structural bone can be obtained, and the current driving vector of the structural bone can be used as the current actual driving amount of the arm body, which can be represented as q. a,bkb .

[0076] Continue reading Figure 4 In step 403, the current theoretical drive quantity of the arm body is determined. In some embodiments, the arm body may be a continuous body robotic arm (e.g., Figure 3The arm body 300 shown includes at least one segment, which includes a fixing disc and multiple structural bones. A first end of each structural bone is fixedly connected to the fixing disc, and a second end of each structural bone is connected to a drive unit. The current theoretical drive amount of the arm body can be based on the current shape of the arm body and the theoretical drive amount of the structural bones of the arm body.

[0077] In some embodiments, method 400 may include determining the current shape feature parameters of the arm body; and determining the current theoretical driving amount based on the current shape feature parameters.

[0078] In some embodiments, method 400 may include obtaining current shape information of the arm body and determining current shape feature parameters based on the current shape information.

[0079] In some embodiments, the arm includes a shape sensor, which can be used to obtain the current shape information of the arm, and to determine the current shape feature parameters of the arm based on the current shape information. For example, measurement signals can be received from the shape sensor, and the current shape information of the arm can be determined based on the measurement signals. In some embodiments, the measurement signals can include curvature at multiple locations along the axial direction of the arm, and the current shape information of the arm can be determined based on the curvature. In some embodiments, the curvature at multiple locations can be continuously processed using a continuity algorithm to determine the current shape information of the arm. For example, the continuity algorithm can be an interpolation algorithm. In this way, the shape information of the arm determined based on the continuous curvature is more accurate.

[0080] In some embodiments, the number of shape sensors may be one or more. In some embodiments, the shape sensors may be uniformly or non-uniformly arranged on the arm body. In some embodiments, the shape sensor includes a fiber Bragg grating sensor, which includes gratings located at multiple positions along the axial direction of the arm body. Figure 5 A schematic diagram of an arm 500 equipped with a shape sensor according to some embodiments of the present disclosure is shown. In some embodiments, a fiber Bragg grating sensor may be installed in the arm 500. Figure 5As shown, the fiber Bragg grating sensor includes gratings 510a-d located at multiple positions along the axial direction of the arm body 500. Measurement signals can be obtained through the gratings 510, and the shape of the arm body 500 can be reconstructed based on the measurement signals. The current shape information of the arm body is obtained based on the reconstructed arm body shape. The fiber Bragg grating sensor uses optical measurement, has good electromagnetic compatibility, and can obtain accurate shape measurement data. The optical measurement method occupies less internal space in the deformable robotic arm, making data acquisition more flexible. The overall shape of the arm body can be obtained by setting a shape sensor in the arm body. The fiber Bragg grating sensor is an existing medical imaging device in hospitals, can be reused, and has the advantage of low cost in data measurement. In some embodiments, the shape sensor can also be an electromagnetic sensor. The electromagnetic sensor can include an electromagnetic induction device and multiple electromagnetic reflectors discretely arranged at multiple positions on the arm body. The electromagnetic induction device relies on the mutual inductance principle of electromagnetic fields to determine the pose information of the electromagnetic reflectors discretely arranged at multiple positions on the arm body, reconstructs the shape of the arm body based on the discrete pose information, and obtains the current shape information of the arm body based on the reconstructed arm body shape. In some embodiments, medical imaging equipment such as computed tomography (CT), magnetic resonance imaging (MRI), and stereoscopic vision can also be used to obtain the current shape information of the arm.

[0081] In some embodiments, method 400 may include obtaining the current end-effector pose of the arm and determining current shape feature parameters based on the current end-effector pose.

[0082] In some embodiments, method 400 may include using an electromagnetic sensor to obtain the current end-effector pose of the arm. In some embodiments, an electromagnetic reflector may be fixed to the end of the arm to detect the end-effector pose of the arm, and the electromagnetic induction device relies on the mutual inductance principle of electromagnetic fields to determine the pose information of the electromagnetic reflector operating in its excitation magnetic field space to obtain the current end-effector pose of the arm.

[0083] In some embodiments, method 400 may include obtaining the current end-effector pose of the arm using a visual detection algorithm. In some embodiments, a positioning image is acquired using an image acquisition device, which may include, but is not limited to, a dual-lens image acquisition device or a single-lens image acquisition device, such as a binocular or monocular camera. In some embodiments, the image acquisition device may be used to acquire the positioning image. The positioning image may include part or all of the image of the arm. In some embodiments, a positioning tag is provided at or near the distal end of the arm, and the positioning tag is used for detecting the end-effector pose of the arm. The positioning tag may include multiple pose identifiers and angle identifiers. In some embodiments, the positioning tag is within the field of view of the image acquisition device, and the end-effector pose of the arm can be obtained from the positioning image acquired by the image acquisition device. In some embodiments, method 400 may include obtaining a positioning image of the arm; identifying multiple identifiers located on the arm in the positioning image; and determining the current end-effector pose of the arm based on the multiple identifiers.

[0084] In some embodiments, the surgical robot system may include a visual guidance arm with an imaging device (e.g., a binocular endoscope) mounted at its distal end and at least one surgical execution arm with a surgical actuator mounted at its distal end. The imaging device mounted at the distal end of the visual guidance arm acts as an image acquisition device, acquiring positioning images of the surgical execution arm to obtain the current end-effector pose of the surgical execution arm. Figure 6 This diagram illustrates the acquisition of the end-effector pose in a surgical robot system 600 according to some embodiments of the present disclosure. For example... Figure 6 As shown, the surgical robot system 600 includes at least one surgical execution arm 610 and a visual guidance arm 620. A binocular endoscope 621 is mounted on the distal end of the visual guidance arm 620, and a surgical actuator 611 is mounted on the distal end of the surgical execution arm 610. In some embodiments, a positioning tag is provided on the distal end of the surgical execution arm 610. Figure 6 (Not shown in the image), the distal end of the surgical arm 610 is within the field of view of the binocular endoscope 621. The positioning image acquired by the binocular endoscope 621 may include a positioning image of the distal end of the surgical arm 610, and the current end-effector pose of the surgical arm 610 is obtained from the positioning image. In some embodiments, positioning images from the binocular endoscope 621 may be received and processed to obtain the current end-effector pose of the surgical arm 610.

[0085] In some embodiments, such as Figure 3The arm body 300 shown (e.g., the arm body 320 or the distal end 310 of the arm body) has a plurality of pose markers and at least one angle marker distributed thereon. For example, the plurality of pose markers are distributed circumferentially on the distal end 310 of the arm body, and the plurality of angle markers are distributed circumferentially on the distal end 310 of the arm body. The plurality of pose markers and the plurality of angle markers are arranged side by side along the axial direction on the distal end 310 of the arm body. For example, the plurality of pose markers and the plurality of angle markers are disposed on the outer surface of the columnar portion of the distal end 310 of the arm body.

[0086] In some embodiments, each angle marker has a positional association with one of the pose markers. Based on this positional association, the possible distribution area of ​​the angle markers can be determined by the position of the pose markers. Alternatively, the possible distribution area of ​​the pose markers can be determined by the position of the angle markers. The positional association can be determined according to the specific arrangement of the pose markers and angle markers, and can be pre-designed.

[0087] In some embodiments, the positional association may include an axial correspondence between angle markers and pose markers. For example, the positional association may include an axial offset. Based on the axial correspondence, given that the positions of one or more pose markers on the distal end of the arm are known, an axial offset by a certain distance can determine the area where angle markers may exist. For example, the positional association may also include axial oblique alignment, etc.

[0088] In some embodiments, multiple pose markers and multiple angle markers may be set on a label attached to the periphery of the distal end of the arm.

[0089] In some embodiments, a pose identifier may include a pose identifier pattern and pose identifier pattern corner points, and an angle identifier may include an angle identifier pattern and angle identifier pattern corner points. In some embodiments, the pose identifier pattern and angle identifier pattern may be disposed on a label attached to the distal end of the arm, or may be printed on the distal end of the arm, or may be patterns formed by the physical structure of the distal end of the arm itself, for example, including recesses or protrusions and combinations thereof. In some embodiments, the pose identifier pattern or angle identifier pattern may include patterns formed with brightness, grayscale, color, etc. In some embodiments, the pose identifier pattern and angle identifier pattern may include patterns that actively (e.g., self-illuminating) or passively (e.g., reflecting light) provide information to be detected by an image acquisition device (e.g., an imaging device, such as a binocular endoscope, installed on the distal end of the visual guidance arm of a robotic system). Those skilled in the art will understand that in some embodiments, the pose of the pose identifier may be represented by the pose of the pose identifier pattern corner point coordinate system, and the pose of the angle identifier may be represented by the pose of the angle identifier pattern corner point coordinate system.

[0090] Figure 7A schematic diagram of a positioning tag 700 including multiple pose markers and multiple angle markers according to some embodiments of the present disclosure is shown. Figure 8 A schematic diagram is shown of a positioning tag 800 disposed on the peripheral side of the distal end of the arm body and formed into a cylindrical shape according to some embodiments of the present disclosure. It will be understood that, for simplicity, the positioning tag 700 may include the same pose marking pattern and angle marking pattern as the positioning tag 800.

[0091] See Figure 7 Multiple pose markers (represented by the symbol "○" for corner points in this disclosure) and multiple angle markers (represented by the symbol "△" for corner points in this disclosure) are arranged side by side. The multiple pose marker patterns 711 may be identical or similar, and the corner points of the multiple pose marker patterns are located within the multiple pose marker patterns 711. The multiple angle marker patterns 721-726 may be different, and the corner points of the multiple angle marker patterns are located within the multiple angle marker patterns 721-726.

[0092] Each angle marker and one of the pose markers can have a positional association. For example, such as Figure 7 As shown, in the direction indicated by the arrow, some pose markers (e.g., pose marker pattern 711) and corresponding angle markers (e.g., angle marker pattern 721) are arranged along the arrow direction and have a spacing d1. See also Figure 8 In the circumferential setting state, label 700 becomes label 800 with a spatial structure of a cylinder. The positional association between each angle identifier and one of the pose identifiers can include the angle identifier and the pose identifier in the axial direction (e.g., Figure 8 The correspondence between the angle markers and the pose markers along the positive Z-axis is established. Based on this axial correspondence, given the known positions of one or more pose markers on the distal end of the arm, the region where the angle markers may exist can be determined by offsetting by a certain distance (e.g., distance d1) along the axial direction. In some embodiments, the axial correspondence between the angle markers and the pose markers can be represented by the axial correspondence between the corner points of the angle marker pattern and the corner points of the pose marker pattern. In some embodiments, based on the axial correspondence between the angle markers and the pose markers, the projections of one of the corner points of the angle marker pattern and the corner points of the pose marker pattern along the Z-axis coincide.

[0093] In some embodiments, the about-axis angle or roll angle of the angle marker or pose marker can be represented by the about-axis angle of the corner point of the angle marker pattern or the corner point of the pose marker pattern. The corner point of the angle marker pattern is relative to the arm body coordinate system (e.g., a coordinate system established at the distal end of the arm body, such as...). Figure 8 The angles of the XY coordinate system shown are known or predetermined, for example... Figure 8In the XY coordinate system, the angle between corner point R8 of the angle marker pattern and the X-axis is θ. Based on the positional relationship, the angle between corner point P8 of the pose marker pattern associated with its position and the X-axis can be obtained as angle θ. It should be understood that the angle θ corresponding to corner point R8 of the angle marker pattern and corner point P8 of the pose marker pattern can be called the axial angle or roll angle of the angle marker or pose marker about the Z-axis. In this disclosure, the axial angle or roll angle refers to the angle about the Z-axis. It is understood that, for clarity, Figure 8 The corner point R8 of the angle marker pattern and the corner point P8 of the pose marker pattern are shown as separate, but they are overlapping.

[0094] In some embodiments, the arm body can be obtained through a visual detection algorithm (e.g., Figure 3 The arm shown is 300mm long. Figure 5 The arm shown Figure 6 The surgical arm shown is 610. Figure 9 The arm body shown is 900 or Figure 13 The current end-effector pose of the arm (1311) shown. In some embodiments, method 400 may include obtaining a positioning image of a positioning tag disposed at the distal end of the arm; identifying multiple markers (including pose markers and angle markers) located on the arm in the positioning image; and determining the current end-effector pose of the arm based on the multiple markers. Figure 9 This diagram illustrates the end-effector pose of the arm 900 obtained using a visual detection algorithm based on a binocular endoscope 921 according to some embodiments of the present disclosure. Figure 9 In the middle, the arm body is 900 as follows Figure 3 The arm body 300 and arm body 900 shown may include an end effector (e.g., a surgical actuator) mounted at the distal end of the arm body. Therefore, the end effector pose of the arm body 900 refers to the pose of the end effector, such as the pose of the surgical actuator. In some of the following embodiments, the end effector is described using a surgical actuator as an example. The following describes... Figure 9 The definition of the coordinate system is explained below: Surgical actuator coordinate system and Figure 3 Consistent with the above, please refer to the following: Figure 3 The surgical actuator coordinate system {tip} is attached to the surgical actuator and is fixed at the distal end of the distal arm 310 at a set distance from the fixed disk coordinate system {2e} at the end of the second component 3202 in the main body of the arm 320. and Consistent direction direction such as Figure 3 or Figure 9 As shown. Sheath coordinate system. and Figure 3 It is consistent with the middle, and it is attached to the outlet of the sheath. With the first segment base disk coordinate system {1b} Consistent direction direction such as Figure 3 or Figure 9 As shown. Location label coordinate system. The positioning tag is attached to the peripheral side of the distal end of the arm body, with the origin being the center of the circle containing the corner points of multiple pose marker patterns. The axis direction points from the origin to one of the corner points of the pose marker pattern. The direction is parallel to the axis of the arm's end. Axis perpendicular to Plane. Binocular endoscope left lens coordinate system. It is attached to the left lens, with its origin located at the center of the left lens. The direction is parallel to the optical axis of the left lens. direction such as Figure 9 As shown. Coordinate system of the right lens of the binocular endoscope. It is attached to the right lens, with its origin located at the center of the right lens. The direction is parallel to the optical axis of the right lens. direction such as Figure 9 As shown. In some embodiments, as Figure 9 As shown, the left and right lenses of the binocular endoscope 921 can obtain positioning images respectively. The pose of the positioning tag in the binocular endoscope lens coordinates can be estimated using a visual detection algorithm based on one image (left lens image or right lens image) of the binocular endoscope 921 (for example, a method similar to that disclosed in some embodiments of Chinese patent document CN115708128A can be used to obtain the pose of the positioning tag in the binocular endoscope lens coordinates), such as the pose of the positioning tag in the left lens coordinate system {ll} of the binocular endoscope. ll p wm , ll R wm Or the pose of the positioning tag in the coordinate system {rl} of the right lens of the binocular endoscope. rl p wm , rl R wm The following is the pose of the positioning tag in the coordinate system {ll} of the left lens of the binocular endoscope. ll p wm , ll R wm Let's take an example to illustrate. The pose of the positioning tag in the coordinate system of the left lens of the binocular endoscope ( ll p wm , ll R wm It can be determined through image analysis, and the homogeneous transformation matrix of the positioning tag relative to the coordinate system of the left lens of the binocular endoscope can also be obtained. ll T wmTherefore, the homogeneous transformation matrix of the surgical actuator coordinates {tip} at the end of the arm 900 relative to its sheath coordinate system {tc} can be, for example, shown in the following formula (13):

[0095]

[0096] in, Let represent the homogeneous transformation matrix of the surgical actuator coordinates {tip} obtained based on the visual detection algorithm relative to its sheath coordinate system {tc}. This represents the current end-effector pose of the arm obtained based on a visual detection algorithm (i.e., the pose of the surgical actuator relative to the sheath coordinate system at the current moment). This is the current end position of the arm. This is the current end-effector posture of the arm. tc T ll This represents the homogeneous transformation matrix of the left lens coordinate system of the binocular endoscope relative to the sheath coordinate system. ll T wm This represents the homogeneous transformation matrix of the positioning tag coordinate system relative to the coordinate system of the left lens of the binocular endoscope. wm T tip This is the homogeneous transformation matrix of the surgical actuator coordinates relative to the positioning label coordinate system.

[0097] In some embodiments, the binocular endoscope is fixed to the distal end of the visual guidance arm (e.g., as shown in the figure). Figure 6 As shown, the binocular endoscope 621 is mounted on the distal end of the visual guidance arm 620. The end-effector pose of the visual guidance arm can be obtained using the same method as described in some of the above embodiments (e.g., the end-effector pose of the visual guidance arm corresponds to the pose of the binocular endoscope). Based on the end-effector pose of the visual guidance arm, the poses of the left and right lenses in the binocular endoscope can be obtained respectively. This allows for the determination of the transformation matrix of the binocular endoscope lens coordinate system relative to the sheath coordinate system, such as the homogeneous transformation matrix of the left lens coordinate system relative to the sheath coordinate system. tc T ll In some embodiments, the surgical actuator is fixedly disposed at the distal end of the distal arm body, and a positioning tag is fixedly disposed at the distal arm body (e.g., the positioning tag is fixedly disposed on the outer surface of the columnar portion of the distal arm body). Therefore, wm T tip It is known or predetermined. In some embodiments, based on tc T ll , ll T wm , wm T tip It has been determined that the current end-effector pose of the arm can be obtained based on formula (13).

[0098] In some embodiments, method 400 may further include determining current shape feature parameters by minimizing the difference between the theoretical end-effector pose and the current end-effector pose based on the current end-effector pose and the kinematic model of the arm body.

[0099] In some embodiments, the bending deformation of each segment in the arm can be estimated based on the constant curvature assumption. Here, constant curvature means that the magnitude of the curvature κ(s) of the center curve of each segment in the arm is a constant with respect to its arc length s. For example, as... Figure 2 As shown, in the i-th segment, the curvature change information of the segment undergoing bending deformation can be represented as θ. i θ i for about or Rotate to The required rotation angle. A single segment 200 can be described by a kinematic model. In some embodiments, the position of the end of the i-th segment (e.g., the fixed disk coordinate system {ie}) relative to the base disk coordinate system {ib} can be determined using formulas (1) to (5) as described in some of the embodiments above. ib p ie ,attitude ib R ie The segment parameter ψ of a single segment 200 i It can be determined based on formula (6) in some of the above embodiments. Therefore, the shape characteristic parameters of the arm body can be determined according to its working state (e.g., C1-C4 configuration). For example, the shape characteristic parameter ψ of the arm body can be determined based on formulas (9) to (12). For example, as... Figure 3 The shape characteristic parameters of the arm body 300 shown in the C3 configuration can be expressed as follows:

[0100] In some embodiments, the bending deformation of the segments in the arm can be estimated based on the linear curvature assumption. Here, linear curvature means that the magnitude of the curvature κ(s) of the center curve of each segment in the arm varies linearly with its arc length variable s. For example, as... Figure 2 As shown, in the i-th segment, the curvature change information of the segment undergoing bending deformation can be represented as κ. i (s i ), κ i (s i It can be represented as shown in the following formula (14):

[0101] κ i (s i ) = a i s i +b i (14)

[0102] Among them, κ i (s i ) represents the center curve of the i-th segment (e.g., the center curve corresponding to the virtual structural bone of the i-th segment) at an arc length s. i The magnitude of curvature at point a i and b i θ represents the slope and intercept of the curvature change of the i-th segment, respectively. i With κ i (s i The conversion relationship between them can be shown in the following formula (15):

[0103]

[0104] Among them, L i The virtual structural skeleton for the i-th segment (e.g., Figure 2 The length of the virtual structure bone 221 shown in the figure.

[0105] The segment parameter of a single segment 200 can be expressed as ψ. i It means that ψ i It can be shown in the following formula (16):

[0106] ψ i =[a i ,b i ,δ i ] T (16)

[0107] Where, δ i Let be the bending direction angle, representing the bending plane and in the i-th structural section. The included angle.

[0108] like Figure 2 The single segment 200 shown can be represented by a kinematic model. The position of the end of the i-th segment (fixed disk coordinate system {ie}) relative to the base disk coordinate system {ib}. ib p ie ,attitude ib R ie These can be shown in formulas (17) and (18) respectively:

[0109]

[0110] ib R ie = ib R i1 i1 R i2 i2 R ie (18)

[0111] Among them, L i The virtual structural skeleton for the i-th segment (e.g., Figure 2 The length of the virtual structural bone 221 shown in the figure, ib R i1 Let {i1} be the orientation of the bending plane coordinate system of the i-th segment relative to the base disk coordinate system {ib}. i1 R i2 Let be the orientation of the bending plane coordinate system 2{i2} of the i-th segment relative to the bending plane coordinate system 1{i1}. i2 R ie Let {i1} be the orientation of the fixed disk coordinate system {i2} of the i-th segment relative to the curved plane coordinate system {i3}. Similar to some embodiments described above, ib R i1 , i1 R i2 and i2 R ie It can be determined based on formulas (3), (4) and (5) in some of the embodiments above.

[0112] Therefore, the shape characteristic parameters of the arm can be determined based on its working state (e.g., C1-C4 configuration). For example, the shape characteristic parameter ψ of the arm can be determined based on formulas (9) to (12). Figure 3 The shape characteristic parameters of the arm body 300 shown in the C3 configuration can be expressed as follows:

[0113] In some embodiments, such as Figure 3In the arm body 300 shown, the length of the first segment 3201 is much larger than the length of the second segment 3202; for example, L1 is 2 to 3 times the length of L2. Here, L1 is the length of the virtual structural bone of the first segment 3201, and L2 is the length of the virtual structural bone of the second segment 3202. Those skilled in the art should understand that the lengths of the first segment 3201 and the second segment 3202 can be configured according to the requirements of the actual application scenario. Under load, the first segment 3201 is closer to a linear curvature shape, and the shape error caused by estimating the bending deformation of the first segment 3201 based on the linear curvature assumption is also smaller. Similarly, under load, the second segment 3202 is closer to a constant curvature shape, and the shape error caused by estimating the bending deformation of the second segment 3202 based on the constant curvature assumption is also smaller. Therefore, in some embodiments, the bending deformation of the first segment 3201 can be estimated based on the linear curvature assumption, and the bending deformation of the second segment 3202 can be estimated based on the constant curvature assumption. The estimation of bending deformation of the component based on the linear curvature assumption and the estimation of bending deformation of the component based on the constant curvature assumption have been described in detail in the above embodiments, and will not be repeated here. Based on formula (16) in the above embodiments, the component parameter of the first component 3201 can be obtained as ψ1, and the component parameter of the second component 3202 can be obtained as ψ2 based on formula (6) in the above embodiments. ψ1 and ψ2 can be shown as shown in the following formulas (19) and (20), respectively:

[0114] ψ1=[a1,b1,δ1] T (19)

[0115] ψ2=[θ2,δ2] T (20)

[0116] Where a1 and b1 represent the slope and intercept of the curvature change of the first segment 3201, respectively, and δ1 represents the bending plane and the curve in the first segment 3201. The included angle, θ2, represents the angle in the second segment 3202. about or Rotate to The required rotation angle, δ2, represents the bending plane in the second component 3202 and The included angle.

[0117] like Figure 2 The single component 200 shown can be represented by a kinematic model. Based on formulas (17) and (18) in some of the above embodiments, the position of the end of the first component 3201 (fixed disk coordinate system {1e}) relative to the base disk coordinate system {1b} 1b P 1e ,attitude 1b R 1eThey can be shown in formulas (21) and (22) below, respectively.

[0118]

[0119] 1b R 1e = 1b R 11 11 R 12 12 R 1e (twenty two)

[0120] Based on formulas (1) and (2) in some of the above embodiments, the position of the end of the second component 3201 (fixed disk coordinate system {2e}) relative to the base disk coordinate system {2b} 2b P 2e ,attitude 2b R 2e As shown in the following formulas (23) and (24):

[0121]

[0122] 2b R 2e = 2b R 21 21 R 22 22 R 2e (twenty four)

[0123] in, 2b R 21 , 21 R 22 , 22 R 2e It can be determined based on formulas (3), (4) and (5) in some of the above embodiments.

[0124] Therefore, the shape characteristic parameters of the arm can be determined based on its working state (e.g., C1-C4 configuration). For example, the shape characteristic parameter ψ of the arm can be determined based on formulas (9) to (12). Figure 3 The shape characteristic parameters of the arm body 300 shown in the C3 configuration can be expressed as follows:

[0125] In some embodiments, the arm body is reconstructed based on the shape assumptions of the arm body and the current end-effector pose of the arm body to determine the current shape feature parameters of the arm body.

[0126] In some embodiments, the shape reconstruction problem of the arm body is expressed as a nonlinear optimization problem, as shown in Equation (25):

[0127]

[0128] in, The shape feature parameters of the arm body to be optimized are... This represents the current theoretical end-effector pose of the arm. This represents the current theoretical end position of the arm. This indicates the current theoretical end-effector posture of the arm. Represented as the current end-effector pose of the arm, in some embodiments, This can be the current end-effector pose of the arm, obtained based on a visual detection algorithm. In some embodiments, Based on the shape assumptions of the arm body, the shape assumptions in some of the above embodiments can be used to determine the shape.

[0129] In some embodiments, the current theoretical end-effector pose of the arm can be described based on shape assumptions about the arm and through a kinematic model. For example, as... Figure 3 The pose of a single segment (e.g., the first segment 3201 or the second segment 3202) in the arm 300 shown can be determined by combining an estimate of the segment bending deformation in the arm with a kinematic model. For example, the position of the end point of the i-th segment (e.g., the fixed disk coordinate system {ie}) relative to the base disk coordinate system {ib} can be obtained based on the segment parameters of a single segment. ib p ie ,attitude ib R ie The specific details have been described in detail in some of the above embodiments, and will not be repeated here. Furthermore, as... Figure 3 The end-effector pose of the entire arm 300 shown can be described by a kinematic model, thus allowing the theoretical end-effector pose to be obtained given the shape feature parameters of the arm. For example, as Figure 3 The homogeneous transformation matrix of the surgical actuator coordinates {tip} at the end of the arm body 300 relative to its sheath coordinate system {tc} can be expressed as shown in formula (26):

[0130]

[0131] in, Let represent the homogeneous transformation matrix of the surgical actuator coordinates {tip} obtained based on the kinematic model relative to its sheath coordinate system {tc}. This represents the theoretical end-effector pose of the arm 300 (e.g., the pose of the surgical actuator relative to its sheath coordinate system obtained through a kinematic model). This represents the theoretical end-effector position of arm 300 (e.g., the position of the surgical actuator coordinates relative to its sheath coordinate system obtained through a kinematic model). tcT 1b This represents the homogeneous transformation matrix of the base disk of the first component 3201 relative to the sheath coordinate system. 1b T 1e This represents the homogeneous transformation matrix of the fixed disk of the first component 3201 relative to the base disk of the first component 3201. 1e T 2b This represents the homogeneous transformation matrix of the base disk of the second component 3202 relative to the fixed disk of the first component 3201. 2b T 2e This represents the homogeneous transformation matrix of the fixed disk of the second component 3202 relative to the base disk of the second component 3202. 2e T tip This represents the homogeneous transformation matrix of the surgical actuator relative to the fixed disk of the second component 3202. In some embodiments, the surgical actuator is fixedly disposed at the distal end of the fixed disk of the second component 3202, therefore, 2e T tip It is known or predetermined. In some embodiments, the base plate of the second component 3202 is connected to the fixed plate of the first component 3201 via a first straight rod segment 3203, therefore, 1e T 2b It is known or predetermined, for example, in some embodiments, such as Figure 3 The first structural bone of the first segment 3201 and the second segment 3202 of the arm body 300 shown are γ2 apart on the cross section of the arm body 300, and the length of the first straight segment 3202 is L. r (That is, the distance between the base plate of the second component 3202 and the fixed plate of the first component 3201 can be approximated as L) r ),thus, 1e T 2b =Rot z (γ2)·Trans z (L r In some embodiments, the base plate of the first segment 3201 is connected to the second straight rod 3204, and the position of the sheath relative to the base plate of the first segment 3201 is determined by the overall rotation angle of the arm body. Sure, It can be obtained directly from the drive unit, therefore, tc T 1b It can be determined in advance.

[0132] In some embodiments, the shape feature parameters of the arm body to be optimized This can be obtained based on shape assumptions about the arm body, such as the constant curvature assumption and / or linear curvature assumption used in the above embodiments. For example, as... Figure 3The bending deformation of the first segment 3201 of the arm 300 shown is estimated based on the linear curvature assumption, and the bending deformation of the second segment 3202 is estimated based on the constant curvature assumption. The shape characteristic parameters of the arm 300 in the C3 configuration can be expressed as follows: The shape feature parameter vector of the arm body that needs to be optimized. It can be represented as: The parameter to be estimated. Those skilled in the art will understand that, in cases such as... Figure 3 In the arm body 300 shown, when estimating the bending deformation of each component based on the constant curvature assumption, the shape characteristic parameter vector of the arm body 300 to be optimized under the C3 configuration can be expressed as: Here are the parameters to be estimated. When estimating the bending deformation of each component based on the linear curvature assumption, the shape characteristic parameter vector of the arm body 300 in the C3 configuration that needs to be optimized can be expressed as: These are the parameters to be estimated.

[0133] In some embodiments, the feed amount of the boom (e.g., the overall feed length of the boom, the feed length of a segment in the boom, or the feed length of a straight section in the boom) is provided by a drive unit (e.g., a linear drive mechanism that drives the linear feed of the boom). For example, as Figure 3 In the C3 configuration of the arm body 300 shown, the feed length L1 of the first segment 3201 is provided by a drive unit (e.g., a linear drive mechanism for linear feed of the arm body). In some embodiments, the arm body has an elongated section (e.g., longer than 200 mm) between the drive unit and the sheath. This elongated section will generate a certain degree of torsion under external load, thus affecting the overall rotation angle of the arm body. The required driving rotation angle is provided by the drive unit (e.g., a rotary drive mechanism in which the drive arm rotates about its own central axis). Add an extra offset angle to the base Right now Therefore, exemplarily, such as Figure 3 For the arm shown, in the C3 configuration, when estimating the bending deformation of each component based on the constant curvature assumption, the required optimized shape characteristic parameter vector of the arm can be expressed as: Let be the parameters to be estimated. When estimating the bending deformation of each component based on the linear curvature assumption, the required vector of shape characteristic parameters of the arm body to be optimized can be expressed as: Let be the parameters to be estimated. When estimating the bending deformation of the first segment 3201 based on the linear curvature assumption, and estimating the bending deformation of the second segment 3202 based on the constant curvature assumption, the required vector of shape characteristic parameters of the arm body to be optimized can be expressed as: These are the parameters to be estimated.

[0134] In some embodiments, the formula (25) is used to obtain the following: This is determined as the current shape feature parameter.

[0135] In some embodiments, the optimization problem of formula (25) can be solved, for example, using the open-source C++ optimization library CeresSolver developed by Google. The corresponding cost function is calculated by formula (25), and the input of the cost function is... The parameters to be estimated in the data.

[0136] In some embodiments, method 400 includes determining a current theoretical driving amount based on current shape feature parameters. In some embodiments, method 400 may determine the current theoretical driving amount based on current shape feature parameters and the relationship between current shape feature parameters and the driving amount of the arm body.

[0137] In some embodiments, the driving amount of multiple structural bones in the segments of the arm body has a known mapping relationship with the segment parameters. In some embodiments, the mapping relationship between the driving amount of multiple structural bones and the segment parameters can be determined based on the following formula (7). In some embodiments, similar to a single segment, the driving amount of each structural bone in each segment of the arm body can be determined based on formula (7), and then the driving signal of the driving unit can be determined based on the driving amount.

[0138] In some embodiments, for example Figure 3 The arm body 300 shown is estimated based on formula (25) to obtain the current state characteristic parameters of the arm body. The theoretical driving force of at least one structural bone caused by changes in the shape of the arm can be calculated, such as the theoretical driving length of at least one structural bone.

[0139] For example, such as Figure 3In the arm body 300 shown, the deformation of the i-th segment is achieved by driving two sets of symmetrically distributed structural bones through a driving unit. In some embodiments, each set of symmetrically distributed structural bones may include two symmetrically distributed structural bones (e.g., the included angle between the two structural bones is π). In some embodiments, the deformation of the i-th segment can be achieved by driving (e.g., simultaneously pushing and pulling) the two sets of symmetrically distributed structural bones, for example, by two pairs of double-ended screw driving mechanisms. In some embodiments, the double-ended screw driving mechanism may include a double-ended screw and two threaded sliders located on the double-ended screw. When the double-ended screw rotates, the two threaded sliders located on the double-ended screw move in opposite directions at the same speed. The two threaded sliders drive the two symmetrically distributed structural bones to move in opposite directions at the same speed, so that the two symmetrically distributed structural bones are pushed or pulled to achieve the bending deformation of the segment. In some embodiments, the two sets of structural bones driven by the i-th segment are represented by structural bones numbered i1 and i2, respectively, and the theoretical driving amount of the structural bones can be denoted as: q shape,bkb =[q shape,11 ,q shape,12 ,q shape,21 ,q shape,22 ] T In some embodiments, such as Figure 3 In the arm body 300 shown, the drive of the second segment 3202 takes into account the coupling effect of the structural bone of the second segment 3202 through the channel of the first segment 3201. The theoretical drive amount of the structural bone can be shown in the following formula (27):

[0140]

[0141] Where, r i1 Let r be the distance from structural bone i1 in the i-th segment to the virtual structural bone. i2 Let β be the distance from structural bone i2 in the i-th segment to the virtual structural bone. i1 Let β be the angle between structural bone i1 and the first structural bone in the i-th segment. i2 γ is the angle between structural bone i2 and the first structural bone in the i-th segment, and γ is the offset angle of the second segment 3202 relative to the first segment 3201. To estimate the current state characteristic parameters of the arm body based on formula (25) The estimated values ​​of the parameters to be estimated in the model are determined.

[0142] In some embodiments, in the i-th structural segment, the included angle between structural bones numbered i1 and i2 can be π / 2, and structural bone i1 can be taken as the first structural bone. Therefore, β 11 =0,β 21 =0,β 12 =π / 2, β 22=π / 2. In some embodiments, the offset angle γ of the second component 3202 relative to the first component 3201 is π / 4.

[0143] In some embodiments, the bending deformation of the first segment 3201 is estimated based on the linear curvature assumption, and the bending deformation of the second segment 3202 is estimated based on the constant curvature assumption. The parameters to be estimated can be obtained through formula (25). The estimated value can be obtained from The estimated value was calculated. The estimated value can be expressed as: Those skilled in the art should understand that, based on the assumption of constant curvature, the bending deformation of each component can be estimated directly using formula (25), and the parameters to be estimated can be obtained directly. The estimated value; based on the linear curvature assumption, the bending deformation of each component is estimated, and the parameters to be estimated can be obtained through formula (25). The estimated value can be obtained from The estimated value was calculated. The estimated value, The estimated value was calculated. The estimated value can be expressed as:

[0144] Continue reading Figure 4 In step 405, based on the current actual driving quantity and the driving quantity residual calculation model, the first driving quantity residual of the arm body in the free motion state is determined. In this disclosure, the free motion state of the arm body refers to the motion state of the arm body when it is driven by the driving unit without any external load force.

[0145] Due to the presence of factors such as the difficult-to-model and nonlinear drive hysteresis and / or tensile / compressive deformation of the structural bones within the arm body, the drive quantity residuals of the structural bones within the arm body always exist. In some embodiments, the first drive quantity residual of the arm body in a free motion state may include the first drive quantity residual values ​​of multiple structural bones of the arm body, with each structural bone corresponding to a first drive quantity residual value, and the first drive quantity residual values ​​of multiple structural bones are combined into a first drive quantity residual vector. In step 401 above, the current actual drive quantity q of the arm body is obtained. a,bkb The current actual driving force q of the arm body a,bkb This includes the current actual driving value of multiple structural bones. For example, taking an arm body that drives four structural bones as an example, q a,bkb =[q 11 ,q 12 ,q 21 ,q 22 ] T In step 403, the current theoretical driving quantity q of the arm body is obtained. shape,bkbFor example, taking the arm body, which drives four symmetrically distributed structural bones, as an example, q shape,bkb =[q shape,11 ,q shape,12 ,q shape,21 ,q shape,22 ] T In some embodiments, the current theoretical driving quantity q of the arm body is obtained in step 403 through shape reconstruction of the arm body. shape,bkb At that time, the influence of factors such as driving hysteresis and / or elastic tensile and compressive deformation of each structural bone was not taken into account. Therefore, each structural bone has driving residuals, which can be shown in the following formula (28):

[0146] q rsd,bkb =q a,bkb -q shape,bkb ,q rsd,bkb ≠0 (28)

[0147] For example, taking the arm body, which drives four symmetrically distributed structural bones, as an example, q rsd,bkb =[q rsd,11 ,q rsd,12 ,q rsd,21 ,q rsd,22 ] T , where q rsd,ij =q ij -q shape,ij ,ij=11,12,21,22.

[0148] To compensate for the driving force residual that is difficult to model when the arm is without external load, in some embodiments of this disclosure, a driving force residual calculation model is trained using model design and a series of random trajectory motion data. The trained driving force residual calculation model can predict the driving force residual under actual driving force conditions without external load. In this disclosure, the driving force residual predicted by the driving force residual calculation model under actual driving force conditions without external load is denoted as the first driving force residual, which can be expressed as: The driving quantity residual calculated by the actual driving quantity and the theoretical driving quantity (such as the driving residual q calculated by formula (28)) is used. rsd,bkb This is denoted as the residual of the second driving quantity.

[0149] In some embodiments, the driving force residual calculation model can be a machine learning model. This model can be constructed and trained based on machine learning algorithms to calculate the first driving force residual for at least one structural bone (e.g., four structural bones) of a continuum manipulator in a free state. In some embodiments, the driving force residual calculation model can include a support vector regression model, referred to as an SVR (Support Vector Regression) model. This SVR model can directly predict the arm body's current actual driving force (e.g., the driving force vector q of a given structural bone of the arm body). a,bkb The first driving quantity residual when there is no external load force is applied under the driving force, and the first driving quantity residual is denoted as

[0150] In some embodiments, the training dataset for the driving quantity residual calculation model may include: a driving quantity residual dataset of the arm body in free motion under different actual driving quantity conditions.

[0151] In some embodiments, the SVR model can transform the original data space into a high-dimensional linearly separable space through a kernel function, thereby achieving better regression results for nonlinear data. Simultaneously, the SVR model characterizes the entire data feature using a finite number of support vectors and corresponding weight coefficients; therefore, the SVR model can achieve rapid prediction calculations, ensuring the real-time performance of the arm control. In some embodiments, four independent SVR models can be created for the four structural bones within the continuous robotic arm, as shown in the following formula (29):

[0152]

[0153] Among them, SV ij χ is the number of support vectors in the ij-th SVR model (used to predict the first driving force residual of the ij-th bone structure), and χ is the feature vector of the input SVR model. ij,w and m ij,w Let w be the w-th support vector and its weight for the ij-th SVR model; n ij κ(χ) represents the constant term of the ij-th SVR model. ij,w ,χ) is the Gaussian kernel function of the SVR model, which can be defined as shown in the following formula (30):

[0154]

[0155] Where, γ ij γ is an adjustable parameter of the Gaussian kernel function. ij >0. The optimal γ value before training the SVR model. ij The parameters, as well as the soft margin parameter C in the SVR model, are determined through a gridded traversal search strategy and cross-validation.

[0156] In some embodiments, considering the coupling force relationship between the structural bones within the continuum manipulator, that is, the structural bone of the i-th segment not only affects the bending deformation within its own segment, but also affects the deformation of the structural bones of other segments due to the coupling effect between segments. In some embodiments, the feature vector χ of the four SVR models considers the driving effect of all structural bones, that is, the feature vector χ of each SVR model is defined consistently, as shown in the following formula (31):

[0157]

[0158] Among them, h bkb =[h 11 ,h 12 ,h 21 ,h 22 [h] is a hysteresis feature vector characterizing the hysteresis properties of the structural bone drive process, representing the historical state of the structural bone drive process. In some embodiments, for time k, h ij (k) can be represented as an accumulator with saturation properties, as shown in the following formula (32):

[0159]

[0160] Where, μ max Representing the maximum hysteresis of each structural bone, μ is relevant to the continuous manipulator in some embodiments of this disclosure. max Set to 0.2mm. Δq a,ij Let ij be the driving increment of the structural bone numbered ij during the time interval from time k-1 to time k.

[0161] In some embodiments, the SVR model for calculating the first driving quantity residuals constituting the four structural bones is determined based on formulas (29) to (32). In some embodiments, before using the SVR model, a large amount of random dataset needs to be collected to train the SVR model. For example, for the ij-th SVR model, the collected random dataset can be represented as:

[0162] In some embodiments, different bit types (e.g., C1-C4 bit types) can affect the training and prediction results of the SVR model. In some embodiments, such as... Figure 3In the C3 configuration shown, the overall feed length d of the arm 300 directly changes the length L1 of the first segment 3201. Different L1 conditions affect the training and prediction results of the SVR model. In some embodiments, when training the SVR model, L1 is first discretized from a predetermined motion range (e.g., 40mm to 60mm) at predetermined intervals (e.g., 5mm), and a sufficient amount of random data is collected under each L1 condition to train the corresponding L1 SVR model. During the calculation of the first driving quantity residual using the trained SVR model, the most recent set of SVR models is selected for prediction based on the current L1 value. In some embodiments, the training and prediction of the SVR model can be implemented using the LIBSVM support vector machine library developed by Professor Chih-Jen Lin et al. from Taiwan, China.

[0163] For example, L1 = L x The random data collection process under certain circumstances can be carried out in the following manner:

[0164] Set L1 = L x , Based on the maximum bending angle of the segment in the arm (e.g., θ) 1,max =π / 2,θ 2,max =2π / 3) Determine the range of structural bone driving force, and randomly select N groups (e.g., 3000 groups) of structural bone driving force q within the range of structural bone driving force. a,bkb The bone drive of each group of structures and L1 = L x , The driving vector q constitutes a set of continuous robotic arms a For each set of driving variables q a The theoretical pose of the positioning tag coordinate system {wm} relative to the binocular endoscope lens coordinate system is calculated using a kinematic model. For example, the theoretical pose of the positioning tag coordinate system {wm} relative to the binocular endoscope left lens coordinate system {ll}. ll T wmDrive vectors exceeding the visual detection range of the positioning tag on the distal end of the arm are discarded. In some embodiments, the final random drive quantity trajectory can be obtained by interpolation calculation between two adjacent drive vectors after the above-discarded drive vectors at a set interval (e.g., 0.05 mm). The arm is driven to each given drive quantity position within the random drive quantity trajectory using an experimental platform under no external load conditions, and the feature vector (χ) at each given drive quantity position is calculated and recorded. In some embodiments, the end-effector pose of the arm can be obtained by visual detection algorithm as described in some of the above embodiments, shape reconstruction can be performed based on the end-effector pose, and the theoretical drive quantity of the arm under each given drive quantity can be obtained based on the reconstructed shape using the methods described in some of the above embodiments. In some embodiments, the drive quantity residual can be obtained based on the given drive quantity and the calculated theoretical drive quantity. Since the random data acquisition process is experimentally obtained under no external load force, the experimentally obtained drive quantity residual can be considered as the first drive quantity residual of the arm in free motion state, and the experimentally obtained first drive quantity residual is denoted as... Therefore, the feature vectors of all records at all given driving quantities and the corresponding residuals of the first driving quantity constitute a random dataset. In some embodiments, a portion of the obtained random dataset (e.g., 80% of the data in the random dataset) is randomly selected as the training dataset, and the remainder (e.g., the remaining 20% ​​of the data) is used as the test dataset to verify the prediction performance of the SVR model trained on the training dataset.

[0165] Continue reading Figure 4 In step 407, the second driving quantity residual of the arm body is determined based on the current actual driving quantity and the current theoretical driving quantity.

[0166] In some embodiments, the second driving amount residual of the arm body can be the second driving amount residual of the structural bone of the arm body. Based on the above embodiments, the second driving amount residual of the arm body can be as shown in formula (28).

[0167] Continue reading Figure 4 In step 409, a control operation is performed based on the first driving quantity residual and the second driving quantity residual.

[0168] In some embodiments, method 400 includes obtaining the difference between the drive residual amount and the second drive residual, for example as shown in the following formula (33):

[0169]

[0170] in, For the first driving quantity residual, q rsd,bkb For the second driving quantity residual, The difference that drives the residual.

[0171] For example, consider the arm body that drives four symmetrically distributed structural bones:

[0172] q rsd,bkb =[q rsd,11 ,q rsd,12 ,q rsd,21 ,q rsd,22 ] T , in, Let be the difference in the driving residual of the ij-th structural bone. q rsd,ij =q ij -q shape,ij , The values ​​ij are calculated using a driving quantity residual calculation model (e.g., the SVR model), where ij = 11, 12, 21, 22.

[0173] In some embodiments, the arm is determined to be in a constrained state at the current moment based on the difference between the first driving quantity residual and the second driving quantity residual being higher than a threshold (e.g., the arm is moving under the drive of the drive unit and is subjected to an external load force at the current moment).

[0174] In some embodiments, based on the fact that the difference between the first driving quantity residual and the second driving quantity residual is not higher than a threshold, it is determined that the arm is currently in a free motion state (e.g., the arm is currently moving only under the drive of the drive unit without being subjected to any external load force).

[0175] In some embodiments, the difference in driving residuals is obtained based on formula (33), and it is determined that the arm body is currently operating in a constrained state based on the difference in the driving amount residuals of at least one structural bone being higher than a threshold. For example, taking an arm body that drives four symmetrically distributed structural bones as an example, when At that time, it is determined that the arm is currently in a constrained state. Among other things, Let ξ be the difference in the driving residual of the ij-th structural bone. ij The difference threshold that drives the residual.

[0176] In some embodiments, the difference in driving residuals is obtained based on formula (33), and it is determined that the arm body is in a free motion state at the current moment, based on the fact that the difference in driving residuals of all structural bones is not higher than a threshold. For example, taking an arm body that drives four structural bones as an example, when At that moment, it is determined that the arm is in a state of free movement.

[0177] In some embodiments, each structural bone corresponds to a difference threshold ξ for the driving residual. ij ξ ij The minimum value ξij，min The magnitude of this is related to the average prediction error of the driving quantity residual calculation model (e.g., the SVR model).

[0178] In some embodiments, the drive quantity of the arm is updated based on the difference between the first drive quantity residual and the second drive quantity residual being higher than a threshold (i.e., the arm is working in a constrained state). The drive unit is then controlled to drive the arm based on the updated drive quantity to achieve active control of the arm.

[0179] In some embodiments, method 400 further includes: iteratively determining the drive quantity of the arm body at predetermined periods to achieve compliant control of the arm body through multiple motion control cycles. In some embodiments, in each motion control cycle, the working state of the arm body is determined based on the difference between a first drive quantity residual and a second drive quantity residual. For example, if the difference between the first drive quantity residual and the second drive quantity residual is higher than a threshold, the arm body is determined to be in a constrained state; if the difference between the first drive quantity residual and the second drive quantity residual is not higher than a threshold, the arm body is determined to be in a free motion state. In some embodiments, in each motion control cycle, in response to the difference between the first drive quantity residual and the second drive quantity residual being higher than a threshold (e.g., the arm body is in a constrained state), the drive quantity of the arm body is updated, and the drive unit is controlled to drive the arm body based on the updated drive quantity. Through multiple motion control cycles, the difference between the first drive quantity residual and the second drive quantity residual is gradually reduced, causing the arm body to tend to work in a free motion state, thereby minimizing the interaction force between the arm body and the outside, and achieving active compliant control of the arm body.

[0180] In some embodiments, updating the drive quantity of the arm body may include updating the drive quantity of the arm body based on the difference between the first drive quantity residual and the second drive quantity residual.

[0181] In some embodiments, the drive quantity of the arm body can be updated using a proportional control law based on the difference between the first drive quantity residual and the second drive quantity residual. For example, in the k-th motion control cycle, the drive quantity Δq of the arm body in the k-th motion control cycle is updated in the manner shown in the following formula (34). a,bkb (k):

[0182]

[0183] Where, k comp For proportional parameters, This represents the difference in the drive residual of the arm body during the k-th motion control cycle. Let be the difference in the driving residual of the ij-th structural bone over k motion control cycles.

[0184] In some embodiments, the drive quantity of the arm body can be updated using a PD (proportional and derivative) control law based on the difference between the first drive quantity residual and the second drive quantity residual. For example, in the k-th motion control cycle, the drive quantity Δq of the arm body in the k-th motion control cycle is updated in the manner shown in the following formulas (35) and (36). a,bkb (k):

[0185]

[0186]

[0187] in, The drive quantity of the arm body is updated for the k-th motion control cycle. For the (k-1)th motion control cycle, update the arm's driving force, k comp d is a proportional parameter. comp For differential parameters, This represents the difference in the drive residual of the arm body during the k-th motion control cycle. Let be the difference in the driving residual of the ij-th structural bone over k motion control cycles.

[0188] Those skilled in the art should understand that the drive quantity of the arm can be updated using other control laws based on the difference between the first drive quantity residual and the second drive quantity residual, such as PID (proportional, integral, and derivative) control laws.

[0189] Active compliance control of the arm indirectly reduces the interaction contact force between the arm and the external environment by reducing the difference in drive residuals; it does not directly control the contact force. Therefore, in some embodiments, the load determination threshold (e.g., the difference threshold ξ of the drive residuals) can be changed. ij The magnitude of the interaction force between the arm and the external environment is indirectly controlled. It should be understood that the threshold value ξ for the difference in the driving residual is... ij The change is not less than its minimum value ξ ij，min .

[0190] Figure 10 A logic block diagram of active compliance control of the arm body according to some embodiments of the present disclosure is shown. Figure 10 As shown, in some embodiments, the arm body may adopt the following... Figure 3 The continuous robotic arm shown. In some embodiments, active compliance control of the arm includes steps 1001 to 1017.

[0191] In some embodiments, in step 1001, during the k-th motion control cycle, the current actual drive quantity q of the arm body in the k-th motion control cycle can be obtained based on the method for obtaining the current actual drive quantity of the arm body in some of the above embodiments. a,bkb (k).

[0192] In some embodiments, in step 1003, during the k-th motion control loop, the current end-effector pose of the arm in the k-th motion control loop can be obtained based on a visual detection algorithm. in, This represents the current end position of the arm in the k-th motion control cycle. This represents the current end-effector pose of the arm in the k-th motion control cycle. The specific implementation of obtaining the current end-effector pose based on the visual detection algorithm has been described in detail in some of the above embodiments and will not be repeated here.

[0193] In some embodiments, in step 1005, shape reconstruction is performed based on the end-effector pose of the k-th motion control loop arm to obtain the shape feature parameters of the k-th motion control loop arm. The specific implementation of shape reconstruction based on the end pose of the arm to obtain the shape feature parameters of the arm has been described in detail in some of the above embodiments, and will not be repeated here.

[0194] In some embodiments, in step 1007, the shape feature parameters of the k-th motion control loop arm body obtained based on shape reconstruction are... Obtain the current theoretical driving quantity q of the arm body in the k-th motion control cycle. shape,bkb (k). The specific implementation of determining the current theoretical driving amount of the arm body based on the shape feature parameters of the arm body has been described in detail in some of the above embodiments, and will not be repeated here.

[0195] In some embodiments, in step 1009, based on the current actual drive quantity q of the k-th motion control loop arm... a,bkb (k) is calculated using a driving quantity residual calculation model (e.g., SVR model) to obtain the first driving quantity residual of the arm body in the free motion state during the k-th motion control cycle.

[0196] In some embodiments, in step 1011, based on the current actual drive quantity q of the k-th motion control loop arm... a,bkb (k) and the current theoretical driving quantity q of the kth motion control cycle arm. shape,bkb (k) Calculate the second drive quantity residual (e.g., the actual drive quantity residual) q of the arm body in the k-th motion control cycle. rsd,bkb (k), for example, q rsd,bkb (k)=q a,bkb (k)-q shape,bkb (k).

[0197] In some embodiments, in step 1013, based on the first driving quantity residual of the k-th motion control cycle arm in free motion state... and the second driving quantity residual q of the arm body in the kth motion control cycle rsd,bkb (k) Obtain the difference of the drive residual of the kth motion control cycle arm. For example,

[0198] In some embodiments, in step 1015, based on the fact that the difference in the driving residuals of all structural bones in the k-th motion control cycle is not higher than a threshold, it is determined that the arm body is in a free motion state; based on the fact that the difference in the driving amount residuals of at least one structural bone in the k-th motion control cycle is higher than a threshold, it is determined that the arm body is in a constrained state. In some embodiments, the difference in the driving residuals of the arm body in the k-th motion control cycle... Update the driving quantity Δq of the arm body in the k-th motion control cycle. a,bkb (k), for example, Δq can be obtained using formula (34). a,bkb (k), or obtain Δq using formulas (35) and (36). a,bkb (k).

[0199] In some embodiments, in step 1017, during the k-th motion control cycle, Δq a,bkb (k) The control drive unit drives the arm to achieve active control of the arm. Through multiple motion control cycles, the arm tends to work in a free motion state, achieving active compliant control of the arm.

[0200] Method 400 in some embodiments of this disclosure adaptively controls the movement of the arm based on its motion state. For example, when the arm is in a constrained state, the method actively adjusts the arm's movement to achieve an active compliant motion effect. For instance, in a surgical robot system (e.g., a laparoscopic surgical robot system), method 400 can actively adjust the arm's movement when it is in a constrained state to reduce the interaction force between the arm and patient tissues, thereby improving surgical safety. Those skilled in the art should understand that the methods of this disclosure can simultaneously achieve active compliance of both the arm's distal end and the arm's body, where active compliance of the arm's body refers to an active compliant motion effect that occurs when a load is applied at any position on the arm.

[0201] In some embodiments, the effectiveness of the disclosed method 400 is verified through two sets of experiments. The first set of experiments is an active compliance control experiment of the arm body when a human hand applies a load. This can be achieved by the human hand applying a load to the arm body (e.g., a continuous robotic arm, for example, ...). Figure 3 An external load force is applied to the end of the arm (300 shown), the arm body, or both, and the active compliance of the arm body to the external load is observed. The second set of experiments is an active compliance control experiment of the arm body under straight-channel constraints. This involves applying a load to the arm body (e.g., a continuous robotic arm, for example, ...). Figure 3A movable straight channel outside the arm body 300 (shown) is gradually moved from the root of the arm body (e.g., the proximal end of the arm body) to the end of the arm body, and then the channel is moved back to the root of the arm body. Initially, the arm body is in a bent state. The second set of experiments simulates the application scenario of the arm body entering a narrow cavity (e.g., a continuous robotic arm smoothly entering the nasal cavity).

[0202] In some embodiments, such as Figure 3 The arm body 300 shown was used in the experiment. During the compliant control of the arm body, both experimental groups used a PD control law to update the actuation quantity of the arm body. The proportional parameter k of the PD control law... comp and differential parameter d comp The values ​​are set to 0.6 and 0.2 respectively. Based on the prediction results of the SVR model, the difference threshold ξ driving the residual is... ij (ij=11,12,21,22) are all set to 0.06mm.

[0203] In the first set of experiments, regardless of whether the hand load was applied to the arm's end, the arm body, or both simultaneously, the arm body actively moved in accordance with the direction of the applied load. In the second set of experiments, the initially bent arm body gradually conformed to the channel's constraints and tended towards a straight state during movement through the straight channel, and the arm body maintained its straight shape even after the channel returned to the root of the arm body. These two sets of experiments fully demonstrate that the active compliance control of the method 400 of this disclosure possesses the ability to actively comply with both the arm's end and the arm body. Figure 11A The diagram shows the difference curves of the drive residual during the active compliance control process of the arm body when a human hand applies a load according to some embodiments of the present disclosure. Figure 11B The diagram illustrates the difference curves of the drive residuals during the active compliant control process of the arm body under straight-channel constraints according to some embodiments of this disclosure. Figure 10 In Figure 11, the difference in the driving residuals can be obtained using, for example, formula (33). (ij=11,12,21,22) represents the difference in the driving residuals of the ij-th structural bone. Based on Figure 11A , Figure 11B During the active compliance control of the arm body, the difference between the driving residuals of the two sets of experiments continuously moved towards the gray area (i.e., the threshold ξ of the difference in driving residuals). ij The variation is within the range. Since the difference in the driving residual can characterize the difference in the boom shape between loaded and unloaded states, the external load force on the boom is continuously reduced through active compliance control of the boom.

[0204] In some embodiments of this disclosure, a computer device is also provided, including a memory and a processor. The memory may be used to store at least one instruction, and the processor is coupled to the memory for executing the at least one instruction to perform some or all of the steps in the method of this disclosure, such as... Figure 4 Some or all of the steps in the method 400 disclosed in the paper.

[0205] Figure 12 A schematic block diagram of a computer device 1200 according to some embodiments of the present disclosure is shown. See also Figure 12 The computer device 1200 may include a central processing unit (CPU) 1201, a system memory 1204 including random access memory (RAM) 1202 and read-only memory (ROM) 1203, and a system bus 1205 connecting the various components. The computer device 1200 may also include an input / output system 1206 and a mass storage device 1207 for storing an operating system 1213, application programs 1214, and other program modules 1215. The input / output system includes an input / output controller 1206 primarily consisting of a display 1208 and input devices 1209.

[0206] Mass storage device 1207 is connected to central processing unit 1201 via a mass storage controller (not shown) connected to system bus 1205. Mass storage device 1207 or computer-readable media provides non-volatile storage for computer devices. Mass storage device 1207 may include computer-readable media (not shown) such as hard disk or compact disc read-only memory (CD-ROM) drives.

[0207] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, flash memory or other solid-state storage technologies, CD-ROM, or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The aforementioned system memories and mass storage devices can be collectively referred to as memory.

[0208] Computer device 1200 can be connected to network 1012 via network interface unit 1211 connected to system bus 1205.

[0209] The system memory 1204 or mass storage device 1207 is also used to store one or more instructions. The central processing unit 1201 implements all or part of the steps of the methods in some embodiments of this disclosure by executing these one or more instructions, for example, such as... Figure 4 Some or all of the steps in the method 400 disclosed in the paper.

[0210] In some embodiments of this disclosure, a computer-readable storage medium is also provided, storing at least one instruction that is executed by a processor to cause a computer to perform some or all of the steps in the methods of some embodiments of this disclosure, such as... Figure 4 Some or all of the steps in the method 400 disclosed herein. Examples of computer-readable storage media include memory for computer programs (instructions), such as read-only memory (ROM), random access memory (RAM), compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage devices.

[0211] Figure 13 A schematic diagram of a surgical robot system 1300 according to some embodiments of the present disclosure is shown. In some embodiments of the present disclosure, see [link to relevant documentation]. Figure 13 The surgical robot system 1300 may include a surgical instrument 1310 and a processor 1330. The surgical instrument 1310 includes an arm body 1311 and a surgical actuator 1315 disposed at the distal end of the arm body 1311. The processor 1330 is used to perform some or all of the steps in the methods of some embodiments of this disclosure, such as... Figure 4 Some or all of the steps in the method 400 disclosed in the paper.

[0212] Note that the above are merely exemplary embodiments and technical principles of this disclosure. Those skilled in the art should understand that this disclosure is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this disclosure. Therefore, although this disclosure has been described in detail through the above embodiments, this disclosure is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this disclosure, the scope of which is determined by the scope of the appended claims.

Claims

1. A control method for an arm body, characterized by, The method comprises: obtaining a current actual driving amount of the arm body; determining a current theoretical driving amount of the arm body; determining a first driving amount residual of the arm body in a free motion state based on the current actual driving amount and a driving amount residual calculation model; determining a second driving amount residual of the arm body based on the current actual driving amount and the current theoretical driving amount; determining that the arm body works in a constrained state at the current time based on the difference between the first driving amount residual and the second driving amount residual being higher than a threshold value; and adjusting the motion of the arm body based on the arm body working in the constrained state. The method further comprises:

2. The method of claim 1, wherein, determining a current shape feature parameter of the arm body; and determining the current theoretical driving amount based on the current shape feature parameter. The method further comprises: obtaining current shape information and / or a current end pose of the arm body; and 3. The method of claim 2, wherein, determining the current shape feature parameter based on the current shape information and / or the current end pose. The method further comprises: obtaining a positioning image; identifying a plurality of markers located on the arm body in the positioning image; and 4. The method of claim 3, wherein, determining the current end pose based on the plurality of markers. The end of the arm body is provided with an electromagnetic sensor, and the method further comprises: obtaining information of the electromagnetic sensor; and determining the current end pose based on the information of the electromagnetic sensor. The arm body comprises a shape sensor, and the method further comprises:

5. The method of claim 3, wherein, obtaining information of the shape sensor; and determining the current shape information based on the information of the shape sensor. The method further comprises:

6. The method of claim 3, wherein, determining the current shape feature parameter by minimizing the difference between a theoretical end pose and the current end pose based on the current end pose and a kinematics model of the arm body. The method further comprises: determining the current theoretical driving amount based on the current shape feature parameter and the relationship between the current shape feature parameter and the driving amount of the arm body.

7. The method of claim 3, wherein, The driving amount residual calculation model is trained according to a machine learning model; The training data set of the driving amount residual calculation model comprises:

8. The method of claim 2, wherein, a driving amount residual data set of the arm body in a free motion state under different actual driving amount conditions. The driving amount residual calculation model is a support vector regression model.

9. The method of claim 1, wherein, The method further comprises: updating the driving amount of the arm body based on the difference between the first driving amount residual and the second driving amount residual being higher than a threshold value. The method further comprises:

10. The method of claim 1, wherein, controlling a driving unit to drive the arm body based on the updated driving amount.

11. The method of claim 1, wherein, The method further comprises: iteratively determining the driving amount of the arm body at a predetermined period to realize compliant control of the arm body through multiple motion control cycles.

12. The method of claim 11, wherein, Updating the driving amount of the arm body comprises updating the driving amount of the arm body based on the difference between the first driving amount residual and the second driving amount residual. The method further comprises:

13. The method of claim 11, wherein, determining that the arm body works in a free motion state at the current time based on the difference between the first driving amount residual and the second driving amount residual being not higher than a threshold value. The arm body comprises at least one link, and the link comprises a fixed disc and a plurality of structural bones, the first ends of the plurality of structural bones being fixedly connected to the fixed disc, and the second ends of the plurality of structural bones being connected to a driving unit.

14. The method of claim 11, wherein, ​ 15. The method of claim 1, wherein, ​ ​ 16. The method of claim 1, wherein, ​ The current actual driving amount of the arm body includes: a current driving amount output by the driving unit to the structural skeleton of the arm body; The current theoretical driving amount of the arm body includes: a driving amount of the structural skeleton of the arm body in theory based on a current shape of the arm body; The driving amount residual of the arm body includes: a driving amount residual of the structural skeleton of the arm body.

17. A surgical robotic system, characterized by, Comprise: a surgical instrument, the surgical instrument comprising an arm body, a surgical effector provided at an arm body distal end of the arm body; and a processor for executing the method as claimed in any one of claims 1-16.

18. A computer device, comprising: The computer device comprises: a memory for storing at least one instruction; and a processor coupled with the memory and configured to execute the at least one instruction to perform the method as claimed in any one of claims 1-16.

19. A computer-readable storage medium for storing at least one instruction, characterized in that, The at least one instruction, when executed by a computer, causes the robotic system to implement the method as claimed in any one of claims 1-16.

Images