Method for determining force and surgical robot system

Abstract

The present disclosure relates to the field of robotic control and discloses a method for determining force and a surgical robot system. The method for determining force includes: obtaining a current actual position of a target portion of an arm; obtaining a current actual drive amount of the arm; determining a first drive amount residual of the arm in a free motion state based on the current actual drive amount and a drive amount residual calculation model; determining a current ideal position of the target portion of the arm based on the current actual drive amount and the first drive amount residual; and determining a force on the target portion based on the current actual position and the current ideal position.

Description

Technical Field

[0001] The present disclosure relates to the field of medical devices, and in particular to a method for determining force and a surgical robot system. Background Art

[0002] In recent years, numerous surgical robotic systems using robotic arms have been developed for cardiovascular surgery, neurosurgery, and endoscopic surgery. During surgical operations, the surgical arms may generate interaction forces with patient tissue. Currently, robotic surgical systems lack the ability to sense these interaction forces during arm manipulation. Summary of the Invention

[0003] In some embodiments, the present disclosure provides a method for determining force, including: obtaining the current actual position of the target part of the arm; obtaining the current actual driving amount of the arm; determining the first driving amount residual of the arm in a free motion state based on the current actual driving amount and the driving amount residual calculation model; determining the current ideal position of the target part of the arm based on the current actual driving amount and the first driving amount residual; and determining the force on the target part based on the current actual position and the current ideal position.

[0004] In some embodiments, the present disclosure provides a surgical robot system, comprising: a surgical instrument, the surgical instrument comprising an arm body, a surgical actuator disposed at the end of the arm body; and a processor for executing any one of the methods in some embodiments of the present disclosure.

[0005] In some embodiments, the present disclosure provides a computer device, comprising: a memory for storing at least one instruction; and a processor coupled to the memory and for executing the at least one instruction to perform any method in some embodiments of the present disclosure.

[0006] In some embodiments, the present disclosure provides a computer-readable storage medium for storing at least one instruction, which, when executed by a computer, causes a robotic system to implement a method as described in any one of some embodiments of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] To more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly describes the drawings required for describing the embodiments of the present disclosure. The drawings described below only illustrate some embodiments of the present disclosure. Those skilled in the art can, without inventive effort, derive other embodiments based on the contents of the embodiments of the present disclosure and these drawings.

[0008] Figure 1 A flowchart illustrating a method of determining force according to some embodiments of the present disclosure;

[0009] Figure 2 A structural block diagram of a surgical robot system according to some embodiments of the present disclosure is shown;

[0010] Figure 3 A schematic diagram showing the structure of an arm according to some embodiments of the present disclosure;

[0011] Figure 4 A schematic structural diagram of an arm according to some embodiments of the present disclosure is shown;

[0012] Figure 5 A schematic diagram showing the current actual position of the end of an acquisition arm in a surgical robot system according to some embodiments of the present disclosure;

[0013] Figure 6 A schematic diagram illustrating a positioning tag including multiple pose identifiers and multiple angle identifiers according to some embodiments of the present disclosure is shown;

[0014] Figure 7 A schematic diagram showing a cylindrical positioning tag provided on the peripheral side of the end of an operating arm according to some embodiments of the present disclosure;

[0015] Figure 8 A schematic diagram illustrating obtaining the arm end position and posture using a binocular endoscope-based visual detection algorithm according to some embodiments of the present disclosure is shown;

[0016] Figure 9 A flow chart showing a method for determining the motion state of an arm according to some embodiments of the present disclosure;

[0017] Figure 10 A schematic diagram showing the forces acting on an arm according to some embodiments of the present disclosure is shown;

[0018] Figure 11 A schematic diagram illustrating determining the force applied to the end of an arm according to some embodiments of the present disclosure is shown;

[0019] Figure 12A Schematic diagram showing the loading conditions at various positions of the arm during the test experiment according to some embodiments of the present disclosure;

[0020] Figure 12B shows a curve of terminal force perception results in a test experiment according to some embodiments of the present disclosure;

[0021] Figure 13 A schematic block diagram showing a computer device according to some embodiments of the present disclosure;

[0022] Figure 14 A schematic diagram of a surgical robot system according to some embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0023] In order to make the technical problems solved by the present disclosure, the technical solutions adopted and the technical effects achieved more clear, the technical solutions of the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only exemplary embodiments of the present disclosure, rather than all embodiments.

[0024] In the description of the present disclosure, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present disclosure and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present disclosure. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In the description of the present disclosure, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected" and "coupled" should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a communication between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present disclosure can be understood according to the specific circumstances.

[0025] Those skilled in the art will appreciate that the embodiments of the present disclosure may be applicable to deformable arms provided on mechanical devices (e.g., surgical robots) that operate in a variety of environments, including but not limited to, on the surface, underground, underwater, in space, and in living organisms. In the present disclosure, the end close to the operator (e.g., a doctor) is defined as the proximal end, the near part, or the rear end, or the rear part, and the end close to the object of operation (e.g., a surgical patient) is defined as the distal end, the far end, or the front end, or the front part. In the present disclosure, the arm may include an arm distal portion at the distal end, and the distal end of the arm distal portion may be installed with an end instrument (e.g., an imaging device, a surgical actuator, etc.). For clarity, in the present disclosure, the end of the arm refers to the farthest-end portion of the arm and its end instrument. For example, when the arm distal portion of the arm is installed with an end instrument (e.g., an imaging device, a surgical actuator, etc.), the end of the arm may refer to the end instrument; when the arm distal portion of the arm is not installed with an end instrument (e.g., an imaging device, a surgical actuator, etc.), the end of the arm refers to the arm distal portion.

[0026] In this disclosure, the term "position" refers to the positioning of an object or a portion of an object in three-dimensional space (e.g., three translational degrees of freedom that can be described using changes in Cartesian X, Y, and Z coordinates, such as three translational degrees of freedom along the Cartesian X, Y, and Z axes, respectively). In this disclosure, the term "pose" refers to the rotational setting of an object or a portion of an object (e.g., three rotational degrees of freedom that can be described using roll, pitch, and yaw). In this disclosure, the term "pose" refers to the combination of the position and pose of an object or a portion of an object, such as can be described using six of the six degrees of freedom mentioned above.

[0027] Some embodiments of the present disclosure provide a method of determining force. Figure 1 A flow chart of a method 100 for determining force (hereinafter also referred to as "method 100") according to some embodiments of the present disclosure is shown. The method 100 may be implemented or executed by hardware, software, or firmware. In some embodiments, the method 100 may be performed by a surgical robot system (e.g., Figure 2 The surgical robot system shown or Figure 14 In some embodiments, the method 100 may be implemented as computer-readable instructions. These instructions may be executed by a general-purpose processor or a special-purpose processor (e.g., Figure 2 The control device 220 shown or Figure 14 For example, the control device of the surgical robot system (e.g., Figure 2 The control device 220 shown may include a processor configured to execute the method 100. In some embodiments, these instructions may be stored on a computer-readable medium.

[0028] Some embodiments of the present disclosure provide a surgical robot system. Figure 2 FIG. 2 shows a structural block diagram of a surgical robot system 200 according to some embodiments of the present disclosure. Figure 2As shown, the surgical robot system 200 may include a main control trolley 210, an operating trolley 230 and a control device 220. The control device 220 may be communicatively connected to the main control trolley 210 and the operating trolley 230, for example, by a cable connection or a wireless connection, to achieve communication between the main control trolley 210 and the operating trolley 230. The main control trolley 210 includes a master manipulator for remote operation by an operator, and a display for displaying an image of the operating area. The operating trolley 230 includes a slave tool for performing surgery, and the slave tool includes an arm body and an end instrument (such as an imaging device, a surgical actuator, etc.) arranged on the distal end of the arm body. The master-slave mapping between the master manipulator in the main control trolley 210 and the slave tool in the operating trolley 230 is achieved through the control device 220, thereby achieving motion control of the slave tool by the master manipulator. In some embodiments, the surgical trolley 230 includes a plurality of slave tools, and the plurality of slave tools are configured to enter the operating area through a sheath, and the end instrument of at least one slave tool (e.g., an imaging device) can obtain an image of the object to be operated (e.g., human tissue) in the operating area, and the end instrument of at least one slave tool (e.g., a surgical actuator) can contact the object to be operated (e.g., human tissue) in the operating area and generate a force. The sheath can be fixed at the patient's surgical port (e.g., an incision or a natural opening), and the operating area can be the area where the surgery is performed. In some embodiments, the imaging device can include but is not limited to a monocular endoscope, a binocular endoscope, etc., and the surgical actuator can include but is not limited to surgical forceps, an electric knife, an electric hook, etc.

[0029] In some embodiments, the arm may include a deformable robotic arm. For example, the arm may be a robotic arm with multiple degrees of freedom composed of multiple joints, such as a robotic arm that can achieve 6 degrees of freedom movement. For example, the arm may be a deformable continuum robotic arm.

[0030] Figure 3 Schematic diagram of a section 300 of an arm according to some embodiments of the present disclosure is shown. Figure 4 The arm body 400 shown, Figure 5 The surgical execution arm 510 and the visual guidance arm 520 are shown. Figure 14 The arm 1411) shown may include at least one deformable segment 300. Figure 3 As shown, the deformable structural segment 300 includes a fixed plate 310 and a plurality of structural bones 320. The first ends of the plurality of structural bones 320 are fixedly connected to the fixed plate 310, and the second ends are connected to a drive unit (not shown). In some embodiments, the fixed plate 310 can be a ring-shaped structure, a disc-shaped structure, etc., and the cross-section can be a variety of shapes such as circular, rectangular, polygonal, etc.

[0031] In some embodiments, the structural bone 320 in the deformable segment 300 can be made of an elastic material and have a certain degree of flexibility. For example, the material of the structural bone 320 can be a superelastic alloy, an air / liquid cavity, a memory alloy, a polymer structural material, etc., such as a nickel-titanium alloy. Based on the elastic properties of the structural bone 320, when subjected to an external force and / or the driving action of the driving unit (for example, a push-pull action), the deformable segment 300 can change in shape. The shape change of the deformable segment 300 can be manifested as bending deformation, telescopic deformation or twisting deformation, etc. For example, the driving unit drives the structural bone 320 to make the segment 300 be in the following state: Figure 2 The bent state shown. In some embodiments, the second ends of the multiple structural bones 320 pass through the base plate 330 and are connected to the drive unit, so that the drive unit drives the structural bones 320 to change the shape of the deformable segment 300. In some embodiments, similar to the fixed plate 210, the base plate 330 can include, but is not limited to, an annular structure, a disc-shaped structure, etc., and the cross-section can be various shapes such as circular, rectangular, polygonal, etc. In some embodiments, the drive unit can include a linear motion mechanism, a drive segment, or a combination of the two. The linear motion mechanism can be connected to the multiple structural bones 320 to push or pull the multiple structural bones 320, thereby driving the segment 300 to bend. The drive segment can include a fixed plate and multiple structural bones, wherein one end of the multiple structural bones is fixedly connected to the fixed plate. The other ends of the multiple structural bones of the drive segment are connected to the multiple structural bones 320 or are integrally formed, so that the bending of the drive segment drives the bending of the deformable segment 300.

[0032] In some embodiments, at least one spacer disk 340 is further included between the fixed disk 310 and the base disk 330. The first ends of the plurality of structural bones 320 pass through the at least one spacer disk 340 and are fixedly connected to the fixed disk 310. Similarly, the driving segment may also include a spacer disk.

[0033] Figure 4 FIG. 4 is a schematic diagram showing the structure of the arm 400 according to some embodiments of the present disclosure. Figure 4 As shown, the arm 400 is a continuous body robotic arm, and the arm 400 may include an arm distal end 410 and an arm body 420. The arm body 420 may include one or more segments, such as a first segment 4201 and a second segment 4202. In some embodiments, the structures of the first segment 4201 and the second segment 4202 may be the same as those of the embodiment of FIG. Figure 3 The structure section 300 shown is similar. In some embodiments, as Figure 4 As shown, the arm body 420 further includes a first straight rod segment 4203 located between the first segment 4201 and the second segment 4202. The first end of the first straight rod segment 4203 is connected to the base plate of the second segment 4202, and the second end is connected to the fixed plate of the first segment 4201. In some implementations, such as Figure 4 As shown, the operating arm body 420 also includes a second straight rod segment 4204, and the first end of the second straight rod segment 4204 is connected to the base plate of the first structural section 4201. In some embodiments, the arm body distal end portion 410 is located at the distal end of the arm body 420. In some embodiments, the arm body distal end portion 410 may include a columnar portion located at the distal end of the arm body 420. In some embodiments, the distal end of the arm body distal end portion 410 may be provided with an end instrument (e.g., an imaging tool, a surgical actuator, etc.) 430. In some embodiments, the outside of the arm body 400 may be wrapped with a coating or cover.

[0034] In some embodiments, the arm 400 and the segments it contains can be described by a kinematic model. In some embodiments, the structure of each segment can be specifically described as follows: Figure 3 The structure section 300 shown. Figure 3 As shown, the base coordinate system Attached to the base plate 330 of the i-th (i=1, 2, 3...) structure, its origin is located at the center of the base plate 330, and the XY plane coincides with the plane of the base plate 330, From the center of the base plate 330 to the first structural bone (the first structural bone can be understood as a structural bone arbitrarily designated as a reference from the multiple structural bones 320). Its origin coincides with the origin of the base coordinate system, and the XY plane coincides with the bending plane. and Coincident. Fixed disk coordinate system Attached to the fixed disk 310 of the i-th structural section, its origin is located at the center of the fixed disk 310, and the XY plane coincides with the plane of the fixed disk 310, From the center of the fixed plate 310 to the first structural bone. Its origin is located at the center of the fixed plate 310, and the XY plane coincides with the bending plane. and coincide.

[0035] In some embodiments, as Figure 3 The single segment 300 shown can be described by a kinematic model. The position of the end of the i-th segment (eg, the fixed disk coordinate system {ie}) relative to the base disk coordinate system {ib} ib p ie ,attitude ib R ie They can be expressed as the following formulas (1) and (2):

[0036]

[0037] ib R ie = ib Ri1 i1 R i2 i2 R ie (2)

[0038] Among them, L i is the virtual structural bone of the i-th segment (for example, Figure 3 The length of the virtual structural bone 321 shown in FIG; θ i is the bending angle of the i-th structural section, indicating that in the i-th structural section, about or Rotate to Required rotation angle; δ i is the bending direction angle of the i-th structural section, indicating that in the i-th structural section, the bending plane and The angle between ib R i1 is the posture of the bending plane coordinate system 1 {i1} of the i-th structural segment relative to the base coordinate system {ib}; i1 R i2 is the posture of the bending plane coordinate system 2{i2} of the i-th structural segment relative to the bending plane coordinate system 1{i1}; i2 R ie is the posture of the fixed disk coordinate system {ie} of the i-th structural section relative to the curved plane coordinate system 2{i2}.

[0039] ib R i1 、 i1 R i2 and i2 R ie It can be determined based on the following formulas (3), (4) and (5):

[0040]

[0041]

[0042]

[0043] like Figure 3 The node parameter ψ of a single node 300 is shown i It can be determined based on the following formula (6):

[0044] ψ i =[θ i ,δ i ] T (6)

[0045] In some embodiments, the driving amount of multiple structural bones and the joint parameters have a known mapping relationship. Based on the target joint parameters and the mapping relationship, the driving amount of multiple structural bones can be determined. The driving amount of multiple structural bones can be understood as the amount of each bone moving from the initial state (e.g., θ i =0) is the length of the structural bone pushed or pulled when the structural bone is bent to the target bending angle. In some embodiments, the mapping relationship between the driving amount of multiple structural bones and the joint parameters can be determined based on the following formula (7):

[0046] q ij =-r ij θ i cos(δ i -β ij ) (7)

[0047] Among them, q ij is the driving force of the jth structural bone in the i-th structural segment, r ij is the distance from the jth structural bone to the virtual structural bone in the i-th structural segment, β ij is the angle between the jth structural bone and the first structural bone in the i-th structural segment. The driving signal of the driving unit can be determined based on the driving amounts of the multiple structural bones.

[0048] In some embodiments, based on the overall rotation angle of the arm body in the arm body 400, the feed amount of the arm body (for example, the overall feed length of the arm body, the feed length of the arm body segment or the feed length of the straight rod segment in the arm body), and the individual segments that make up the arm body (for example, Figure 4 The structural parameters of the first structural section 4201 and the second structural section 4202 shown can determine the shape characteristic parameters of the arm body. For example, the shape characteristic parameters of the arm body can be determined based on the following formula (8):

[0049]

[0050] in, is the overall rotation angle of the arm, and d is the overall feed length of the arm. In some embodiments, the overall rotation angle of the arm The overall feed length d of the arm body is provided by the drive unit. For example, the overall feed length d of the arm body is provided by a linear drive mechanism that drives the linear feed of the arm body. The overall rotation angle of the arm body is Provided by a rotary drive mechanism that drives the arm body to rotate around its own central axis.

[0051] Those skilled in the art will appreciate that the arm has different shape characteristic parameters in different working states. For example, Figure 4 The arm 400 shown includes at least four working states, which correspond to four positions of the arm 400, which can be recorded as positions C1-C4. The four working states of the arm 400 are as follows:

[0052] First working state (C1 position): Only the second segment 4202 participates in the posture control of the control device (for example, only the second segment 4202 enters the working space). At this time, the shape characteristic parameters of the arm 400 are shown in the following formula (9):

[0053]

[0054] Among them, ψ c1 is the shape characteristic parameter of the arm body 400 in the first working state, is the overall rotation angle of the arm body 400, L2 is the feed length of the second segment 3202, and L2 is the same as Figure 3 The L in the structure section 300 is shown t The physical meaning of is the same, ψ2 is the node parameter of the second node 4202, and ψ2 can be determined by the above formula (6).

[0055] Second working state (C2 position): The second segment 4202 and the first straight segment 4203 participate in the posture control of the control device (for example, the second segment 4202 enters the working space in its entirety, and the first straight segment 4203 partially enters the working space). At this time, the shape characteristic parameters of the arm 400 are expressed as follows:

[0056]

[0057] Among them, ψ c2 is the shape characteristic parameter of the arm body 400 in the second working state, L r is the feed length of the first straight line segment 4203.

[0058] The third working state (C3 position): the second segment 4202, the first straight segment 4203, and the first segment 4201 participate in the position control of the actuator (for example, the second segment 4202 enters the working space in its entirety, the first straight segment 4203 enters the working space in its entirety, and the first segment 4201 partially enters the working space). At this time, the shape characteristic parameters of the arm 400 are as shown in the following formula (11):

[0059]

[0060] Among them, ψ c3 is the shape characteristic parameter of the arm body 400 in the third working state, L1 is the feed length of the first segment 4201, and L1 is related to Figure 3 The L in the structure section 300 is shown t The physical meanings of are the same, ψ1 is the node parameter of the first node 4201, ψ2 is the node parameter of the second node 4202, and ψ1 and ψ2 can be determined by the above formula (6).

[0061] Fourth working state (C4 position): the second segment 4202, the first straight segment 4203, the first segment 4201, and the second straight segment 4204 participate in the position control of the actuator (for example, the second segment 4202 enters the working space in its entirety, the first straight segment 4203 enters the working space in its entirety, the first segment 4201 enters the working space in its entirety, and the second straight segment 4204 partially enters the working space). At this time, the shape characteristic parameters of the arm 400 are as shown in the following formula (12):

[0062]

[0063] Among them, ψ c4 is the shape characteristic parameter of the arm body 400 in the fourth working state, L s is the feed length of the second straight line segment 4204.

[0064] In some embodiments, similar to a single segment, the driving amount of each structural bone of each segment of the arm body can be determined based on formula (7), and then the driving signal of the driving unit can be determined based on the driving amount.

[0065] Figure 1 A flow chart illustrating a method 100 of determining force according to some embodiments of the present disclosure is shown.

[0066] See Figure 1 In step 101, the current actual position of the target part of the arm is obtained. In the present disclosure, the target part of the arm may include the end of the arm or a part of the arm body.

[0067] In some embodiments, method 100 may include using an electromagnetic sensor to obtain the current actual position of the target portion of the arm. The electromagnetic sensor may include an electromagnetic induction device and a plurality of electromagnetic reflectors discretely arranged at multiple positions of the arm. The electromagnetic induction device relies on the mutual induction principle of the electromagnetic field to determine the position information (including position information and posture information) of the electromagnetic reflectors discretely arranged at the multiple positions of the arm, thereby obtaining the current actual position of the target portion of the arm.

[0068] In some embodiments, the method 100 may include obtaining an image of the arm and analyzing the obtained image to determine the current actual position of the target portion of the arm.

[0069] In some embodiments, the arm may be provided with a positioning tag, and an image capture device may be used to obtain a positioning image of the positioning tag. The positioning image may be processed to obtain the current actual position of the target portion of the arm. The image capture device may include, but is not limited to, a dual-lens image capture device or a single-lens image capture device, such as a binocular or monocular camera.

[0070] In some embodiments, it may also include obtaining images of the arm through medical imaging equipment such as computed tomography (CT), magnetic resonance imaging (MRI), stereoscopic vision, etc., and determining the current actual position of the target part of the arm based on analysis of the image.

[0071] In some embodiments, the arm can be a continuum robotic arm (e.g., Figure 4 The arm body 400 is shown, the arm body including at least one structural segment, each of which includes a fixation plate, multiple structural bones, and at least one spacer plate. The multiple structural bones pass through the at least one spacer plate and are fixedly connected to the fixation plate at first ends. The second ends of the multiple structural bones are connected to the drive unit. In some embodiments, the target portion includes the end of the arm body, and the current actual position includes the current actual position of the end.

[0072] In some embodiments, the end of the arm may include an end instrument (e.g., a surgical actuator) mounted at the distal end of the arm, and the current actual position of the end may be the current actual position of the end instrument. Method 100 may include obtaining the current actual position of the end of the arm using a visual detection algorithm. In some embodiments, a positioning image is captured by an image acquisition device, and the positioning image may include an image of part or all of the arm. In some embodiments, a positioning tag is provided at the distal end of the arm or in the vicinity thereof, and the positioning tag is used to detect the end position or end posture of the arm. The positioning tag may include multiple posture identifiers and angle identifiers. In some embodiments, the positioning tag is within the observation field of the image acquisition device, and the captured positioning image may include a positioning image for positioning the end of the arm, and the end position or end posture of the arm is obtained through the positioning image. In some embodiments, method 100 may include obtaining a positioning image of the arm; identifying multiple identifiers located on the arm in the positioning image; and determining the current actual position of the end based on the multiple identifiers.

[0073] In some embodiments, a surgical robot system may include a vision guidance arm having an imaging device (e.g., a binocular endoscope) mounted at a distal end thereof and at least one surgical execution arm having a surgical actuator mounted at a distal end thereof. The imaging device mounted at the distal end of the vision guidance arm serves as an image acquisition device and can capture a positioning image of the surgical execution arm, thereby obtaining the current actual position of the distal end of the surgical execution arm through the positioning image. Figure 5 Schematic diagram showing the current actual position of the end of the acquisition arm in the surgical robot system 500 according to some embodiments of the present disclosure. Figure 5As shown, the surgical robot system 500 includes at least one surgical execution arm 510 and a visual guidance arm 520. A binocular endoscope 521 is installed on the distal end of the visual guidance arm 520, and a surgical actuator 511 is installed on the distal end of the surgical execution arm 510. In some embodiments, the distal end of the surgical execution arm 510 is provided with a positioning tag ( Figure 5 (not shown in the figure), the distal end of the surgical execution arm 510 is within the field of view of the binocular endoscope 521. The positioning image captured by the binocular endoscope 521 may include a positioning image of the distal end of the surgical execution arm 510. The current actual position of the distal end of the surgical execution arm 510 is obtained through the positioning image. In some embodiments, the positioning image from the binocular endoscope 521 can be received and processed to obtain the current actual position of the distal end of the surgical execution arm 510.

[0074] In some embodiments, as Figure 4 The arm body 400 shown (for example, the arm body main body 420 or the arm body distal end portion 410) is provided with a plurality of posture identifiers and at least one angle identifier. For example, the plurality of posture identifiers are distributed circumferentially on the arm body distal end portion 410, and the plurality of angle identifiers are distributed circumferentially on the arm body distal end portion 410. The plurality of posture identifiers and the plurality of angle identifiers are arranged axially side by side on the arm body distal end portion 410. For example, the plurality of posture identifiers and the plurality of angle identifiers are arranged on the outer surface of the columnar portion of the arm body distal end portion 410.

[0075] In some embodiments, each angle identifier has a positional association with one of the pose identifiers. Based on this positional association, the position of the pose identifiers can be used to determine the area where the angle identifiers are likely to be distributed. Alternatively, the position of the angle identifiers can be used to determine the area where the pose identifiers are likely to be distributed. The positional association can be determined based on the specific arrangement of the pose identifiers and the angle identifiers and can be pre-designed.

[0076] In some embodiments, the position association relationship may include an axial correspondence between the angle identifier and the posture identifier. For example, the position association relationship may include an axial offset. Based on the axial correspondence, when the position of one or more posture identifiers on the distal end of the arm is known, the area where the angle identifier may be located can be determined by offsetting the axial position by a certain distance. For example, the position association relationship may also include an axial oblique alignment, etc.

[0077] In some embodiments, multiple posture identifiers and multiple angle identifiers can be set on a label attached to the peripheral side of the distal end portion of the arm.

[0078] In some embodiments, the posture identification may include a posture identification pattern and a posture identification pattern corner point, and the angle identification may include an angle identification pattern and an angle identification pattern corner point. In some embodiments, the posture identification pattern and the angle identification pattern may be provided on a label attached to the distal end of the arm body, or may be printed on the distal end of the arm body, or may be a pattern formed by the physical structure of the distal end of the arm body itself, for example, it may include a depression or a protrusion and a combination thereof. In some embodiments, the posture identification pattern or the angle identification pattern may include a pattern formed by brightness, grayscale, color, etc. In some embodiments, the posture identification pattern and the angle identification pattern may include a pattern that actively (for example, self-luminous) or passively (for example, reflected light) provides information detected by an image acquisition device (for example, an imaging device mounted on the distal end of the arm body of the visual guidance arm of the robotic system, such as a binocular endoscope). Those skilled in the art should understand that in some embodiments, the posture of the posture identification may be represented by the posture of the posture identification pattern corner point coordinate system, and the posture of the angle identification may be represented by the posture of the angle identification pattern corner point coordinate system.

[0079] Figure 6 A schematic diagram of a positioning tag 600 including multiple pose identifiers and multiple angle identifiers according to some embodiments of the present disclosure is shown. Figure 7 A schematic diagram of a cylindrical positioning tag 700 disposed around the distal end of an arm according to some embodiments of the present disclosure is shown. It is understood that for simplicity, the positioning tag 600 and the positioning tag 700 may include the same posture identification pattern and angle identification pattern.

[0080] See Figure 6 , multiple pose identifiers (in this disclosure, the corner points of the pose identifier pattern are represented by the "○" symbol) and multiple angle identifiers (in this disclosure, the corner points of the angle identifier pattern are represented by the "△" symbol) are arranged side by side. The multiple pose identifier patterns 611 can be the same or similar, and the multiple pose identifier pattern corner points are located in the multiple pose identifier patterns 611. The multiple angle identifier patterns 621-626 can be different, and the multiple angle identifier pattern corner points are located in the multiple angle identifier patterns 621-626.

[0081] Each angle identifier may have a positional association relationship with one of the pose identifiers. Figure 6 As shown, in the direction indicated by the arrow, some posture markers (eg, posture marker pattern 611) and corresponding angle markers (eg, angle marker pattern 621) are arranged along the arrow direction and have an interval distance d1. Figure 7 In the circumferential arrangement, the tag 600 becomes a cylindrical tag 700, and the positional association relationship between each angle mark and one of the posture marks may include the angle mark and the posture mark in the axial direction (such as Figure 7Based on the corresponding relationship in the axial direction, when the positions of one or more posture identifiers on the distal end of the arm are known, the area where the angle identifier may exist can be determined by offsetting a certain distance (such as distance d1) in the axial direction. In some embodiments, the corresponding relationship between the angle identifier and the posture identifier in the axial direction can be represented by the corresponding relationship between the angle identifier pattern corner point and the posture identifier pattern corner point in the axial direction. In some embodiments, based on the corresponding relationship between the angle identifier and the posture identifier in the axial direction, the projection of the angle identifier pattern corner point coincides with that of one of the posture identifier pattern corner points along the Z-axis direction.

[0082] In some embodiments, the angle around the axis or the roll angle of the angle mark or the posture mark can be represented by the angle around the axis of the angle mark pattern corner point or the posture mark pattern corner point. The angle mark pattern corner point is relative to the arm body coordinate system (for example, the coordinate system established at the distal end of the arm body, such as Figure 7 The angles of the XY coordinate system shown are known or predetermined, e.g. Figure 7 The angle of the angle identification pattern corner point R7 in the XY coordinate system with the X-axis is θ. Based on the position association relationship, it can be obtained that the angle of the posture identification pattern corner point P7 associated with its position relative to the X-axis is angle θ. It should be understood that the angle θ corresponding to the angle identification pattern corner point R7 and the posture identification pattern corner point P7 can be called the axis angle or roll angle around the Z axis of the angle identification or posture identification. In this disclosure, the axis angle or roll angle refers to the angle around the Z axis. It can be understood that for the sake of clarity, Figure 7 The angle identification pattern corner point R7 and the pose identification pattern corner point P7 are shown as separated in FIG, but they are coincident.

[0083] In some embodiments, the arm body can be obtained by a visual detection algorithm (e.g., Figure 4 The arm body 400 shown, Figure 5 The surgical execution arm 510 and the visual guidance arm 520 are shown. Figure 14 In some embodiments, the method 100 may include obtaining a positioning image of a positioning tag disposed at a distal end of the arm; identifying, in the positioning image, a plurality of identifiers (including a position identifier and an angle identifier) ​​located on the arm; and determining, based on the plurality of identifiers, the current actual position of the end of the arm, wherein the current actual position of the end includes the current actual position of the end and the current actual attitude of the end. Figure 8 Schematic diagram showing the actual current position of the end of the arm 800 obtained by the visual detection algorithm based on the binocular endoscope 821 according to some embodiments of the present disclosure. Figure 8 In the embodiment, the arm 800 is as follows Figure 4The end of the arm 400 and the arm 800 shown may include an end instrument (e.g., a surgical actuator) mounted at the distal end of the arm. Therefore, the current actual posture of the end of the arm 800 refers to the current actual posture of the end instrument (e.g., the current actual posture of the surgical actuator). In some of the following embodiments, the end instrument is described by taking the surgical actuator as an example. Figure 8 The definition of the coordinate system is explained: surgical actuator coordinate system and Figure 4 The same, please refer to Figure 4 The surgical actuator coordinate system {tip} is attached to the surgical actuator, which is fixed at the distal end of the distal end portion 410 of the arm body and at a set distance from the fixed plate coordinate system {2e} at the end of the second segment 4202 in the arm body 420. and The same direction, The direction of Figure 4 or Figure 8 Sheath coordinate system and Figure 4 It is consistent with the sheath outlet, With the first segment base coordinate system {1b} The same direction, The direction of Figure 4 or Figure 8 Positioning label coordinate system Attached to the positioning label set on the peripheral side of the distal end of the arm body, the origin is the center of the cross-section circle where the corner points of the multiple posture identification patterns are located. The axis direction is from the origin to one of the corner points of the pose identification pattern. The direction is parallel to the axis of the distal end of the arm, Axis perpendicular to Plane. Binocular endoscope left lens coordinate system Attached to the left lens, its origin is located at the center of the left lens, The direction is parallel to the left lens optical axis, The direction of Figure 8 As shown. Binocular endoscope right lens coordinate system Attached to the right lens, its origin is located at the center of the right lens, The direction is parallel to the right lens optical axis, The direction of Figure 8 In some embodiments, as shown Figure 8As shown, the left lens and the right lens of the binocular endoscope 821 can respectively obtain positioning images, and the position of the positioning tag in the binocular endoscope lens coordinates can be estimated by using a visual detection algorithm based on an image of the binocular endoscope 821 (the left lens image or the right lens image) (for example, the position of the positioning tag in the binocular endoscope lens coordinates can be obtained by using a method similar to that disclosed in some embodiments of the Chinese patent document CN115708128A). For example, the position of the positioning tag in the binocular endoscope left lens coordinate system {ll} ( ll p wm , ll R wm ) or the position of the positioning tag in the right lens coordinate system {rl} of the binocular endoscope ( rl p wm , rl R wm ),in, ll p wm 、 rl p wm Indicates location, ll R wm 、 rl R wm Indicates the posture. The following is the posture of the positioning tag in the left lens coordinate system {ll} of the binocular endoscope ( ll p wm , ll R wm ) is used as an example to illustrate. The position of the positioning tag in the left lens coordinate system of the binocular endoscope ( ll p wm , ll R wm ) can be determined by image analysis, and the homogeneous transformation matrix of the positioning tag relative to the left lens coordinate system of the binocular endoscope can also be obtained ll T wm Therefore, the homogeneous transformation matrix of the surgical actuator coordinate {tip} at the end of the arm 800 relative to its sheath coordinate system {tc} can be expressed as the following formula (13):

[0084]

[0085] in, represents the homogeneous transformation matrix of the surgical actuator coordinate {tip} relative to its sheath coordinate system {tc} obtained based on the visual detection algorithm, represents the current actual position of the end of the arm obtained based on the visual detection algorithm (i.e., the current actual position of the surgical actuator relative to the sheath coordinate system), is the actual current position of the end of the arm, is the actual current posture of the end of the arm, tc Tll Represents the homogeneous transformation matrix of the left lens coordinate system of the binocular endoscope relative to the sheath coordinate system, ll T wm Represents the homogeneous transformation matrix of the positioning tag coordinate system relative to the left lens coordinate system of the binocular endoscope, wm T tip is the homogeneous transformation matrix of the surgical actuator coordinates relative to the positioning tag coordinate system.

[0086] In some embodiments, the binocular endoscope is fixed to the distal end of the vision guidance arm (e.g., Figure 5 As shown, the binocular endoscope 521 is installed at the distal end of the visual guidance arm 520), and the current actual position of the end of the visual guidance arm can be obtained in the same way as in some of the above embodiments (for example, the current actual position of the end of the visual guidance arm corresponds to the current actual position of the binocular endoscope). Based on the current actual position of the end of the visual guidance arm, the positions of the left and right lenses in the binocular endoscope can be obtained respectively, and then the transformation matrix of the binocular endoscope lens coordinate system relative to the sheath coordinate system can be determined, for example, the homogeneous transformation matrix of the binocular endoscope left lens coordinate system relative to the sheath coordinate system tc T ll In some embodiments, the surgical actuator is fixedly arranged at the distal end of the distal end of the surgical execution arm, and the positioning tag is fixedly arranged at the distal end of the arm (for example, the positioning tag is fixedly arranged at the outer surface of the columnar portion of the distal end of the arm). wm T tip is known or predetermined. In some embodiments, based on tc T ll 、 ll T wm 、 wm T tip It has been determined that the actual current position of the end of the arm can be obtained based on formula (13) Based on the actual current pose of the end of the arm The actual position of the end of the arm can be obtained

[0087] Continue to read Figure 1 In step 103, the current actual driving amount of the arm is obtained. In some embodiments, the arm can be a continuous body robot arm (e.g., Figure 4The arm body 400 shown in FIG. 4 includes at least one structural section, each of which includes a fixation plate, multiple structural bones, and at least one spacer plate. The multiple structural bones pass through the at least one spacer plate, and the first ends of the multiple structural bones are fixedly connected to the fixation plate. The second ends of the multiple structural bones are connected to the drive unit. The current actual drive amount of the arm body may include the current actual drive amount output by the drive unit to the multiple structural bones of the arm body.

[0088] In some embodiments, the operator issues a control instruction for controlling the movement of the arm (for example, the control instruction includes drive information), and the drive unit responds to the control instruction and realizes the movement of the structural bone by pushing and / or pulling the structural bone, thereby meeting the operator's operational requirements for the arm. In some embodiments, the current control instruction of the arm can be obtained, and the drive information of the arm (for example, a drive amount vector) can be obtained from the current control instruction. Based on the drive information of the arm, the current actual drive amount output by the drive unit to the structural bone of the arm is obtained. In some embodiments, the state (for example, rotation angle or stroke) of the drive device (for example, a drive motor) can be sensed by a sensor (for example, a potentiometer) to obtain the current actual drive amount of the structural bone of the arm.

[0089] In some embodiments, the driving amount vector of the arm may include the driving amounts q of multiple structural bones of the arm ij , ij is the number of the structural bone, and the driving amount of multiple structural bones constitutes the driving amount vector of the structural bone, which can be recorded as q a,bkb For example, Figure 4 In the arm body 400 shown, the deformation of the i-th structural section is achieved by driving two groups of symmetrically distributed structural bones by a driving unit. In some embodiments, each group of symmetrically distributed structural bones can include two symmetrically distributed structural bones (for example, the angle between the two structural bones is π). In some embodiments, the deformation of the i-th structural section can be achieved by driving (for example, pushing and pulling at the same time) two groups of symmetrically distributed structural bones, for example, by driving two pairs of double-headed screw driving mechanisms. In some embodiments, the double-headed screw driving mechanism may include a double-headed screw and two threaded sliders located on the double-headed screw. When the double-headed screw rotates, the two threaded sliders located on the double-headed screw perform linear motion in opposite directions at the same speed. The two threaded sliders drive the two symmetrically distributed structural bones to perform linear motion in opposite directions at the same speed, so that the two symmetrically distributed structural bones are pushed or pulled to achieve bending deformation of the section. In some embodiments, the two groups of structural bones of the i-th structural section are represented by structural bones numbered i1 and i2, respectively. Therefore, the driving amount vector of the structural bone can be expressed as: q a,bkb =[q 11 ,q 12 ,q 21 ,q 22 ] TIn some embodiments, the current driving amount vector of the structural bone can be obtained, and the current driving amount vector of the structural bone is used as the current actual driving amount of the arm, which can be expressed as q a,bkb .

[0090] Continue to read Figure 1 In step 105, a first drive amount residual of the arm in a free motion state is determined based on the current actual drive amount and the drive amount residual calculation model. In this disclosure, the free motion state of the arm refers to the motion state of the arm when it is driven by the drive unit without the action of an external load force.

[0091] Due to factors such as nonlinear drive hysteresis and / or tensile and compressive deformation of structural bones within the arm body that are difficult to model, drive residuals for the structural bones within the arm body always exist. In some embodiments, the first drive residual of the arm body in a free motion state may include the first drive residual values ​​of multiple structural bones in the arm body in a free motion state, with each structural bone corresponding to a first drive residual value, and the first drive residual values ​​of the multiple structural bones are combined into a first drive residual vector.

[0092] In order to compensate for the driving amount residual that is difficult to model when the arm is without external load force, in some embodiments of the present disclosure, a driving amount residual calculation model without external load force is trained through model design and a series of random trajectory motion data. The trained driving amount residual calculation model can predict the driving amount residual under actual driving amount conditions and without external load force, that is, the driving amount residual calculation model can predict the driving amount residual of the arm in a free motion state under actual driving amount conditions. In the present disclosure, the driving amount residual predicted by the driving amount residual calculation model is recorded as the first driving amount residual, and the first driving amount residual can be expressed as For example, taking the arm body that drives four sets of symmetrically distributed structural bones as an example, ij=11,12,21,22, is the first driving amount residual of the ijth structural bone.

[0093] In some embodiments, the driving amount residual calculation model can be a machine learning model, which can be based on a machine learning algorithm to construct and train a machine learning model for calculating the first driving amount residual of at least one structural bone (e.g., four structural bones) of a continuum robotic arm in a free state. In some embodiments, the driving amount residual calculation model can include a support vector regression model, referred to as an SVR (Support Vector Regression) model. The SVR model can directly predict the actual driving amount of the arm (e.g., the driving amount vector q of the structural bone of the current given arm) a,bkb ) is driven by the first driving quantity residual when there is no external load force. The first driving quantity residual can be expressed as

[0094] In some embodiments, the training data set of the driving amount residual calculation model may include: a driving amount residual data set of the arm in a free motion state under different actual driving amount conditions.

[0095] In some embodiments, the SVR model can transform the original data space into a high-dimensional linearly separable space through the kernel function, thereby achieving a better regression effect for nonlinear data. At the same time, the SVR model characterizes the entire data feature through a limited number of support vectors and corresponding weight coefficients. Therefore, the SVR model can achieve fast prediction calculations and ensure the real-time control of the arm. In some embodiments, independent SVR models can be created for each structural bone in the continuous robotic arm. For example, in some embodiments, Figure 4 In the arm 400 shown, the deformation of the i-th segment is achieved by driving two sets of symmetrically distributed structural bones by the driving unit. The driving of the two sets of structural bones of the i-th segment is represented by the structural bones numbered i1 and i2 respectively. Figure 4 In the arm 400 including the first and second segments, four independent SVR models are created for the four representative structural bones (ij=11, 12, 21, 22) in the arm 400, as shown in the following formula (14):

[0096]

[0097] Among them, SV ij is the number of support vectors of the ijth SVR model (the SVR model used to predict the first driving amount residual of the ijth root structural bone), χ is the eigenvector of the input SVR model, χ ij,w and m ij,w are the wth support vector and its weight of the ijth SVR model, n ij is the constant term of the ijth SVR model, κ(χ ij,w ,χ) is the Gaussian kernel function of the SVR model. κ(χ ij,w ,χ) can be defined as follows:

[0098]

[0099] Among them, γ ij is the adjustable parameter of the Gaussian kernel function, γ ij > 0. Before training the SVR model, the optimal γ ij The parameters and the soft margin parameter C in the SVR model are determined by a grid traversal search strategy and cross-validation.

[0100] In some embodiments, the coupling force relationship between the structural bones in the continuum manipulator is taken into account. That is, the structural bones of the i-th segment not only affect the bending deformation within the segment, but also affect the deformation of the structural bones of other segments due to the coupling between the segments. In some embodiments, the eigenvectors χ of the four SVR models take into account the driving effects of all structural bones. That is, the eigenvectors χ of each SVR model are defined consistently, as shown in the following formula (16):

[0101]

[0102] Among them, h bkb =[h 11 ,h 12 ,h 21 ,h 22 ] is a hysteresis feature vector that characterizes the hysteresis characteristics of the structural bone drive, and the hysteresis feature vector represents the historical state of the structural bone drive process. In some embodiments, for time k, h ij (k) is expressed as an accumulated number with saturation characteristics, which can be expressed as the following formula (17):

[0103]

[0104] Among them, μ max represents the maximum driving hysteresis of each structural bone. For the continuum robotic arm of some embodiments of the present disclosure, μ max Set to 0.2mm. Δq a,ij is the driving increment of the structural bone numbered ij from time k-1 to time k.

[0105] In some embodiments, an SVR model for calculating the first driving amount residual of the four structural bones is determined based on formulas (14) to (17). In some embodiments, before using the SVR model, a large number of random data sets need to be collected to train the SVR model. For example, for the ijth SVR model, the collected random data set can be expressed as: χ1, χ2, ... are random data of eigenvectors, is the residual data of the first driving amount of the arm in the free motion state under the actual driving amount conditions corresponding to the eigenvector random data χ1, χ2, ..., Can be obtained through experiments.

[0106] In some embodiments, different arm configurations (e.g., C1-C4 configurations) may affect the training of the SVR model and its prediction results. Figure 4The arm body 400 shown is in the C3 position, and the overall feed length d of the arm body 400 directly changes the feed length L1 of the first segment 4201. Different L1 conditions will affect the training of the SVR model and its prediction results. In some embodiments, when performing SVR model training, L1 is first discretized from a predetermined motion range (for example, 40mm to 60mm) at predetermined intervals (for example, 5mm), and a sufficient number of random data sets are collected under each L1 condition to train the SVR model of the corresponding L1. In the process of calculating the first driving amount residual using the trained SVR model, the corresponding nearest set of SVR models is selected for prediction based on the size of the current L1. In some embodiments, the training and prediction of the SVR model can be implemented using the support vector machine library LIBSVM developed by Professor Chih-JenLin and others.

[0107] For example, L1=L x The random data collection process in this case can be, for example, carried out in the following manner:

[0108] Set L1 = L x , in, is the overall rotation angle of the arm, according to the maximum bending angle of the segments in the arm (e.g., θ 1,max =π / 2,θ 2,max =2π / 3) to determine the structural bone driving amount range, and randomly select N groups (for example, 3000 groups) of structural bone driving amount vectors q within the structural bone driving range. a,bkb , the sum of the driving vectors of each group of structural bones is L1=L x , A set of arm drive vectors For each set of driving quantity q a , the kinematic model is used to calculate the theoretical position of the positioning tag coordinate system {wm} relative to the binocular endoscope lens coordinate system, for example, the theoretical position of the positioning tag coordinate system {wm} relative to the binocular endoscope left lens coordinate system {ll} ll T wm, the driving vectors that exceed the visual detection range of the positioning tag on the distal end of the arm are eliminated. In some embodiments, the final random driving quantity trajectory can be obtained by performing interpolation calculations at a set interval (for example, 0.05mm) between the two adjacent driving vectors after the above elimination. The experimental platform is used to drive the arm to each given driving quantity in the random driving quantity trajectory under the condition of no external load, and the characteristic vector (χ) at each given driving quantity is calculated and recorded. In some embodiments, the end posture of the arm can be obtained by a visual detection algorithm as in some of the above embodiments. Shape reconstruction is performed based on the end posture, and the theoretical driving quantity of the arm at each given driving quantity is obtained based on the reconstructed shape. Specifically, it can be implemented in the manner described in some embodiments below. In some embodiments, based on the given driving quantity and the calculated theoretical driving quantity, a driving quantity residual can be obtained. Since the random data acquisition process is experimentally obtained without the action of external load force, the driving quantity residual obtained by the experiment can be considered as the first driving quantity residual of the arm in a free motion state. The first driving quantity residual obtained by the experiment is recorded as Thus, the recorded feature vectors at all given driving variables and the corresponding first driving variable residuals constitute a random data set. In some embodiments, a portion of the obtained random data set (e.g., 80% of the data in the random data set) is randomly extracted as a training data set, and the remaining data (e.g., the remaining 20% ​​of the data) is used as a test data set. The test data set is used to verify the prediction effect of the SVR model trained with the training data set.

[0109] Continue to see Figure 1 In step 107, the current ideal position of the target part of the arm is determined based on the current actual driving amount and the first driving amount residual. In some embodiments, the arm can be a continuous body robot arm (e.g., Figure 4 The arm body 400 shown in the figure comprises at least one structural section, which comprises a fixed plate, a plurality of structural bones and at least one spacer plate. The plurality of structural bones pass through the at least one spacer plate, and the first ends of the plurality of structural bones are fixedly connected to the fixed plate, and the second ends of the plurality of structural bones are connected to the driving unit.

[0110] In some embodiments, method 100 includes determining the ideal driving amount corresponding to the current free motion state of the arm body based on the current actual driving amount and the first driving amount residual. In the present disclosure, the first driving amount residual is the driving amount residual of the structural bones in the arm body obtained by taking into account factors such as the driving hysteresis and / or tensile and compressive deformation of the structural bones that are difficult to model and nonlinear in the arm body. In some embodiments, the ideal driving amount corresponding to the current free motion state of the arm body includes the ideal joint driving amount corresponding to the current free motion state of the arm body. The ideal structural bone driving amounts of multiple structural bones corresponding to the current free motion state of the arm body can be obtained by the current actual driving amount and the first driving amount residual. Based on the ideal structural bone driving amounts of multiple structural bones, the overall feed length of the arm body, and the overall rotation angle of the arm body, the ideal joint driving amount corresponding to the current free motion state of the arm body can be obtained.

[0111] For example, Figure 4 In the arm 400 shown, the deformation of the i-th segment is achieved by driving the two sets of symmetrically distributed structural bones by the driving unit. The specific manner in which the driving unit drives the segment to bend and deform is consistent with some of the above embodiments and will not be repeated here. Figure 4 The driving of the two groups of structural bones in the i-th segment of the arm 400 is represented by the structural bones numbered i1 and i2 respectively. The arm 400 includes two segments, namely the first segment 4201 and the second segment 4202. Therefore, there are four groups of structural bones in the arm 400. The driving of the four groups of structural bones in the arm 400 is represented by the structural bones numbered ij=11, 12, 21, and 22 respectively. The current actual driving amount of the arm can be obtained by the method in some of the above embodiments. The current actual driving amount of the arm can be recorded as q a,bkb =[q 11 ,q 12 ,q 21 ,q 22 ] T The first driving amount residual of the arm in the free motion state can be obtained by the method in some of the above embodiments. The first driving amount residual can be recorded as Based on the current actual driving amount q of the arm a,bkb and the residual of the first driving amount of the arm in the free motion state The ideal structural bone drive amount of multiple structural bones corresponding to the current free motion state of the arm can be determined, which can be recorded as It can be expressed as in, Indicates the ideal structural bone driving amount of the ijth structural bone corresponding to the current free motion state of the arm. Based on the ideal structural bone driving amount of the structural bone corresponding to the current free motion state of the arm, the overall feed length d of the arm, and the overall rotation angle of the arm The ideal joint drive value corresponding to the current free motion state of the arm can be obtained, which can be recorded as It can be expressed as In some embodiments, the overall feed length d of the arm is provided by the drive unit. In some embodiments, the arm has a slender section (e.g., greater than 200 mm in length) between the drive unit and the sheath, which will produce a certain degree of torsion under the external load force, so the overall rotation angle of the arm is The driving rotation angle that needs to be provided by the driving unit (for example, the rotation driving mechanism that drives the arm body to rotate around its own central axis) Add an additional offset angle based on Right now Among them, the additional offset angle In some embodiments, the shape of the arm (e.g., a continuum manipulator) can be reconstructed based on its shape information and / or end position to obtain the shape characteristic parameters of the arm. In the shape reconstruction process, the additional offset angle As a parameter to be estimated, the estimated value of the additional deflection angle is obtained by estimating The specific content can be implemented in some embodiments described later. It can be expressed as

[0112] In some embodiments, method 100 includes determining a current ideal position of a target portion of the arm based on an ideal drive amount corresponding to the current free motion state of the arm. In some embodiments, ideal posture information (including position information and posture information) of the arm in the free motion state can be obtained based on the ideal drive amount corresponding to the current free motion state of the arm and the kinematic model of the arm, thereby obtaining the current ideal position of the target portion of the arm.

[0113] In some embodiments, the arm comprises at least one segment, the segment comprising a fixation plate, multiple structural bones, and at least one spacer plate, the multiple structural bones passing through the at least one spacer plate, the first ends of the multiple structural bones being fixedly connected to the fixation plate, and the second ends of the multiple structural bones being connected to the drive unit. In some embodiments, the target portion comprises the end of the arm, and the current ideal position comprises the current ideal position of the end.

[0114] In some embodiments, the end of the arm may include an end instrument (e.g., a surgical actuator) mounted at the distal end of the arm, and the current ideal position of the end instrument may be the current ideal position of the end instrument. ) and the kinematic model of the arm can obtain the current ideal posture of the end of the arm in the free motion state. The current ideal posture of the end can be expressed as is the current ideal position of the end of the arm, The kinematic model of the arm has been described in detail in some of the above embodiments and will not be repeated here.

[0115] Continue to see Figure 1 In step 109, the force on the target part is determined based on the current actual position and the current ideal position.

[0116] In some embodiments, the method 100 includes determining the actual position deformation of the target part based on the current actual position and the current ideal position, for example, Figure 11 As shown, based on the current actual position and the current ideal position of the end of the arm, the actual position deformation Δp of the end of the arm is determined. load In some embodiments, if the actual position deformation of the target portion exceeds the position deformation threshold, the force on the target portion is determined based on the current actual position and the current ideal position. In some embodiments of the present disclosure, the position deformation threshold can be configured as an empirical value or can be predetermined.

[0117] In some embodiments, method 100 may further include determining a motion state of the arm. In some embodiments, the motion state includes a free motion state (e.g., a motion state without an external load force) and / or a constrained state (e.g., a motion state subject to an external load force). In some embodiments, based on the arm being in the constrained state, it is determined that the arm is subject to an external load force, and the force applied to the target portion is determined based on the current actual position and the current ideal position.

[0118] Figure 9 A flow chart of a method 900 (hereinafter referred to as "method 900") for determining the motion state of an arm according to some embodiments of the present disclosure is shown. The method 900 may be implemented or executed by hardware, software, or firmware. In some embodiments, the method 900 may be implemented or executed by a surgical robot system (e.g., Figure 2 The surgical robot system shown, Figure 5 The surgical robot system 500 or Figure 14 In some embodiments, the method 900 may be implemented as computer-readable instructions. These instructions may be executed by a general-purpose processor or a special-purpose processor (e.g., Figure 2 The control device 220 shown or Figure 14 For example, the control device of the surgical robot system (e.g., Figure 2The control device 220 shown may include a processor configured to execute the method 900. In some embodiments, these instructions may be stored on a computer-readable medium.

[0119] Some embodiments of the present disclosure disclose the reconstruction of the shape of the arm body based on the current shape information / end posture of the arm body (for example, a continuum manipulator) to obtain the shape characteristic parameters of the arm body. In some embodiments, the theoretical driving amount of multiple structural bones in the arm body caused by the change of the arm body shape can be calculated by the reconstructed shape characteristic parameters. Due to the existence of factors such as the driving hysteresis and / or tensile and compressive deformation of the structural bones that are difficult to model in the arm body, the driving amount residual of the structural bones in the arm body always exists. The driving amount residual is the difference between the actual driving amount output by the driving mechanism (such as the driving motor) in the driving unit and the theoretical driving amount calculated by the reconstructed shape characteristic parameters. The present disclosure records the difference between the actual driving amount and the theoretical driving amount as the second driving amount residual. In order to compensate for the driving amount residual that is difficult to model the arm body when there is no external load force, in some embodiments of the present disclosure, a driving amount residual calculation model when there is no external load force is trained by designing a support vector regression model and a series of random trajectory motion data. The trained driving amount residual calculation model can predict the driving amount residual under actual driving amount conditions and without external load force. In the present disclosure, the driving amount residual predicted by the driving amount residual calculation model under actual driving amount conditions and without external load force is recorded as the first driving amount residual. However, under the same actual driving amount conditions, the external load force will change the shape of the arm, thereby changing the second driving amount residual of each corresponding structural bone. Based on this, some embodiments of the present disclosure use the difference between the driving amount residual (second driving amount residual) calculated by the actual driving amount and the theoretical driving amount and the driving amount residual (first driving amount residual) predicted by the driving amount residual calculation model to determine the motion state of the arm (for example, free motion state or constrained state).

[0120] The following, combined Figure 9 The method 900 for determining the motion state of an arm according to some embodiments of the present disclosure is described in detail.

[0121] See Figure 9 In step 901, the current actual driving amount of the arm is obtained. In some embodiments, the same Figure 1 The current actual driving amount of the arm is obtained in the same manner as step 103. This has been described in detail in some of the above embodiments and will not be repeated here.

[0122] Continue to see Figure 9 In step 903, the current theoretical drive amount of the arm is determined. In some embodiments, the arm can be a continuum robot arm (e.g., Figure 4The arm body 400 shown in FIG. 1 includes at least one structural segment, each segment including a fixation plate and multiple structural bones. The first ends of the multiple structural bones are fixedly connected to the fixation plate, and the second ends of the multiple structural bones are connected to the drive unit. The current theoretical drive amount of the arm body can be the theoretical drive amount of the structural bones of the arm body based on the current shape of the arm body.

[0123] In some embodiments, the method 900 may include determining current shape characteristic parameters of the arm body; and determining a current theoretical driving amount based on the current shape characteristic parameters.

[0124] In some embodiments, method 900 may include obtaining current shape information of the arm body, and determining current shape feature parameters based on the current shape information.

[0125] In some embodiments, the arm includes a shape sensor, and the current shape information of the arm can be obtained based on the shape sensor, and the current shape characteristic parameters of the arm can be determined based on the current shape information. For example, a measurement signal can be received from the shape sensor, and the current shape information of the arm can be determined based on the measurement signal. In some embodiments, the measurement signal can include the curvature of the arm at multiple positions along the axial direction, and the current shape information of the arm can be determined based on the curvature. In some embodiments, the curvature at multiple positions can be continuousized by a continuation algorithm to determine the current shape information of the arm. For example, the continuation algorithm can be an interpolation algorithm. In this way, the shape information of the arm determined based on the continuous curvature is more accurate.

[0126] In some embodiments, the number of shape sensors can be one or more. In some embodiments, the shape sensors can be evenly or unevenly arranged on the arm. In some embodiments, the shape sensor comprises a fiber Bragg grating (FBG) sensor, comprising gratings (Gr) located at multiple locations along the arm's axial direction. Measurement signals can be obtained from the Gratings, and the arm's shape can be reconstructed based on the measurement signals. The current shape information of the arm can be obtained based on the reconstructed arm shape. Fiber Bragg grating (FBG) sensors utilize optical measurement, have good electromagnetic compatibility, and can obtain accurate shape measurement data. Optical measurement methods occupy less space within the deformable robotic arm, making data acquisition more flexible. By placing shape sensors within the arm, the overall shape of the arm can be determined. Fiber Bragg grating (FBG) sensors, as existing medical imaging equipment in hospitals, are reusable and offer the advantage of low-cost data measurement. In some embodiments, the shape sensor can also be an electromagnetic sensor. The electromagnetic sensor can include an electromagnetic induction device and multiple electromagnetic reflectors discretely arranged at multiple locations on the arm. The electromagnetic induction device uses the principle of mutual induction of electromagnetic fields to determine the position information of the discrete electromagnetic reflectors. The arm's shape is reconstructed based on the discrete position information, and the current shape information of the arm is obtained based on the reconstructed arm shape. In some embodiments, the current shape information of the arm can also be obtained using medical imaging equipment such as computed tomography (CT), magnetic resonance imaging (MRI), and stereoscopic vision.

[0127] In some embodiments, method 900 may include obtaining the current actual posture of the end of the arm body, and determining the current shape feature parameters based on the current actual posture of the end of the arm body. In some embodiments, the current actual posture of the end of the arm body can be obtained based on an electromagnetic sensor, which has been described in detail in some of the above embodiments and will not be repeated here. In some embodiments, the current actual posture of the end of the arm body can also be obtained through a visual detection algorithm, which has been described in detail in some of the above embodiments and will not be repeated here.

[0128] In some embodiments, the method 900 may further include determining the current shape feature parameters by minimizing the difference between the theoretical end position and the current actual position of the end based on the current actual position of the end and the kinematic model of the arm.

[0129] In some embodiments, the bending deformation of each segment in the arm can be estimated based on the constant curvature assumption. The constant curvature means that the curvature magnitude κ(s) of the center curve of each segment in the arm is a fixed constant along with its arc length variable s. For example, Figure 3 As shown, in the i-th section, the curvature change information of the bending deformation of the section can be expressed as θ i ,θ i for about or Rotate to The required rotation angle. A single segment 300 can be described by a kinematic model. In some embodiments, the position of the end of the i-th segment (e.g., the fixed disk coordinate system {ie}) relative to the base disk coordinate system {ib} can be determined using formulas (1) to (5) as described in some of the above embodiments. ib p ie ,attitude ib R ie The node parameter ψ of a single node 300 is i It can be determined based on formula (6) in some of the above embodiments. Thus, the shape characteristic parameters of the arm body can be determined according to the working state of the arm body (for example, C1-C4 position). For example, the shape characteristic parameter ψ of the arm body can be determined based on formula (9) to formula (12). For example, Figure 4 The shape characteristic parameters of the arm body 400 shown in the C3 position can be expressed as:

[0130] In some embodiments, the bending deformation of the segments in the arm body can be estimated based on the linear curvature assumption. The linear curvature means that the curvature magnitude κ(s) of the center curve of each segment in the arm body changes linearly with its arc length variable s. For example, Figure 3 As shown in the figure, in the i-th section, the curvature change information of the bending deformation of the section can be expressed as κ i (s i ), κ i (s i ) can be expressed as the following formula (18):

[0131] κ i (s i )=a i s i +b i (18)

[0132] Among them, κ i (s i ) represents the center curve of the i-th segment (for example, the center curve corresponding to the virtual structural bone of the i-th segment) in the arc length s i The curvature at a i and b i are the slope and intercept of the curvature change of the i-th section. i and κ i (s i The conversion relationship between them can be shown as the following formula (19):

[0133]

[0134] Among them, L iis the virtual structural bone of the i-th segment (for example, Figure 3 The length of the virtual structural bone 321) shown in FIG.

[0135] The node parameters of a single node 300 can be expressed as ψ i Indicates that ψ i It can be expressed as the following formula (20):

[0136] ψ i =[a i ,b i ,δ i ] T (20)

[0137] Among them, δ i is the bending direction angle, indicating that in the i-th structural section, the bending plane and Angle.

[0138] like Figure 3 The single segment 300 shown can be represented by a kinematic model. The position of the end of the i-th segment (fixed disk coordinate system {ie}) relative to the base disk coordinate system {ib} ib p ie ,attitude ib R ie They can be expressed as the following formulas (21) and (22):

[0139]

[0140] ib R ie = ib R i1 i1 R i2 i2 R ie (twenty two)

[0141] Among them, L i is the virtual structural bone of the i-th segment (for example, Figure 3 The length of the virtual structural bone 221) shown in ib R i1 is the posture of the bending plane coordinate system 1 {i1} of the i-th structural segment relative to the base plate coordinate system {ib}, i1 R i2 is the posture of the bending plane coordinate system 2{i2} of the i-th structural section relative to the bending plane coordinate system 1{i1}, i2 R ie is the posture of the fixed disk coordinate system {ie} of the i-th segment relative to the curved plane coordinate system 2 {i2}. Similar to some of the above embodiments, ib R i1 、i1 R i2 and i2 R ie It can be determined based on formula (3), formula (4) and formula (5) in some embodiments above.

[0142] Thus, the shape characteristic parameters of the arm body can be determined according to the working state of the arm body (e.g., C1-C4 position). For example, the shape characteristic parameters ψ of the arm body can be determined based on formulas (9) to (12). For example, Figure 4 The shape characteristic parameters of the arm body 400 shown in the C3 position can be expressed as:

[0143] In some embodiments, as Figure 4 In the illustrated arm 400, the length of the first segment 4201 is significantly greater than the length of the second segment 4202. For example, L1 is 2 to 3 times the length of L2, where L1 represents the length of the virtual structural bone of the first segment 4201 and L2 represents the length of the virtual structural bone of the second segment 4202. Those skilled in the art will appreciate that the lengths of the first segment 4201 and the second segment 4202 can be configured based on the actual application scenario. When loaded, the first segment 4201 more closely resembles a linear curvature shape, and the shape error caused by the bending deformation of the first segment 4201 estimated based on the linear curvature assumption is also smaller. When loaded, the second segment 4202 more closely resembles a constant curvature shape, and the shape error caused by the bending deformation of the second segment 4202 estimated based on the constant curvature assumption is also smaller. Therefore, in some embodiments, the bending deformation of the first segment 4201 can be estimated based on the linear curvature assumption, while the bending deformation of the second segment 4202 can be estimated based on the constant curvature assumption. The estimation of the bending deformation of the segment based on the linear curvature assumption and the estimation of the bending deformation of the segment based on the constant curvature assumption have been described in detail in some of the above embodiments and will not be repeated here. Based on formula (20) in some of the above embodiments, the segment parameter of the first segment 4201 can be obtained by using ψ1, and based on formula (6) in some of the above embodiments, the segment parameter of the second segment 4202 can be obtained by using ψ2. ψ1 and ψ2 can be shown as the following formulas (23) and (24), respectively:

[0144] ψ1=[a1,b1,δ1] T (twenty three)

[0145] ψ2=[θ2,δ2] T (twenty four)

[0146] Wherein, a1 and b1 represent the slope and intercept of the curvature change of the first segment 4201, respectively, and δ1 represents the curvature plane and the curvature of the first segment 4201. θ2 represents the angle of the second segment 4202. about or Rotate to The required rotation angle, δ2 represents the bending plane in the second segment 4202 and Angle.

[0147] like Figure 3 The single segment 300 shown can be represented by a kinematic model. Based on formulas (21) and (22) in some of the above embodiments, the position of the end of the first segment 4201 (fixed disk coordinate system {1e}) relative to the base disk coordinate system {1b} is 1b P 1e ,attitude 1b R 1e They can be expressed as the following formulas (25) and (26) respectively.

[0148]

[0149] 1b R 1e = 1b R 11 11 R 12 12 R 1e (26)

[0150] Based on formula (1) and formula (2) in some of the above embodiments, the position of the end of the second segment 4201 (fixed disk coordinate system {2e}) relative to the base disk coordinate system {2b} is 2b P 2e ,attitude 2b R 2e They can be expressed as the following formulas (27) and (28):

[0151]

[0152] 2b R 2e = 2b R 21 21 R 22 22 R 2e (28)

[0153] in, 2b R 21 、 21 R 22 、 22 R 2e It can be determined based on formula (3), formula (4) and formula (5) in some of the above embodiments.

[0154] Thus, the shape characteristic parameters of the arm body can be determined according to the working state of the arm body (e.g., C1-C4 position). For example, the shape characteristic parameters ψ of the arm body can be determined based on formulas (9) to (12). For example, Figure 4 The shape characteristic parameters of the arm body 400 shown in the C3 position can be expressed as:

[0155] In some embodiments, the shape of the arm is reconstructed based on the shape assumption of the arm and the current actual position of the end of the arm to determine the current shape characteristic parameters of the arm.

[0156] In some embodiments, the arm shape reconstruction problem is expressed as the following nonlinear optimization problem, as shown in formula (29):

[0157]

[0158] in, The shape characteristic parameters of the arm that need to be optimized, is the current theoretical end pose of the arm, is the current theoretical end position of the arm, Indicates the current theoretical end pose of the arm. It is represented by the actual current position of the end of the arm. In some embodiments, The actual current position of the end of the arm can be obtained based on the visual detection algorithm. In some embodiments, The shape assumption based on the arm body may be determined by adopting the shape assumption method in some of the above embodiments.

[0159] In some embodiments, the current theoretical end-of-arm pose can be described by a kinematic model based on the shape assumption of the arm. Figure 4 The pose of a single segment (e.g., the first segment 4201 or the second segment 4202) in the arm 400 shown can be determined based on an estimation of the bending deformation of the segments in the arm in combination with a kinematic model. For example, the position of the end of the i-th segment (e.g., the fixed disk coordinate system {ie}) relative to the base disk coordinate system {ib} is obtained based on the segment parameters of the single segment. ib p ie ,attitude ib R ie The specific content has been described in detail in some of the above embodiments and will not be repeated here. Figure 4 The end position of the entire arm 400 shown can be described by a kinematic model, so that the theoretical end position of the arm can be obtained when the shape characteristic parameters of the arm are given. Figure 4The homogeneous transformation matrix of the surgical actuator coordinate {tip} at the end of the arm 400 relative to its sheath coordinate system {tc} can be expressed as shown in formula (30):

[0160]

[0161] in, represents the homogeneous transformation matrix of the surgical actuator coordinate {tip} relative to its sheath coordinate system {tc} obtained based on the kinematic model, represents the theoretical end posture of the arm 300 (for example, the posture of the surgical actuator relative to its sheath coordinate system obtained by the kinematic model), represents the theoretical end position of the arm 300 (for example, the position of the surgical actuator coordinate relative to its sheath coordinate system obtained by the kinematic model), tc T 1b represents the homogeneous transformation matrix of the base plate of the first segment 4201 relative to the sheath coordinate system, 1b T 1e represents the homogeneous transformation matrix of the fixed disk of the first structural section 4201 relative to the base disk of the first structural section 4201, 1e T 2b represents the homogeneous transformation matrix of the base plate of the second segment 4202 relative to the fixed plate of the first segment 4201, 2b T 2e represents the homogeneous transformation matrix of the fixed disk of the second structural section 3202 relative to the base disk of the second structural section 4202, 2e T tip represents the homogeneous transformation matrix of the surgical implement relative to the fixed disk of the second segment 4202. In some embodiments, the surgical implement is fixedly disposed at the distal end of the fixed disk of the second segment 4202, and thus, 2e T tip In some embodiments, the base plate of the second segment 4202 is connected to the fixed plate of the first segment 4201 via the first straight rod segment 4203, so that 1e T 2b is known or predetermined, for example, in some embodiments, Figure 4 The first structural bones of the first segment 4201 and the second segment 4202 of the arm body 400 shown in the figure differ by γ2 on the cross section of the arm body 400. The length of the first straight rod segment 4202 is L r (That is, the distance between the base plate of the second segment 4202 and the fixed plate of the first segment 4201 can be approximately L r ),thus, 1e T 2b =Rot z (γ2)·Transz (L r In some embodiments, the base plate of the first segment 4201 is connected to the second straight rod 4204, and the position of the sheath tube relative to the base plate of the first segment 4201 is determined by the overall rotation angle of the arm body. Sure, can be obtained directly from the drive unit, so tc T 1b It can be determined in advance.

[0162] In some embodiments, the shape characteristic parameters of the arm body to be optimized are It can be obtained based on the shape assumption of the arm, for example, by adopting the constant curvature assumption and / or the linear curvature assumption in the above embodiment. Figure 4 For the arm 400 shown, the bending deformation of the first segment 4201 is estimated based on the linear curvature assumption, and the bending deformation of the second segment 4202 is estimated based on the constant curvature assumption. The shape characteristic parameters of the arm 400 in the C3 position can be expressed as: The shape characteristic parameter vector of the arm to be optimized is It can be expressed as: is the parameter to be estimated. It should be understood by those skilled in the art that Figure 4 In the arm 400 shown, when estimating the bending deformation of each segment based on the constant curvature assumption, the shape characteristic parameter vector of the arm 400 that needs to be optimized in the C3 position can be expressed as: When estimating the bending deformation of each segment based on the linear curvature assumption, the shape characteristic parameter vector of the arm body 400 that needs to be optimized in the C3 position can be expressed as: is the parameter to be estimated.

[0163] In some embodiments, the feed amount of the arm (e.g., the overall feed length of the arm, the feed length of the segments in the arm, or the feed length of the straight rod segments in the arm) is provided by a driving unit (e.g., a linear driving mechanism that drives the linear feed of the arm). Figure 4 The arm 400 shown is in the C3 position, and the feed length L1 of the first segment 4201 is provided by the drive unit (e.g., a linear drive mechanism that drives the arm to feed linearly). In some embodiments, the arm has a slender section (e.g., greater than 200 mm in length) between the drive unit and the sheath. This slender section will produce a certain degree of torsion under external load, so the overall rotation angle of the arm is The driving rotation angle that needs to be provided by the driving unit (for example, the rotation driving mechanism that drives the arm body to rotate around its own central axis) Add an additional offset angle based on Right now Thus, illustratively, Figure 4 For the arm shown in the figure, when estimating the bending deformation of each segment based on the constant curvature assumption in the C3 position, the shape characteristic parameter vector of the arm that needs to be optimized can be expressed as: is the parameter to be estimated. When estimating the bending deformation of each segment based on the linear curvature assumption, the shape characteristic parameter vector of the arm to be optimized can be expressed as: When estimating the bending deformation of the first segment 4201 based on the linear curvature assumption and estimating the bending deformation of the second segment 4202 based on the constant curvature assumption, the shape characteristic parameter vector of the arm body to be optimized can be expressed as: is the parameter to be estimated.

[0164] In some embodiments, based on formula (29) Determined as the current shape feature parameters.

[0165] In some embodiments, the optimization problem of formula (29) can be solved, for example, using the open source C++ optimization library CeresSolver developed by Google. The corresponding cost function is calculated by formula (29), and the input of the cost function is The parameters to be estimated in .

[0166] In some embodiments, the method 900 includes determining a current theoretical driving amount based on the current shape characteristic parameters. In some embodiments, the method 900 may determine the current theoretical driving amount based on the current shape characteristic parameters and the relationship between the current shape characteristic parameters and the driving amount of the arm.

[0167] In some embodiments, in a segment of the arm, the driving amounts of the multiple structural bones and the segment parameters have a known mapping relationship. In some embodiments, the mapping relationship between the driving amounts of the multiple structural bones and the segment parameters can be determined based on formula (7). In some embodiments, similar to a single segment, the driving amount of each structural bone in each segment of the arm can be determined based on formula (7), and the driving signal of the driving unit can be determined based on the driving amount.

[0168] In some embodiments, for example Figure 4 The arm 400 shown in FIG. 1 is estimated based on formula (29) to obtain the current characteristic parameters of the arm The theoretical driving amount of the at least one structural bone caused by the change in the arm shape may be calculated, for example, the theoretical driving length of the at least one structural bone.

[0169] For example, Figure 4 In the arm body 400 shown, the deformation of the i-th structural section is achieved by driving two groups of symmetrically distributed structural bones by a driving unit. In some embodiments, each group of symmetrically distributed structural bones can include two symmetrically distributed structural bones (for example, the angle between the two structural bones is π). In some embodiments, the deformation of the i-th structural section can be achieved by driving two groups of symmetrically distributed structural bones by two pairs of double-headed screw driving mechanisms. In some embodiments, the double-headed screw driving mechanism can include a double-headed screw and two threaded sliders located on the double-headed screw. When the double-headed screw rotates, the two threaded sliders located on the double-headed screw perform linear motion in opposite directions at the same speed. The two threaded sliders drive the two symmetrically distributed structural bones to perform linear motion in opposite directions at the same speed, so that the two symmetrically distributed structural bones are pushed or pulled to achieve bending deformation of the section. In some embodiments, the two groups of structural bones of the i-th structural section are represented by structural bones numbered i1 and i2, respectively, and the theoretical driving amount of the structural bones can be recorded as: q shape,bkb =[q shape,11 ,q shape,12 ,q shape,21 ,q shape,22 ] T In some embodiments, as Figure 4 In the arm 400 shown, the driving of the second segment 4202 takes into account the coupling effect of the structural bone of the second segment 4202 through the channel of the first segment 4201. The theoretical driving amount of the structural bone q shape,bkb =[q shape,11 ,q shape,12 ,q shape,21 ,q shape,22 ] T It can be expressed as the following formula (31):

[0170]

[0171] Among them, r i1 is the distance from the structural bone i1 to the virtual structural bone in the i-th segment, r i2 is the distance from the structural bone i2 to the virtual structural bone in the i-th segment, β i1 is the angle between the structural bone i1 and the first structural bone in the i-th segment, β i2 is the angle between the structural bone i2 and the first structural bone in the i-th segment, and γ is the offset angle of the second segment 4202 relative to the first segment 4201. To estimate the current characteristic parameters of the arm based on formula (29): The estimated values ​​of the parameters to be estimated in are determined.

[0172] In some embodiments, in the i-th structural segment, the angle between the structural bones numbered i1 and i2 may be π / 2, and the structural bone i1 may be used as the first structural bone. Therefore, β 11 =0,β 21 =0,β 12 =π / 2,β 22 =π / 2. In some embodiments, the offset angle γ of the second segment 4202 relative to the first segment 4201 is =π / 4.

[0173] In some embodiments, the bending deformation of the first segment 4201 is estimated based on the linear curvature assumption, and the bending deformation of the second segment 4202 is estimated based on the constant curvature assumption. The parameters to be estimated can be obtained by formula (29): The estimated value of The estimated value of The estimated value of can be expressed as Those skilled in the art should understand that by estimating the bending deformation of each segment based on the constant curvature assumption, the parameters to be estimated can be directly obtained through formula (29): The estimated value of ; Based on the linear curvature assumption, the bending deformation of each node is estimated, and the estimated parameter can be obtained by formula (29) The estimated value of The estimated value of The estimated value of The estimated value of The estimated value of can be expressed as

[0174] Continue to see Figure 9 In step 905, based on the current actual driving amount and the driving amount residual calculation model, the first driving amount residual of the arm in the free motion state is determined. Figure 1 In some embodiments, the same method as step 105 can be used to determine the first driving amount residual of the arm in the free motion state. The specific content has been described in detail in some of the above embodiments and will not be repeated here.

[0175] Continue to read Figure 9 In step 907, a second driving amount residual of the arm is determined based on the current actual driving amount and the current theoretical driving amount. In some embodiments, the second driving amount residual of the arm may be a second driving amount residual of a structural bone of the arm.

[0176] In some embodiments, the current theoretical drive amount of the arm can be obtained by reconstructing the shape of the arm. In some embodiments, the arm can be a continuum robot arm (e.g., Figure 4The arm body 400 shown in the figure includes at least one structural section, the structural section includes a fixed plate and multiple structural bones, the first ends of the multiple structural bones are fixedly connected to the fixed plate, and the second ends of the multiple structural bones are connected to the driving unit. The current theoretical driving amount of the arm body can be based on the current shape of the arm body and the theoretical driving amount of the structural bones of the arm body. For example, Figure 4 In the arm 400 shown, the deformation of the i-th segment is achieved by driving the two sets of symmetrically distributed structural bones by the driving unit. The specific manner in which the driving unit drives the segment to bend and deform is consistent with some of the above embodiments and will not be repeated here. Figure 4 The two groups of structural bone drives in the i-th segment of the arm 400 are represented by structural bones numbered i1 and i2 respectively. The theoretical drive amount of the representative structural bones can be recorded as q shape,ij , ij=11,12,21,22, the theoretical driving quantities of multiple structural bones constitute the theoretical driving quantity vector of the structural bones, which can be recorded as: q shape,bkb =[q shape,11 ,q shape,12 ,q shape,21 ,q shape,22 ] T In some embodiments, the shape of the arm (e.g., a continuum robotic arm) is reconstructed based on its shape information and / or end position to obtain shape characteristic parameters of the arm. The reconstructed shape characteristic parameters can be used to calculate the theoretical driving amount of the structural bones in the arm caused by the shape change of the arm (e.g., q shape,ij ), the details of which will be described later. In some embodiments, the current theoretical driving amount vector of the structural bone can be obtained (for example, q shape,bkb ), the current theoretical driving amount vector of the structural bone is used as the current theoretical driving amount of the arm, that is, the current theoretical driving amount of the arm can be expressed as q shape,bkb .

[0177] In some embodiments, the current theoretical driving amount q of the arm is obtained by reconstructing the shape of the arm. shape,bkb When calculating the drive hysteresis and / or elastic tensile and compressive deformation of each structural bone, the influence of other factors is not taken into account. Therefore, each structural bone has a drive residual. In this disclosure, the drive residual calculated by the actual drive amount of the arm and the theoretical drive amount of the arm is recorded as the second drive residual. The second drive residual can be expressed as the following formula (32):

[0178] q rsd,bkb =q a,bkb -q shape,bkb ,q rsd,bkb ≠0 (32)

[0179] For example, taking the arm body that drives four sets of symmetrically distributed structural bones as an example, q rsd,bkb =[q rsd,11 ,qrsd,12 ,q rsd,21 ,q rsd,22 ] T , where q rsd,ij =q ij -q shape,ij ,ij=11,12,21,22.

[0180] Continue to read Figure 9 In step 909, the motion state of the arm is determined based on the first driving amount residual and the second driving amount residual. In some embodiments, the motion state includes a free motion state (e.g., a motion state without an external load force) and / or a constrained state (e.g., a motion state with an external load force).

[0181] In some embodiments, the method 900 includes obtaining a difference in the driving residual amount based on the first driving residual amount and the second driving residual amount, for example, as shown in the following formula (33):

[0182]

[0183] in, is the first driving quantity residual, q rsd,bkb is the second driving quantity residual, is the difference in driving residuals.

[0184] For example, taking the arm body driving four sets of symmetrically distributed structural bones as an example, q rsd,bkb =[q rsd,11 ,q rsd,12 ,q rsd,21 ,q rsd,22 ] T , in, is the difference of the driving residual of the ijth structural bone, Calculated by a driving amount residual calculation model (eg, an SVR model), ij=11, 12, 21, 22.

[0185] In some embodiments, based on the difference between the first driving amount residual and the second driving amount residual being higher than a threshold, it is determined that the arm is in a constrained state at the current moment (for example, the arm moves under the drive of the driving unit and is affected by an external load force at the current moment).

[0186] In some embodiments, based on the fact that the difference between the first driving amount residual and the second driving amount residual is not higher than a threshold, it is determined that the arm is in a free motion state at the current moment (for example, the arm moves only under the drive of the driving unit at the current moment without being affected by any external load force).

[0187] In some embodiments, the difference of the driving residuals is obtained based on formula (33), and the difference of the driving residuals of at least one structural bone is higher than a threshold value, and it is determined that the arm is in a constrained state at the current moment. For example, taking the arm of four sets of symmetrically distributed structural bones as an example, when When , it is determined that the arm is in a constrained state at the current moment. is the difference of the driving residual of the ijth structural bone, ξ ij is the difference threshold of the driving residual amount.

[0188] In some embodiments, the difference of the driving residuals is obtained based on formula (33), and the difference of the driving residuals of all structural bones is not higher than the threshold value, and it is determined that the arm is in a free motion state at the current moment. For example, taking the arm of the four symmetrically distributed structural bones as an example, when , it is determined that the arm is in a free motion state at the current moment.

[0189] In some embodiments, each structural bone corresponds to a driving residual difference threshold ξ ij ,ξ ij The minimum value ξ ij，min The size of is related to the average prediction error of the driving quantity residual calculation model (such as the SVR model).

[0190] Continue to read Figure 1 In step 109, the force on the target part is determined based on the current actual position and the current ideal position. For example, in response to the arm being in a constrained state (for example, ), to determine whether the arm is subjected to external load force.

[0191] In some embodiments, method 100 further includes obtaining a current stiffness of the target portion; and determining a force applied to the target portion based on the current actual position, the current ideal position, and the current stiffness. In the present disclosure, the force applied to the target portion can be an external force acting on the target portion. For example, the external force can include a force generated when the target portion of the arm contacts the tissue being operated on during a surgical procedure.

[0192] In some embodiments, the actual position deformation amount of the target portion can be determined based on the current actual position and the current ideal position. The actual position deformation amount of the target portion can be a position difference vector generated after the target position deformation occurs.

[0193] In some embodiments, a current stiffness of the target portion corresponding to the direction of the actual position deformation can be obtained, and the force on the target portion can be determined based on the actual position deformation of the target portion and the current stiffness. In some embodiments, the force on the target portion is determined as the product of the actual position deformation of the target portion and the current stiffness based on Hooke's law.

[0194] In some embodiments, the arm body includes at least one segment, the segment includes a fixed plate, a plurality of structural bones and at least one spacer plate, the plurality of structural bones pass through the at least one spacer plate and the end is fixedly connected to the fixed plate. In some embodiments, the target portion includes the end of the arm body, the current actual position includes the current actual position of the end, and the current ideal position includes the current ideal position of the end. Method 100 may include determining the actual position deformation of the end of the arm body based on the current actual position of the end and the current ideal position of the end. Method 100 may also include obtaining the current stiffness of the end of the arm body corresponding to the direction of the actual position deformation, and may determine the force on the end of the arm body based on the actual position deformation and the current stiffness of the end of the arm body.

[0195] In some embodiments, method 100 further includes: obtaining an average end stiffness of the end of the arm; and determining an end virtual force of the end of the arm based on the current actual position of the end, the current ideal position of the end and the average end stiffness.

[0196] In some embodiments, the end of the arm body may include an end instrument (e.g., a surgical actuator) mounted at the distal end of the arm body. Therefore, the end average stiffness of the end of the arm body may be the average stiffness of the end instrument. In embodiments of the present disclosure, the end average stiffness of the end of the arm body may be determined based on the material properties of the end of the arm body.

[0197] In some embodiments, the actual position deformation amount of the end of the arm is determined based on the current actual position of the end and the current ideal position of the end, for example, Figure 11 As shown, the actual position deformation of the end of the arm can be determined by the following formula (34):

[0198]

[0199] Where Δp load is the actual position deformation of the end of the arm, is the actual current position of the end, The current ideal position of the end.

[0200] Based on Hooke's law, the product of the actual position deformation of the end of the arm and the average stiffness of the end is determined as the end virtual force of the end of the arm, for example, Figure 11 As shown, the end virtual force of the end of the arm can be determined by the following formula (35):

[0201]

[0202] Among them, f virtual is the terminal virtual force, is the average stiffness of the tip, for example, in some implementations, Can be set to 20N / m, Δp load is the actual position deformation of the end of the arm.

[0203] In some embodiments, method 100 also includes determining the end virtual position deformation of the end of the arm body based on the current actual driving amount of the arm body, the end virtual force and the mechanical model of the arm body, wherein the mechanical model is determined based on the distribution of multiple structural bones of the arm body on the cross-section of the arm body and the physical properties of the multiple structural bones.

[0204] In some embodiments, the mechanical model may include a constitutive relationship related to the structural bone. The constitutive relationship may represent the material properties of the structural bone, for example, the material properties may be expressed through the internal forces and deformations of the structural bone.

[0205] In some embodiments, the arm includes multiple structural bones (j is the structural bone number, j = 1, 2, 3 ... m), and the internal force constitutive relationship of the arm can be determined based on the internal force constitutive relationship of the structural bones. The internal force constitutive relationship of the arm can be determined based on the shear-tensile stiffness matrix of the multiple structural bones.

[0206] In some embodiments, the arm includes a constraint structure (e.g., a spacer disk, a fixing disk, a coating, etc.) and a structural bone. The internal force constitutive relationship of the arm can be determined based on the constraint structure of the arm and the internal force constitutive relationship of the structural bone. For example, the internal force constitutive relationship of the deformable robotic arm is shown in formula (36),

[0207]

[0208] In formula (36), n all is the internal force of the arm, R is the rotation matrix, K SE is the shear-tension stiffness matrix of the arm's restraining structure, K SEj is the shear-tensile stiffness matrix of the jth structural bone in the arm, v is the linear velocity of the arm's posture along the arc length of the reference line, for example, the reference line can be the central axis of the arm, v min is the linear velocity of the arm in its natural state. For example, v min It can be the linear velocity when there is no external force or driving action, v min =[0 0 1] T .

[0209] In some embodiments, the arm includes multiple structural bones (j is the structural bone number, j = 1, 2, 3 ... m), and the internal moment constitutive relationship of the arm can be determined based on the internal moment constitutive relationship of the structural bones. The internal moment constitutive relationship of the arm can be determined based on the bending-torsion stiffness matrix of the structural bones.

[0210] In some embodiments, the arm includes a constraint structure and a structural bone, and the internal moment constitutive relation of the arm can be determined based on the constraint structure of the arm and the internal force constitutive relation of the structural bone. For example, the internal moment constitutive relation of the arm is shown in formula (37):

[0211]

[0212] In formula (37), m all is the internal torque of the arm, R is the rotation matrix, K BT is the bending torsional stiffness matrix of the arm's constraint structure, K BTj is the bending torsional stiffness matrix of the jth structural bone in the arm, u is the angular velocity of the arm's posture along the arc length of the reference line, and u min is the angular velocity in a natural state, for example, the angular velocity when not subject to external forces or driving forces, u min =0.

[0213] In some embodiments, the mechanical model includes a mechanical equilibrium relationship associated with the structural bone. The mechanical equilibrium relationship includes a force balance relationship of the structural bone, and the force balance relationship includes the force balance at various locations along the axial direction of the structural bone. In some embodiments, the force balance relationship includes the balance between the external force and the internal force of the structural bone at the force-bearing location along the axial direction.

[0214] In some embodiments, the mechanical balance relationship includes a force balance relationship of the arm body, and the force balance relationship includes force balance at various locations along the axial direction of the structural bone. In some embodiments, the force balance relationship of the arm body includes that the external force and the internal force of the arm body at the force-bearing location along the axial direction are balanced.

[0215] Figure 10 FIG2 shows a schematic diagram of the force acting on the arm 1000 according to some embodiments. Figure 10 In (a), for a structural unit [s,s+Δs] of the arm, the force balance relationship of the structural unit [s,s+Δs] can be seen from formula (38).

[0216]

[0217] The arm can include a constraining structure (e.g., a spacer disc, a fixing disc, a covering layer, etc.) and a structural bone. In formula (38), n(s) represents the internal force of the arm skeleton at position s, n(s+Δs) represents the internal force of the arm's constraining structure at position (s+Δs), Δs is a small increment, and f e (ξ) represents the distributed external force of the arm at ξ. j (s) represents the internal force of the j-th structural bone at position s, n j (s+Δs) represents the internal force of the j-th structural bone at (s+Δs).

[0218] Based on formula (38), the force balance relationship in the arm body is obtained, as shown in formula (39),

[0219] n′ all +f e =0 (39)

[0220] In formula (39), n all is the internal force of the arm, ()′ represents the derivative, f e is a distributed external force (such as the weight of the arm). In some embodiments, the distributed external force can be ignored.

[0221] In some embodiments, the mechanical balance relationship of the arm includes a moment balance relationship, and the moment balance relationship includes the moment of the structural bone along the axial direction. Figure 10 In (b), for a structural unit [s,s+Δs] of the arm, the moment balance relationship of the structural unit [s,s+Δs] can be seen from formula (40).

[0222]

[0223] In formula (40), m(s) represents the internal moment of the arm's constraint structure at s, m(s+Δs) represents the internal moment of the arm's constraint structure at s+Δs, and Δs is a small increment. p(s+Δs) represents the position of the arm's constraint structure at s+Δs, and n(s+Δs) represents the internal force of the arm's constraint structure at s+Δs. p(s) represents the position of the arm's constraint structure at s, and n(s) represents the internal force of the arm's constraint structure at s. e (ξ) represents the distributed moment of the arm's constraint structure at ξ (in some embodiments, the distributed moment can be ignored), p(ξ) is the position of the arm's constraint structure at ξ, and f e (ξ) represents the distributed external force of the arm's constraint structure at ξ. m j (s) represents the internal moment of the jth structural bone at position s, m j (s+Δs) represents the internal moment of the jth structural bone at s+Δs. j (s+Δs) represents the position of the jth structural bone at s+Δs, n j (s+Δs) represents the internal force of the jth structural bone at s+Δs. j (s) represents the position of the jth structural bone at s, n j (s) represents the internal force of the jth structural bone at position s.

[0224] Based on formula (40), the moment balance relationship of the arm at position s is obtained, see formula (41),

[0225]

[0226] In formula (41), m all is the internal moment of the arm, p is the position of the reference line of the arm, n all is the internal force of the arm, l e is the distributed moment of the arm, which can be ignored in some embodiments. R is the rotation matrix of the reference line of the arm, r j is the distribution of the jth structural bone on the cross section of the arm, for example, r j It can be the distribution coordinates of the structural bone on the cross section of the arm, u is the angular velocity of the arm posture along the arc length of the reference line, K SEj The shear-tensile stiffness matrix of the jth structural bone, is the linear strain, which represents the difference in linear velocity of the j-th structural bone along the arc length of the reference line before and after the deformation occurs.

[0227] In some embodiments, a moment boundary condition of the arm at the end of the arm is applied, and the moment boundary condition includes that the sum of the moments of the arm at the end of the arm is zero, see formula (42),

[0228]

[0229] In formula (42), m e is the external torque of the arm at the end L of the arm, m(L) is the internal torque of the arm at the end L of the arm, and R(L) is the rotation matrix of the arm at the end L of the arm.

[0230] In some embodiments, a force boundary condition is applied to the arm at the end of the arm. The force boundary condition includes that the sum of the forces on the arm at the end of the arm is zero, see formula (43),

[0231] n e -n(L)=0 (43)

[0232] In formula (43), n e is the external force of the arm at the end L of the arm, and n(L) is the internal force of the arm at the end L of the arm.

[0233] In some embodiments, the mechanical model of the arm includes the relationship between the length variation of the structural bones along the axial direction and the distribution of the structural bones on the cross section of the arm. For example, the length variation of each structural bone in the arm (j is the structural bone number, j = 1, 2, 3, ..., m) along the axial direction is shown in formula (44),

[0234] q′ j =||v+u^r j||-1 (44)

[0235] In formula (44), q j is the length change of the jth structural bone along the axial direction, r j is the distribution of structural bones in the cross section of the arm, for example, r j It can be the distribution position coordinates of the structural bone on the cross section of the arm, v is the linear velocity of the arm posture changing along the arc length of the reference line, and u is the angular velocity of the arm posture changing along the arc length of the reference line.

[0236] In some embodiments, the length change q of the structural bone j (j is the number of the structural bone, j = 1, 2, 3, ..., m) can be related to the driving information, for example, the driving information can be the information of the driving mechanism driving the structural bone to move. In some embodiments, the length change of the structural bone can also be related to the deformation of the arm. For example, the deformation of the arm can be the telescopic deformation of its structural bone. In some embodiments, the length boundary condition of the structural bone is applied, and the length boundary condition includes the length change of the structural bone at the end q j (L) is equal to the length drive q aj and length deformation ε j The driving information of the structural bones includes the length driving amount. For example, the length change of all the structural bones in the arm along the axial direction can be found in formula (45).

[0237] q(L)=q a +L all ε (45)

[0238] In formula (45), q(L)=[q1(L)q2(L)...q m (L)] T represents the length of each structural bone at s = L, q a =[q a1 q a2 ... q am ] T Indicates the driving length of each structural bone, L all represents the total length of the structural bone, ε=[ε1 ε2 ... ε m ] T represents the linear strain on each structural bone. For example, ε j It can be a percentage, L all ε j Represents the expansion and contraction deformation of the jth structural bone.

[0239] Formula (38) to formula (45) constitute the mechanical model of the arm. In some embodiments, the driving length q of each structural bone is given a , given the known distributed external force fe , distributed torque I e , external torque m e and external force n e Under the condition of , the end position of the end of the arm can be calculated. For example, in some embodiments, the current actual driving amount of the arm is obtained, such as q a,bkb , and the terminal virtual forces, such as f virtual , then let q a =q a,bkb , n e =f virtual , m e =0, distributed external force f e and distributed torque I e can be ignored, then f e =0,I e = 0, solve the mechanical model of the arm and calculate the end virtual posture of the end of the arm (including the end virtual position and end virtual posture). For example, the end virtual posture can be recorded as ( tc p tip (f virtual ), tc R tip (f virtual )),in, tc p tip (f virtual ) is the end virtual position, tc R tip (f virtual ) is the terminal virtual posture.

[0240] In some embodiments, the terminal virtual position deformation amount can be determined based on the terminal virtual position and the current actual position of the terminal, for example, Figure 11 As shown, the end virtual deformation amount Δp of the end of the arm virtual It can be determined by the following formula (46):

[0241]

[0242] Where Δp virtual is the virtual position deformation of the end, The actual current position of the end.

[0243] In some embodiments, based on the end virtual force and the end virtual position deformation, the current stiffness of the end of the arm body can be determined. For example, it can be determined by the following formula (47):

[0244] k stiffness =||f virtual || / ||Δp virtual || (47)

[0245] Among them, k stiffness is the current stiffness of the end, f virtual is the virtual force at the end, Δp virtual is the virtual position deformation of the end.

[0246] In some embodiments, the method 100 includes determining the force on the end of the arm based on the current actual position of the end, the current ideal position of the end, and the current stiffness of the end. For example, the actual position deformation variable Δp of the end of the arm can be obtained based on the current actual position of the end and the current ideal position of the end using formula (34) as described in some embodiments above: load The current stiffness k of the end can be obtained by using formula (47) as some of the above implementations stiffness Based on Hooke's law, the actual position deformation Δp of the end of the arm is used load The current stiffness of the terminal can determine the force on the terminal. For example, the force on the terminal can be determined using the following formula (48):

[0247]

[0248] in, The force at the end.

[0249] In some of the above embodiments, formula (48) assumes that the force direction of the end of the arm body is related to the actual position deformation Δp of the end of the arm body. load The parallel direction avoids the uncertainty in force perception caused by the force direction. This directional uncertainty arises from the anisotropic stiffness of the end of the arm (the axial stiffness of the end of the arm is infinite). However, for the lateral force perception of the end of the arm, the directional error of the force generated by this assumption is acceptable.

[0250] Figure 11 Schematic diagram showing the force applied to the end of the arm according to some embodiments of the present disclosure. In some embodiments, the actual position deformation of the end of the arm can be determined based on the current actual position of the end and the current ideal position of the end, for example, Figure 11 As shown, the current actual position of the end can be the actual driving amount of the arm (for example, q a,bkb ) and the arm is loaded (for example, the force at the end is is the parameter to be estimated) the actual position of the end of the end of the arm body; the ideal position of the end can be the ideal driving amount (for example, ) and the arm is in a free motion state (e.g., no load); the actual position deformation of the arm end can be obtained by using formula (34) as in some of the above embodiments (e.g., Figure 11 Δp shownload Based on the actual position deformation and the average stiffness of the end of the arm, the end virtual force of the end of the arm can be obtained by using formula (35) as in some of the above embodiments (for example, Figure 11 f shown virtual ). Based on the ideal drive amount of the arm (for example, ) and the arm is loaded (for example, the end of the arm is subjected to the end virtual force f virtual After the arm is calibrated, the virtual position of the end of the arm can be obtained based on the mechanical model of the arm. Based on the current actual position of the end of the arm and the virtual position of the end, the virtual deformation of the end of the arm can be obtained using formula (46) (for example, Figure 11 Δp shown virtual Those skilled in the art will appreciate that due to the anisotropic stiffness of the end of the arm, the end virtual force (e.g., f virtual ) and terminal dummy shape variables (e.g., Δp virtual ) have a certain direction difference. Based on the end virtual force and end virtual deformation of the end of the arm, the current end stiffness of the end of the arm (for example, k stiffness Based on the actual position deformation of the end of the arm and the current stiffness of the end, the force on the end can be determined by using formula (48) in some of the above embodiments.

[0251] In some embodiments, when the actual position deformation of the end of the arm (for example, Figure 11 Δp shown load ) changes, the corresponding end current stiffness (e.g., k stiffness ) is updated and calculated. For example, the updated current stiffness of the terminal can be calculated using the methods in some of the above embodiments. The force on the terminal can be determined based on the updated current stiffness of the terminal and the actual position deformation of the terminal of the arm.

[0252] In some embodiments, the method further comprises performing a response operation based on the force on the target portion. In some embodiments, the response operation comprises feeding back the force on the target portion and / or indicating the force information on the target portion. In some embodiments, the target portion may be, for example, the end of an arm. In a surgical robot system (e.g., Figure 2 The surgical robot system 200 shown, Figure 5 The surgical robot system 500 or Figure 14During a surgical procedure, the surgical robot system 1400 (shown) obtains the force applied to the end of the arm and feeds the force applied to the end of the arm back to the main control vehicle and / or sends the force information applied to the end of the arm to the main control vehicle and indicates the relevant force information. In some embodiments, based on the feedback of the force applied to the target part and / or the indicated force information applied to the target part, an operator at the main control station (e.g., a doctor) can be guided to control the movement of the arm to perform the surgical procedure.

[0253] In some embodiments, the static force sensing accuracy of the disclosed method 100 is tested through experiments. Figure 4 The arm 400 shown in FIG. 1 has 11 arm configurations (Cfg) with different bending plane distributions. In some configurations, only one direction (-x or +y) of loading is applied, while in other configurations, loading is applied in two directions (-x and +y). The loading conditions at each configuration are shown in FIG. Figure 12A According to the test results of the SVR model, the difference threshold value ξ of the driving residual value for load judgment in the experiment ij During the experiment, the arm was first driven to 11 pre-planned Cfg positions without load, and then 10g, 20g, and 50g weights were placed in each loading direction. When the arm was stable, the end force estimated by the method 100 of the present disclosure under each loading condition (one loading size in one loading direction) was recorded. and the actual position deformation Δp of the end load The terminal force sensing results under the two loading directions of -x and +y are as follows: Figure 12B As shown in (a) and (b), Figure 12B In the figure, the solid line, dashed line, and dotted line represent the 10g, 20g, and 50g loads, respectively. The average perceived force errors for all loading conditions (including -x and +y directions) with 10g, 20g, and 50g weights are 4.20 (2.47, 1.75, and 1.92 in the x, y, and z directions, respectively), 4.77 (2.36, 1.56, and 3.11 in the x, y, and z directions, respectively), and 8.20 (3.01, 2.51, and 6.43 in the x, y, and z directions, respectively) grams, respectively. In specific applications, such as providing auxiliary force feedback to doctors, the perceived force errors (e.g., lateral force perception errors in the -x and +y directions) of the above experiments are acceptable.

[0254] In some embodiments of the present disclosure, the present disclosure further provides a computer device, the computer device including a memory and a processor. The memory may be used to store at least one instruction, and the processor is coupled to the memory and is used to execute at least one instruction to perform some or all steps in the method of the present disclosure, for example, Figure 1 Some or all of the steps in the method 100 disclosed in, or Figure 9 Some or all of the steps in method 900 disclosed in.

[0255] Figure 13 FIG1 shows a schematic block diagram of a computer device 1300 according to some embodiments of the present disclosure. Figure 13 The computer device 1300 may include a central processing unit (CPU) 1301, a system memory 1304 including a random access memory (RAM) 1302 and a read-only memory (ROM) 1303, and a system bus 1305 connecting the various components. The computer device 1300 may also include an input / output system 1306 and a mass storage device 1307 for storing an operating system 1313, application programs 1314, and other program modules 1315. The input / output system includes an input / output controller 1306 mainly composed of a display 1308 and input devices 1309.

[0256] The mass storage device 1307 is connected to the central processing unit 1301 through a mass storage controller (not shown) connected to the system bus 1305. The mass storage device 1307 or computer-readable medium provides non-volatile storage for the computer device. The mass storage device 1307 may include a computer-readable medium (not shown) such as a hard disk or a Compact Disc Read-Only Memory (CD-ROM) drive.

[0257] Without loss of generality, computer-readable media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, flash memory or other solid-state storage technologies, CD-ROM, or other optical storage, magnetic cassettes, magnetic tape, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media is not limited to the aforementioned types. The above-mentioned system memory and mass storage devices may be collectively referred to as memory.

[0258] The computer device 1300 can be connected to a network 1312 via a network interface unit 1311 connected to the system bus 1305 .

[0259] The system memory 1304 or the mass storage device 1307 is further configured to store one or more instructions. The CPU 1301 implements all or part of the steps of the method in some embodiments of the present disclosure by executing the one or more instructions, for example, Figure 4Some or all of the steps in method 400 disclosed in.

[0260] In some embodiments of the present disclosure, the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, and the at least one instruction is executed by a processor to enable a computer to perform some or all steps of the method of some embodiments of the present disclosure, for example, Figure 4 Examples of computer-readable storage media include computer program (instruction) storage, such as read-only memory (ROM), random access memory (RAM), compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device.

[0261] Figure 14 A schematic diagram of a surgical robot system 1400 according to some embodiments of the present disclosure is shown. Figure 14 The surgical robot system 1400 may include a surgical instrument 1410 and a processor 1430. The surgical instrument 1410 includes an arm 1411 and a surgical actuator 1415 disposed at the distal end of the arm 1411. The processor 1430 is configured to execute some or all of the steps in the method of some embodiments of the present disclosure, for example, Figure 1 Some or all of the steps in the method 100 disclosed in, or Figure 9 Some or all of the steps in method 900 disclosed in.

[0262] Note that the above are only exemplary embodiments of the present disclosure and the technical principles used. It should be understood by those skilled in the art that the present disclosure is not limited to the specific embodiments herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present disclosure. Therefore, although the present disclosure has been described in more detail through the above embodiments, the present disclosure is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present disclosure, and the scope of the present disclosure is determined by the scope of the appended claims.

Claims

1. A method for determining the external force of a deformable robotic arm, characterized in that: include: Get the current actual position of the target part of the arm; Obtaining the current actual driving amount of the arm; Determining a first driving amount residual of the arm in a free motion state based on the current actual driving amount and the driving amount residual calculation model; determining a current ideal position of the target portion of the arm based on the current actual driving amount and the first driving amount residual; determining an actual position deformation amount of the target portion based on the current actual position and the current ideal position; as well as determining the force on the target part based on the actual position deformation; In which, the arm body includes: at least one segment, the segment includes multiple structural bones, and the multiple structural bones are used to cause the segment to deform; the target part includes the end of the arm body; the current actual position includes the current actual position of the end of the arm body; the current ideal position includes the current ideal position of the end of the arm body; the current actual driving amount includes the current actual driving amount of the multiple structural bones; the first driving amount residual includes the first driving amount residual of the multiple structural bones when the arm body is in a free motion state.

2. The method according to claim 1, wherein Also includes: Determining an ideal driving amount corresponding to the arm body in a current free motion state based on the current actual driving amount and the first driving amount residual; as well as Based on the ideal driving amount corresponding to the current free motion state of the arm, the current ideal position of the target part of the arm is determined.

3. The method according to claim 2, characterized in that include: Based on the ideal driving amount corresponding to the current free motion state of the arm and the kinematic model of the arm, the current ideal position of the target part of the arm is determined.

4. The method according to claim 1, wherein Also includes: A response operation is performed based on the force applied to the target portion.

5. The method according to claim 4, characterized in that The response operation includes: feeding back the force on the target part and / or indicating information about the force on the target part.

6. The method according to claim 1, characterized in that Also includes: In response to the actual position deformation amount of the target portion exceeding a position deformation threshold, a force applied to the target portion is determined based on the actual position deformation amount.

7. The method according to claim 1, wherein Also includes: Determining the motion state of the arm, wherein the motion state includes a free motion state and a constrained state; as well as In response to the arm being in the constrained state, the force on the target part is determined based on the actual position deformation.

8. The method according to claim 7, characterized in that Also includes: determining a current theoretical drive amount of the arm; Determining a second driving amount residual of the arm body based on the current actual driving amount and the current theoretical driving amount; as well as determining the motion state of the arm based on the first driving amount residual and the second driving amount residual; The current theoretical driving amount includes the driving amount of the structural bones of the arm body in theory based on the current shape of the arm body.

9. The method according to claim 8, characterized in that determining that the arm is in the constrained state based on a difference between the first driving amount residual and the second driving amount residual being higher than a threshold; Based on the fact that the difference between the first driving amount residual and the second driving amount residual is not higher than a threshold, it is determined that the arm is in the free motion state.

10. The method according to claim 1, wherein Also includes: obtaining a current stiffness of the target portion; as well as The force on the target part is determined based on the actual position deformation amount and the current stiffness of the target part.

11. The method according to claim 10, wherein Also includes: Obtaining a current stiffness of the target part corresponding to a direction of the actual position deformation; as well as The force on the target part is determined based on the actual position deformation and the current stiffness in the direction of the actual position deformation.

12. The method according to any one of claims 1 to 11, characterized in that The segment further includes a fixing plate and at least one spacer plate. The plurality of structural bones pass through the at least one spacer plate and ends thereof are fixedly connected to the fixing plate.

13. The method according to claim 2, wherein The ideal driving amount corresponding to the current free motion state of the arm body includes the ideal joint driving amount corresponding to the current free motion state of the arm body, and the method further includes: Determining ideal structural bone driving amounts of the plurality of structural bones corresponding to the current free motion state of the arm based on the current actual driving amount and the first driving amount residual; and Based on the ideal structural bone drive amount, the overall feed length of the arm and the overall rotation angle of the arm, the ideal joint drive amount corresponding to the current free motion state of the arm is determined.

14. The method according to claim 1, wherein The method further comprises: Obtaining an average end stiffness of an end of the arm body; Determining an end virtual force of the end of the arm body based on the current actual position of the end, the current ideal position of the end, and the average stiffness of the end; and Based on the current actual driving amount of the arm body, the end virtual force and the mechanical model of the arm body, the end virtual position deformation of the end of the arm body is determined, wherein the mechanical model is determined based on the distribution of the multiple structural bones of the arm body on the cross-section of the arm body and the physical properties of the multiple structural bones.

15. The method according to claim 14, wherein Also includes: Determining a current stiffness of the end of the arm based on the end virtual force and the end virtual position deformation; as well as The force on the end of the arm is determined based on the current actual position of the end, the current ideal position of the end, and the current stiffness of the end.

16. The method according to claim 15, characterized in that The mechanical model includes a constitutive relationship and a mechanical equilibrium relationship related to the multiple structural bones.

17. The method according to claim 16, characterized in that The constitutive relationship includes an internal force constitutive relationship and an internal moment constitutive relationship. The internal force constitutive relationship is based on the shear-tension stiffness matrix of the multiple structural bones, and the internal moment constitutive relationship is based on the bending-torsion stiffness matrix of the multiple structural bones.

18. The method according to claim 16, characterized in that The mechanical balance relationship includes the force balance relationship and the moment balance relationship of the arm body. The force balance relationship includes the axial force of the multiple structural bones, and the moment balance relationship includes the axial moment of the multiple structural bones.

19. A surgical robot system, characterized in that: include: A surgical instrument, comprising an arm body and a surgical implement disposed at a distal end of the arm body; as well as A processor, configured to execute the method according to any one of claims 1 to 18.

20. A computer device, characterized in that: The computer device comprises: a memory for storing at least one instruction; and A processor is coupled to the memory and configured to execute the at least one instruction to perform the method according to any one of claims 1 to 18.

21. A computer-readable storage medium for storing at least one instruction, characterized in that: When the at least one instruction is executed by a computer, the robot system is caused to implement the method according to any one of claims 1 to 18.

Images