Method and device for determining a pose of an introducer, surgical robot, electronic device

Abstract

The application discloses a method and device for determining the pose of a guide, a surgical robot and an electronic device. The method comprises the following steps: obtaining a target path, a preset distance range and a minimum moving distance; the target path is a path of a target object provided in the guide moving from a skin area of a target membrane to a target position in a target membrane body; the preset distance range is a distance range of the guide to the skin area; and the minimum moving distance is a minimum distance of single movement of the target object. Based on the length of the target path, the preset distance range and the minimum moving distance, a first distance is obtained, the first distance is the distance of the guide to the skin area, and the sum of the first distance and the length of the target path is an integer multiple of the minimum moving distance. The pose of an optical tracking device is obtained. Based on the target path, the first distance and the pose of the optical tracking device, a first pose is obtained. The first pose of the guide is determined based on the method, and the accuracy of the first pose can be improved.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and in particular to a method and apparatus for determining the position of a guide, a surgical robot, and electronic equipment. Background Technology

[0002] In the field of medical devices, it is necessary to move a target object to a target location within the body membrane. The target object is placed on a guide device, and by driving the guide device to move and adjust its posture, the posture of the target object can be changed simultaneously, thereby controlling the movement path of the target object to reach the preset target location within the body membrane. Current technology has low accuracy in determining the posture of the guide device, resulting in poor movement of the target object. Summary of the Invention

[0003] This application provides a method and apparatus for determining the pose of a guide, a surgical robot, and an electronic device to improve the accuracy of the guide's pose.

[0004] In a first aspect, a method for determining the pose of a guide is provided, the method comprising: The system acquires the target path, a preset distance range, and a minimum movement distance. The target path is the path taken by the target object, which is set on the guide, to move from the skin area of ​​the target membrane to the target position within the target membrane. The preset distance range is the range of distances from the guide to the skin area. The minimum movement distance is the minimum distance the target object moves in a single movement. Based on the length of the target path, the preset distance range, and the minimum movement distance, a first distance is obtained. The first distance is the distance from the guide to the skin area. The first distance is within the distance range, and the sum of the first distance and the length of the target path is an integer multiple of the minimum movement distance. The pose of an optical tracking device is acquired, wherein the optical tracking device is used to determine the pose of the guide by locating an optical marker disposed on the guide; Based on the target path, the first distance, and the pose of the optical tracking device, a first pose is obtained; when the pose of the guide is the first pose, the movement path of the target object is aligned with the target path, and the angle between the target normal vector and the optical axis of the optical tracking device is less than a preset angle, wherein the target normal vector is the normal vector of the plane formed by at least three optical markers set on the guide.

[0005] In the first aspect, the determining device first determines the first distance from the guide to the skin region of the target membrane based on the length of the target path, a preset distance range, and the minimum moving distance. This ensures that when the target object is positioned on the guide, the distance from the target object to the target position within the target membrane is an integer multiple of the minimum moving distance, thus facilitating control of the target object's movement to the target position. Then, based on the target path, the first distance, and the pose of the optical tracking device, the guide's first pose is obtained, which satisfies the following conditions: when the guide's pose is the first pose, the target object's movement path is aligned with the target path, and the angle between the target normal vector and the optical axis of the optical tracking device is less than a preset angle. This improves the accuracy of the first pose.

[0006] In any embodiment of this application, the optical markers disposed on the guide include n marker groups, wherein each marker group includes at least three optical markers, and the planes formed by the optical markers in different marker groups are not parallel, and n is an integer greater than 1; The process of obtaining the first pose based on the target path, the first distance, and the pose of the optical tracking device includes: Based on the target path and the first distance, the second pose of the guide is obtained; when the pose of the guide is the second pose, the movement path of the target object is aligned with the target path; Based on the second pose, n reference vectors are obtained. The reference vector is the normal vector of the plane formed by the optical markers in the n marker sets when the pose of the guide is the second pose. The reference vector corresponds one-to-one with the marker set. Determine the angle between the optical axis and the n reference vectors to obtain n reference angles, and the reference angles correspond one-to-one with the reference vectors; If none of the n reference angles is smaller than the preset angle, the difference between the minimum value of the n reference angles and the preset angle is determined to obtain the reference angle difference. Based on the reference angle difference, the second pose is corrected to obtain the first pose. The first pose is obtained by rotating the second pose around the target path by a target angle, and the target angle is obtained based on the reference angle difference.

[0007] In any embodiment of this application, the guide is provided with a first torque sensor, and the method further includes: A first control command is obtained, the first control command comes from a teleoperation device, and the first control command instructs the guide to adjust its pose to the first pose by movement; In response to the first control command, the guide is controlled to move, and a first torque signal is acquired, wherein the first torque signal is a signal collected by the first torque sensor during the movement of the guide; A first torque sub-signal and a second torque sub-signal are determined from the first torque signal, wherein the frequency of the first torque sub-signal is less than the frequency of the second torque signal; Based on the first torque sub-signal, a first feedback sub-signal is obtained. The frequency of the first feedback sub-signal is the same as the frequency of the first torque sub-signal. The energy of the first feedback sub-signal is obtained by multiplying the energy of the first torque sub-signal with a first energy gain, where the first energy gain is a positive number. Based on the second torque sub-signal, a second feedback sub-signal is obtained. The frequency of the second feedback sub-signal is the same as the frequency of the second torque sub-signal. The energy of the second feedback sub-signal is obtained based on the product of the energy of the second torque sub-signal and the second energy gain. The first energy gain is less than the second energy gain. Based on the first feedback sub-signal and the second feedback sub-signal, the remote operation device is controlled to output the first feedback information.

[0008] In conjunction with any embodiment of this application, before controlling the teleoperation device to output first feedback information based on the first feedback sub-signal and the second feedback sub-signal, the method further includes: Based on the first control command, the first pose change amount is obtained, which is the pose change amount that the guide should achieve as indicated by the first control command. Obtain the second pose change amount, which is the actual pose change amount achieved by the guide based on the first control command; Determine the difference between the first pose change and the second pose change to obtain the pose change difference; The step of controlling the teleoperation device to output first feedback information based on the first feedback sub-signal and the second feedback sub-signal includes: Based on the first feedback sub-signal and the second feedback sub-signal, the first feedback signal is obtained; Based on the pose change difference, a third energy gain of the energy of the first feedback signal is obtained, and the pose change difference is positively correlated with the third energy gain. Based on the third energy gain, the energy of the first feedback signal is adjusted to obtain a second feedback signal. The frequency of the first feedback signal is the same as the frequency of the second feedback signal, and the energy of the second feedback signal is obtained based on the product of the third energy gain and the energy of the first feedback signal. Based on the second feedback signal, the remote operation device is controlled to output the first feedback information.

[0009] In conjunction with any embodiment of this application, before obtaining the third energy gain of the energy of the first feedback signal based on the pose change difference, the method further includes: A reference feedback signal is acquired, which is collected by the first torque sensor when the guide does not collide with an obstacle and the guide does not vibrate. The difference between the energy of the reference feedback signal and the energy of the first feedback signal is determined to obtain the target energy difference; The third energy gain, obtained based on the pose change difference, of the energy of the first feedback signal includes: When the target energy difference is greater than the third threshold, the third energy gain is obtained based on the pose change difference.

[0010] In any embodiment of this application, obtaining the second feedback sub-signal based on the second torque sub-signal includes: The second energy gain is obtained based on the frequency of the second torque sub-signal; when the frequency of the second torque sub-signal is less than the first threshold, the second energy gain is less than 1; when the frequency of the second torque sub-signal is greater than or equal to the first threshold, the second energy gain is greater than or equal to 1. Based on the second energy gain, the energy of the second torque sub-signal is adjusted to obtain the second feedback sub-signal.

[0011] In any embodiment of this application, the target object is provided with a second torque sensor, the guide is provided with a driving device, the driving device is used to drive the target object to move, and the method further includes: Obtain a second control command, the second control command being from the teleoperation device, the second control command instructing the drive device to drive the target object to move based on the target path; In response to the second control command, the guide is controlled to move and a second torque signal is acquired. The second torque signal is a signal collected by the second torque sensor during the movement of the target object within the target membrane. A third torque sub-signal is determined from the second torque signal, wherein the frequency of the third torque sub-signal is the same as the frequency of the second torque signal; Based on the third torque sub-signal, a third feedback sub-signal is obtained. The frequency of the third feedback sub-signal is the same as that of the third torque sub-signal. The energy of the third feedback sub-signal is obtained by multiplying the energy of the third torque sub-signal with the third energy gain. The third energy gain is greater than the second energy gain. Based on the third feedback sub-signal, the remote operation device is controlled to output the second feedback information.

[0012] Secondly, an apparatus for determining the pose of a guide is provided, the apparatus comprising: The acquisition unit is used to acquire the target path, the preset distance range, and the minimum movement distance; the target path is the path by which the target object set on the guide moves from the skin area of ​​the target body membrane to the target position inside the target body membrane; the preset distance range is the range of distances from the guide to the skin area; and the minimum movement distance is the minimum distance the target object moves in a single movement. The processing unit is configured to obtain a first distance based on the length of the target path, the preset distance range, and the minimum movement distance. The first distance is the distance from the guide to the skin region. The first distance is within the distance range, and the sum of the first distance and the length of the target path is an integer multiple of the minimum movement distance. The acquisition unit is further configured to acquire the pose of the optical tracking device, wherein the optical tracking device is configured to determine the pose of the guide by locating the optical markers disposed on the guide; The processing unit is further configured to obtain a first pose based on the target path, the first distance, and the pose of the optical tracking device; when the pose of the guide is the first pose, the movement path of the target object is aligned with the target path, and the angle between the target normal vector and the optical axis of the optical tracking device is less than a preset angle, wherein the target normal vector is the normal vector of the plane formed by at least three optical markers set on the guide.

[0013] In any embodiment of this application, the optical markers disposed on the guide include n marker groups, wherein each marker group includes at least three optical markers, and the planes formed by the optical markers in different marker groups are not parallel, and n is an integer greater than 1; The process of obtaining the first pose based on the target path, the first distance, and the pose of the optical tracking device includes: Based on the target path and the first distance, the second pose of the guide is obtained; when the pose of the guide is the second pose, the movement path of the target object is aligned with the target path; Based on the second pose, n reference vectors are obtained. The reference vector is the normal vector of the plane formed by the optical markers in the n marker sets when the pose of the guide is the second pose. The reference vector corresponds one-to-one with the marker set. Determine the angle between the optical axis and the n reference vectors to obtain n reference angles, and the reference angles correspond one-to-one with the reference vectors; If none of the n reference angles is smaller than the preset angle, the difference between the minimum value of the n reference angles and the preset angle is determined to obtain the reference angle difference. Based on the reference angle difference, the second pose is corrected to obtain the first pose. The first pose is obtained by rotating the second pose around the target path by a target angle, and the target angle is obtained based on the reference angle difference.

[0014] In any embodiment of this application, the guide is provided with a first torque sensor, and the acquisition unit is further configured to acquire a first control command, the first control command being from a remote operating device, the first control command instructing the guide to adjust its pose to the first pose through movement; The processing unit is further configured to respond to the first control command, control the movement of the guide, and acquire a first torque signal, wherein the first torque signal is a signal collected by the first torque sensor during the movement of the guide; The processing unit is further configured to determine a first torque sub-signal and a second torque sub-signal from the first torque signal, wherein the frequency of the first torque sub-signal is less than the frequency of the second torque signal; The processing unit is further configured to obtain a first feedback sub-signal based on the first torque sub-signal, wherein the frequency of the first feedback sub-signal is the same as the frequency of the first torque sub-signal, and the energy of the first feedback sub-signal is obtained based on the product of the energy of the first torque sub-signal and a first energy gain, wherein the first energy gain is a positive number. The processing unit is further configured to obtain a second feedback sub-signal based on the second torque sub-signal, wherein the frequency of the second feedback sub-signal is the same as the frequency of the second torque sub-signal, and the energy of the second feedback sub-signal is obtained based on the product of the energy of the second torque sub-signal and the second energy gain, wherein the first energy gain is less than the second energy gain. The processing unit is further configured to control the teleoperation device to output first feedback information based on the first feedback sub-signal and the second feedback sub-signal.

[0015] In any embodiment of this application, the processing unit is further configured to obtain a first pose change amount based on the first control instruction, wherein the first pose change amount is the pose change amount that the guide should achieve as indicated by the first control instruction. The acquisition unit is further configured to acquire a second pose change, wherein the second pose change is the actual pose change achieved by the guide based on the first control command. The processing unit is also used to determine the difference between the first pose change and the second pose change to obtain the pose change difference. The processing unit is further configured to obtain a first feedback signal based on the first feedback sub-signal and the second feedback sub-signal; The processing unit is further configured to obtain a third energy gain of the energy of the first feedback signal based on the pose change difference, wherein the pose change difference is positively correlated with the third energy gain. The processing unit is further configured to adjust the energy of the first feedback signal based on the third energy gain to obtain a second feedback signal, wherein the frequency of the first feedback signal is the same as the frequency of the second feedback signal, and the energy of the second feedback signal is obtained based on the product of the third energy gain and the energy of the first feedback signal; The processing unit is further configured to control the teleoperation device to output the first feedback information based on the second feedback signal.

[0016] In conjunction with any embodiment of this application, the acquisition unit is further configured to acquire a reference feedback signal, which is collected by the first torque sensor when the guide does not collide with an obstacle and the guide does not vibrate; The processing unit is further configured to determine the difference between the energy of the reference feedback signal and the energy of the first feedback signal, and obtain the target energy difference; The processing unit is further configured to obtain the third energy gain based on the pose change difference when the target energy difference is greater than the third threshold.

[0017] In any embodiment of this application, the processing unit is further configured to obtain the second energy gain based on the frequency of the second torque sub-signal; when the frequency of the second torque sub-signal is less than a first threshold, the second energy gain is less than 1; when the frequency of the second torque sub-signal is greater than or equal to the first threshold, the second energy gain is greater than or equal to 1. The processing unit is further configured to adjust the energy of the second torque sub-signal based on the second energy gain to obtain the second feedback sub-signal.

[0018] In any embodiment of this application, the target object is provided with a second torque sensor, the guide is provided with a driving device, the driving device is used to drive the target object to move, and the acquisition unit is further used to acquire a second control command, the second control command comes from the teleoperation device, and the second control command instructs the driving device to drive the target object to move based on the target path; The processing unit is further configured to respond to the second control command, control the movement of the guide, and acquire a second torque signal, wherein the second torque signal is a signal collected by the second torque sensor during the movement of the target object within the target membrane; The processing unit is further configured to determine a third torque sub-signal from the second torque signal, wherein the frequency of the third torque sub-signal is the same as the frequency of the second torque sub-signal; The processing unit is further configured to obtain a third feedback sub-signal based on the third torque sub-signal, wherein the frequency of the third feedback sub-signal is the same as the frequency of the third torque sub-signal, and the energy of the third feedback sub-signal is obtained based on the product of the energy of the third torque sub-signal and a third energy gain, wherein the third energy gain is greater than the second energy gain. The processing unit is also used to control the teleoperation device to output second feedback information based on the third feedback sub-signal.

[0019] Thirdly, a surgical robot is provided, including a device for determining the pose of a guide as described in the second aspect. In this third aspect, the surgical robot can perform a method for determining the pose of a guide using the device for determining the pose of the guide, thereby improving the accuracy of the guide's pose.

[0020] Fourthly, an electronic device is provided, comprising: a processor and a memory, the memory for storing computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs a method as described in the first aspect above and any possible implementation thereof.

[0021] Fifthly, another electronic device is provided, comprising: a processor, a transmitting device, an input device, an output device, and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs a method as described in the first aspect above and any possible implementation thereof.

[0022] In a sixth aspect, a computer-readable storage medium is provided, wherein a computer program is stored therein, the computer program including program instructions that, when executed by a processor, cause the processor to perform a method as described in the first aspect above and any possible implementation thereof.

[0023] In a seventh aspect, a computer program product is provided, the computer program product comprising a computer program or instructions, wherein, when the computer program or instructions are executed on a computer, the computer performs the method described in the first aspect and any possible implementation thereof.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments of this application will be described below.

[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0027] Figure 1 A flowchart illustrating a method for determining the pose of a guide provided in an embodiment of this application; Figure 2 A schematic diagram of a device for determining the pose of a guide provided in an embodiment of this application; Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0029] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0030] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments. It should be understood that in this application, "at least one" means one or more, "more" means two or more, and "at least two" means two or three or more.

[0031] The execution subject of this application embodiment is a device for determining the pose of a guide (hereinafter referred to as the determining device), wherein the determining device can be any electronic device capable of executing the technical solution disclosed in the method embodiments of this application. Optionally, the determining device can be one of the following: a computer, a server.

[0032] It should be understood that the method embodiments of this application can also be implemented by a processor executing computer program code. The embodiments of this application are described below with reference to the accompanying drawings. Please refer to... Figure 1 , Figure 1 This is a flowchart illustrating a method for determining the pose of a guide provided in an embodiment of this application.

[0033] 101. The device determines the target path, preset distance range, and minimum travel distance.

[0034] In this embodiment, the target path is the path taken by the target object, which is positioned on the guide, from the skin region of the target membrane to a target location within the target membrane. Optionally, the guide includes a placement slot for placing the target object. When the target object is placed in the placement slot, the movement path of the target object coincides with the axis of the placement slot. For example, the target object is a needle.

[0035] In some schemes, the target path is planned based on CT images of the target membrane and ultrasound images of the target image.

[0036] The target phantom is a human body model, which is a physical model. Optionally, the material used for the target phantom includes at least one of the following: silicone, resin, gel, and polyurethane. For example, the target phantom is a gelatin phantom or an agar phantom. The target location is a location within the target membrane; optionally, the target location is the location of a lesion within the target membrane.

[0037] The preset distance range is the range of distances from the guide to the skin area. Optionally, the preset distance range is a safe distance range from the guide to the skin area; for example, the preset distance range is 1 cm to 2 cm. Optionally, when the target object is placed in the placement slot, the endpoint of the target object closest to the skin area coincides with the endpoint of the placement slot closest to the skin area. The distance from the guide to the skin area is the distance from the endpoint of the placement slot closest to the skin area to the skin area.

[0038] The minimum movement distance is the minimum distance the target object can move in a single movement; that is, the minimum distance is the movement step size of the target object. For example, if the target object is a needle with graduated lines and the distance between two adjacent graduated lines is 3 millimeters, the minimum movement distance can be 3 millimeters, which is beneficial for controlling the movement of the needle. As another example, if the target object can be moved by a control drive device, and the drive device has a driving accuracy of 4 millimeters, then the minimum movement distance is 4 millimeters.

[0039] 102. The device determines the first distance based on the length of the target path, the preset distance range, and the minimum moving distance.

[0040] In this embodiment, the first distance is the distance from the guide to the skin area. The first distance is within a certain range, and the sum of the first distance and the length of the target path is an integer multiple of the minimum movement distance. Because when the target object is positioned on the guide, the total movement distance from the start to the target position is the sum of the length of the target path and the distance from the guide to the skin area. Therefore, the fact that the sum of the first distance and the length of the target path is an integer multiple of the minimum movement distance is beneficial for controlling the movement of the target object. For example, if the minimum movement distance is 7 mm, the target path is 26 mm, and the preset distance range is 8 mm to 18 mm, then the first distance is 9 mm or 16 mm.

[0041] Optionally, if the number of distances that satisfy the following condition is greater than: the sum of the distances within the distance range and the length of the target path is an integer multiple of the minimum moving distance, the minimum distance that satisfies the condition is determined as the first distance.

[0042] 103. The device determines the pose of the optical tracking device, which is used to determine the pose of the guide by locating the optical markers set on the guide.

[0043] The optical markers set on the guide can be fixed to the guide. The optical markers have the ability to reflect light emitted by the optical tracking device. Optionally, the number of optical markers set on the guide is greater than or equal to three. In some embodiments, the optical markers set on the guide comprise n marker groups, wherein each marker group includes at least three optical markers. After the optical tracking device determines the position of an optical marker in any marker group, the position and / or attitude of the guide can be determined based on the position of the optical marker in the marker group.

[0044] The optical tracking device is equipped with a light source and an optical acquisition unit. The optical acquisition unit has a preset optical axis, which extends outward as the reference axis of the optical tracking device's coordinate system. For example, the light source is an infrared light source, and the optical acquisition unit is a camera for acquiring infrared images. The optical tracking device emits probe rays into space through the light source. A passive optical marker fixedly mounted on the guide receives and reflects these probe rays. The optical acquisition unit captures the light reflected back from the marker and, using the coordinate system established by its optical axis as a reference, calculates the three-dimensional position of the optical marker in that coordinate system using a triangulation algorithm based on the imaging information of the reflected light. Then, based on the relative positional relationship between the optical marker and the guide, the three-dimensional position and / or attitude of the guide can be determined.

[0045] When the normal vector of the plane formed by the optical markers forms different angles with the optical axis emitted by the optical tracking device, it affects the intensity of the reflected light returning to the optical acquisition unit, thus affecting the positioning accuracy of the optical tracking device, and consequently, the accuracy of the position determined by the optical tracking device. Specifically, if the angle is small, the normal vector points towards the tracking device, and the optical axis is approximately perpendicular to the incident marker plane, the reflected light can return to the optical acquisition unit efficiently, the signal-to-noise ratio of the reflected light is high, the error in the calculated position of the optical marker is smaller, and the accuracy is higher. As the angle increases, the inclination of the incident light increases, the reflected light deviates from the optical axis direction of the optical acquisition unit, the energy of the returned light attenuates, the signal-to-noise ratio of the reflected light decreases, the error in the calculated position of the optical marker increases, and the accuracy decreases accordingly.

[0046] During the movement of the guide and the target object, the pose of the optical tracking device remains unchanged. In step 103, the pose of the optical tracking device acquired by the device is determined to be the fixed pose of the optical tracking device.

[0047] 104. The device determines the first pose based on the target path, the first distance, and the pose of the optical tracking device.

[0048] As described in step 103, the angle between the normal vector of the plane formed by the optical markers and the optical axis of the optical tracking device affects the accuracy of the position determined by the optical tracking device. The pose of the guide affects the position of the optical markers placed on the guide, which in turn affects the direction of the normal vector of the plane formed by the optical markers, thus affecting the angle between the normal vector and the optical axis of the optical tracking device. Therefore, when determining the pose of the guide, the determining device needs to consider the angle between the normal vector of the plane formed by the optical markers and the optical axis of the optical tracking device. Furthermore, because the movement path of the target object is related to the pose of the guide when it is placed on the guide, the deviation between the target object's movement path and the target path also needs to be considered when determining the pose of the guide. In addition, the pose of the guide must also satisfy the condition that the distance from the guide to the skin region is a first distance.

[0049] Based on this, the first pose obtained by the device through step 104 satisfies the following pose condition: when the guide's pose is the first pose, the target object's movement path is aligned with the target path, and the angle between the target normal vector and the optical axis of the optical tracking device is less than a preset angle. Here, the target normal vector is the normal vector of the plane formed by at least three optical markers on the guide. In this condition, the angle between the target normal vector and the optical axis being less than the preset angle indicates a small angle, resulting in high accuracy in the position determined by the optical tracking device. Determining the guide's pose requires the positions of at least three optical markers; therefore, the target normal vector in the pose condition is the normal vector of the plane formed by at least three optical markers.

[0050] In one possible implementation, the determining device uses the above pose conditions as constraints, and based on the target path, the first distance, and the pose of the optical tracking device, it can obtain a first pose that satisfies the above pose conditions.

[0051] exist Figure 1 In this method, the determining device first determines the first distance from the guide to the skin region of the target membrane based on the length of the target path, a preset distance range, and the minimum moving distance. This ensures that when the target object is positioned on the guide, the distance from the target object to the target position within the target membrane is an integer multiple of the minimum moving distance, thus facilitating control of the target object's movement to the target position. Then, based on the target path, the first distance, and the pose of the optical tracking device, the guide's first pose is obtained. This first pose satisfies the following conditions: when the guide's pose is the first pose, the target object's movement path is aligned with the target path, and the angle between the target normal vector and the optical axis of the optical tracking device is less than a preset angle. This improves the accuracy of the first pose.

[0052] The higher the accuracy of the guide's pose, the smaller the deviation between the target object's movement path and the target path, and the higher the accuracy of the position determined by the optical tracking device.

[0053] In one optional implementation, the optical markers disposed on the guide include n marker groups, wherein each marker group includes at least three optical markers, and the planes formed by the optical markers in different marker groups are not parallel, where n is an integer greater than 1. That is, the position and / or attitude of the guide can be determined based on the position of the optical markers in any one of the n marker groups. Moreover, the directions of the normal vectors of the planes formed by the optical markers in different marker groups are different.

[0054] The determining device performs the following steps during the execution of step "obtaining the first pose based on the target path, the first distance, and the pose of the optical tracking device": Based on the target path and the first distance, the guide's second pose is obtained. With the guide in the second pose, the target object's movement path is aligned with the target path. Based on the second pose, n reference vectors are obtained, where each reference vector is the normal vector of the plane formed by the optical markers in the n marker sets when the guide's pose is the second pose, and each reference vector corresponds one-to-one with a marker set. The angle between the optical axis and the n reference vectors is determined, resulting in n reference angles, each corresponding one-to-one with a reference vector. If none of the n reference angles is smaller than a preset angle, the difference between the minimum value of the n reference angles and the preset angle is determined, resulting in a reference angle difference. Based on the reference angle difference, the second pose is corrected to obtain the first pose, which is obtained by rotating the second pose around the target path by a target angle, and the target angle is obtained based on the reference angle difference.

[0055] In this embodiment, the determining device first aligns the movement path of the target object with the target path based on the target path and a first distance, obtaining the second pose of the guide. Then, it determines the normal vector of the plane formed by the optical markers in the n marker groups when the guide's pose is the second pose, obtaining n reference vectors. Next, it determines the angle between the optical axis of the optical tracking device and each of the n reference vectors, obtaining n reference angles. Then, it determines whether any of the n reference angles is smaller than a preset angle. If the determination result indicates that an angle smaller than the preset angle exists, the second pose can be directly used as the first pose. If the determination result indicates that no angle smaller than the preset angle exists, it means that the second pose cannot be used as the first pose; therefore, the difference between the minimum value of the n reference angles and the preset angle is further determined to obtain the reference angle difference. Then, based on the reference angle difference, the second pose is corrected to obtain the first pose that meets the pose conditions, thereby improving the accuracy of the first pose. The first pose is obtained by rotating the second pose around the target path by a target angle. The target angle is obtained based on the reference angle difference. In other words, because the target object's movement path is aligned with the target path when the guide's pose is the second pose, when correcting the second pose, it can be rotated around the target path so that among the n reference angles, there is an angle smaller than a preset angle. Therefore, the angle rotated around the target path (i.e., the target angle) is obtained based on the reference angle difference.

[0056] In this embodiment, because the guide has n sets of markers, the angle between the optical axis of the optical tracking device and the normal vector of the plane formed by the optical markers in any set is less than a preset angle, ensuring the accuracy of the position and / or attitude of the guide determined by the optical tracking device. This improves the difficulty of determining the guide's pose. Furthermore, by determining the second pose first and then the first pose, the difficulty of obtaining the first pose is reduced, and the efficiency of obtaining the first pose is improved.

[0057] Optionally, if none of the rotation angles determined based on the reference angle difference can make an angle smaller than the preset angle among the n reference angles, then the determining device can determine the rotation angle of the guide based on the second smallest angle among the n reference angles, until the target angle is determined. When the rotation angle of the guide around the target path is the target angle, an angle smaller than the preset angle can be made among the n reference angles.

[0058] Optionally, if after traversing n reference angles, it is impossible to obtain a rotation angle that makes one of the n reference angles smaller than a preset angle, then the pose of the optical marker set on the guide can be adjusted to obtain the target angle.

[0059] Candidate rotation angle, where the candidate rotation angle is the angle by which the guide rotates around the target path with the second pose as the starting pose.

[0060] In some schemes, the guide is connected to the robotic arm, and the guide moves by controlling the movement of the robotic arm. The determination device performs the following steps during the execution of the step "correcting the second pose based on the reference angle difference to obtain the first pose": Correcting the second pose based on the reference angle difference to obtain a candidate pose, wherein, when the guide's pose is a candidate pose, the target object's movement path is aligned with the target path, and one of the n reference angles is smaller than a preset angle. Based on the guide's current pose and the candidate pose, the process of the guide moving from the current pose to the candidate pose is simulated to obtain a first simulation result. The first simulation result indicates whether the guide collides with an obstacle, whether the robotic arm collides with an obstacle, and the rotation angle of the robotic arm's joints. If the first simulation result meets reasonable conditions, the candidate pose is determined as the first pose. Reasonable conditions include: neither the guide nor the robotic arm collides with an obstacle, and the rotation angle of the robotic arm's joints is within the rotation range of the robotic arm's joints. Optionally, the rotation angle of the robotic arm's joints is obtained based on the current pose, candidate poses, and inverse kinematics.

[0061] If the first simulation result does not meet the reasonable conditions, a set of search poses is obtained based on candidate poses. This set includes multiple search poses, each obtained by rotating the guide around the target path from a candidate pose as the starting pose. Optionally, the search pose can be obtained based on a preset rotation angle and candidate poses. For example, the preset rotation angle is an integer multiple of 5 degrees. Based on the search poses in the set, the process of the guide moving from the current pose to the search pose is simulated, resulting in a set of simulation results. This set includes multiple second simulation results, each corresponding one-to-one with a search pose. The second simulation results indicate whether the guide collides with an obstacle, whether the robotic arm collides with an obstacle, and the rotation angle of the robotic arm's joints. Based on the multiple second simulation results, the search pose that meets the reasonable conditions is determined as the first pose from the search set.

[0062] In one optional implementation, after obtaining a first simulation result or a second simulation result, the determining device can output the first simulation result or the second simulation result so that the user can be aware of the simulation result. Optionally, the determining device generates first 3D preview information, such as a 3D animation, based on the first simulation result and displays the first 3D preview information. Optionally, the determining device generates second 3D preview information based on the second simulation result and displays the second 3D preview information.

[0063] As an optional implementation, the guide is provided with a first torque sensor. The first torque sensor can be used to detect the torque experienced by the guide during movement. Optionally, the first torque sensor is located at the end of the guide. The determining device further performs the following steps: acquiring a first control command, wherein the first control command comes from a teleoperation device, and the first control command instructs the guide to adjust its pose to a first pose by movement. In response to the first control command, controlling the movement of the guide and acquiring a first torque signal, wherein the first torque signal is a signal collected by the first torque sensor during the movement of the guide. Determining a first torque sub-signal and a second torque sub-signal from the first torque signal, wherein the frequency of the first torque sub-signal is less than the frequency of the second torque sub-signal. Based on the first torque sub-signal, obtaining a first feedback sub-signal, wherein the frequency of the first feedback sub-signal is the same as the frequency of the first torque sub-signal, and the energy of the first feedback sub-signal is obtained based on the product of the energy of the first torque sub-signal and a first energy gain, wherein the first energy gain is a positive number. Based on the second torque sub-signal, a second feedback sub-signal is obtained, wherein the frequency of the second feedback sub-signal is the same as the frequency of the second torque sub-signal, and the energy of the second feedback sub-signal is obtained by multiplying the energy of the second torque sub-signal by a second energy gain, wherein the first energy gain is less than the second energy gain. Based on the first and second feedback sub-signals, the remote control device is controlled to output first feedback information.

[0064] In this embodiment, the teleoperation device is used for remote control; optionally, the teleoperation device performs remote control based on teleoperation technology. For example, the teleoperation device remotely controls a robotic arm to move a guide based on teleoperation technology. In one possible scenario, the teleoperation device is deployed outside the operating room, while the robotic arm, guide, and target membrane are all deployed inside the operating room. The user outside the operating room can remotely control the robotic arm inside the operating room to move the guide via the teleoperation device. For example, the user outside the operating room controls the robotic arm to move the guide via the teleoperation device so that the guide's pose is the first position.

[0065] Control commands are generated based on user operations on the teleoperated device. In some implementations, the teleoperated device includes a joystick, which controls the movement of a guide by shaking the joystick; for example, controlling a robotic arm to move the guide. Operations on the teleoperated device include shaking the joystick.

[0066] A communication connection exists between the determining device and the teleoperating device, and data can be transmitted between them based on this communication connection. In the above embodiments, the determining device receives a first control command from the teleoperating device based on this communication connection. In some embodiments, the teleoperating device is located outside the operating room, and the determining device is located inside the operating room. The determining device is used to control the movement of the guide; optionally, the determining device controls a robotic arm to drive the movement of the guide.

[0067] Since the first control command instructs the guide to adjust its pose to the first pose through movement, the determining device responds to the first control command and controls the movement of the guide. Optionally, the first control command instructs the guide's pose adjustment amount. The pose determined based on the pose adjustment amount indicated by the first control command and the guide's current pose is the first pose.

[0068] During the movement of the guide, a first torque sensor collects a first torque signal, which indicates the torque experienced by the guide during movement. In some embodiments, the torque experienced by the guide includes at least one of the following: torque generated by the guide's gravity, torque generated by the guide's amplitude of motion, torque generated by the guide's velocity, torque generated by the guide's acceleration, torque generated by the guide's angular velocity, and torque generated by the guide's collision with an obstacle. That is, the factors generating the torque include static factors and dynamic information. Static factors include the guide's gravity, and dynamic factors include the guide's motion state information, such as amplitude of motion, velocity of motion, and acceleration of motion.

[0069] The first torque sub-signal and the second torque sub-signal are two sub-signals with different frequencies within the first torque signal. In some schemes, the determining device is based on digital filtering processing to determine the first and second torque sub-signals from the first torque signal. Optionally, a low-pass filter or a high-pass filter can be used to divide the first torque signal into the first and second torque sub-signals. In other schemes, the determining device performs a Fast Fourier Transform on the first torque signal to obtain spectral information. Based on the spectral information, the frequency components corresponding to the sub-signals of different frequencies can be determined. An Inverse Fourier Transform is performed on the frequency components corresponding to each sub-signal to restore the sub-signals of different frequencies to time-domain signals, resulting in at least two torque sub-signals. The first and second torque sub-signals are obtained based on the at least two torque sub-signals.

[0070] By outputting feedback information through the remote control device, users can perceive changes in the torque experienced by the guide during movement, allowing them to adjust their operations in a timely manner, thereby improving the accuracy and safety of remote control. For example, if the guide's movement amplitude is too large, the resistance to the user's operation of the remote control device can be increased to avoid the risks associated with excessive guide movement amplitude. This can be achieved by increasing the resistance or damping of the joystick. A higher frequency of the torque sub-signal indicates a faster change in the torque sub-signal, such as a shorter amplitude change period. The faster the change in the torque sub-signal, the more sensitive the change should be to the user; that is, the stronger the feedback intensity should be.

[0071] Therefore, when the frequency of the first torque sub-signal is lower than the frequency of the second torque sub-signal, the first energy gain is lower than the second energy gain. The first energy gain is positively correlated with the feedback strength of the first torque sub-signal, and the second energy gain is positively correlated with the feedback strength of the second torque sub-signal. Specifically, the first feedback sub-signal is used to feed back the first torque sub-signal, and the second feedback sub-signal is used to feed back the second torque sub-signal. The energy of the first feedback sub-signal is obtained by multiplying the energy of the first torque sub-signal by the first energy gain, and the energy of the second feedback sub-signal is obtained by multiplying the energy of the second torque sub-signal by the second energy gain.

[0072] Finally, based on the first and second feedback sub-signals, the remote control device outputs the first feedback information, which can increase the feedback strength of the torque sub-signal with a high frequency and rapid change. For example, the first torque sub-signal is generated by the gravity of the guide, and the second torque sub-signal is generated by the instantaneous acceleration of the guide. Since the gravity of the guide is constant, while the sudden acceleration of the guide is a sudden change, the feedback strength for the instantaneous acceleration of the guide should be greater.

[0073] In some schemes, the determining device obtains a first feedback signal based on a first feedback sub-signal and a second feedback sub-signal. A feedback command is sent to the teleoperated device, wherein the feedback command instructs the teleoperated device to output first feedback information based on the first feedback signal. The first feedback information is used to provide feedback on the torque corresponding to the first torque signal, i.e., the torque experienced by the guide. Optionally, outputting the first feedback information includes generating a force that impedes movement of the teleoperated device; for example, the feedback command instructs the damping device of the teleoperated device to generate a force to impede remotely controlled movement based on the first feedback signal. Alternatively, outputting the first feedback information includes outputting optical information for feedback.

[0074] In this embodiment, after acquiring the first torque signal, the determining device identifies a first torque sub-signal and a second torque sub-signal with different frequencies from the first torque signal. Then, based on the first torque sub-signal, a first feedback sub-signal is obtained for feeding back the first torque sub-signal, and based on the second torque sub-signal, a second feedback sub-signal is obtained for feeding back the second torque sub-signal. Finally, based on the first and second feedback sub-signals, the remote control device outputs first feedback information, which allows for a stronger feedback intensity for torque sub-signals with higher frequencies and faster changes, and a weaker feedback intensity for torque sub-signals with lower frequencies and slower changes. This allows the user to more clearly perceive the changes in torque experienced by the guide, thereby improving the feedback effect and enhancing the safety of remotely controlling the guide's movement using the remote control device.

[0075] In one optional implementation, before executing the step "controlling the teleoperation device to output first feedback information based on the first feedback sub-signal and the second feedback sub-signal," the teleoperation device further performs the following steps: Based on a first control command, obtain a first pose change, wherein the first pose change is the pose change that the guide should achieve as instructed by the first control command. Obtain a second pose change, wherein the second pose change is the actual pose change achieved by the guide based on the first control command. Determine the difference between the first pose change and the second pose change to obtain the pose change difference. After obtaining the pose change difference, during the execution of the step "controlling the teleoperation device to output first feedback information based on the first feedback sub-signal and the second feedback sub-signal," perform the following steps: Based on the first feedback sub-signal and the second feedback sub-signal, obtain a first feedback signal. Based on the pose change difference, obtain a third energy gain of the energy of the first feedback signal, wherein the pose change difference is positively correlated with the third energy gain. Based on the third energy gain, the energy of the first feedback signal is adjusted to obtain a second feedback signal. The frequency of the first feedback signal is the same as the frequency of the second feedback signal, and the energy of the second feedback signal is obtained by multiplying the energy of the third energy gain by the energy of the first feedback signal. Based on the second feedback signal, the remote control device outputs first feedback information.

[0076] In this embodiment, the determining device first determines the difference between the pose change that the guide should achieve (i.e., the first pose change) and the actual pose change (i.e., the second pose change), thus obtaining the pose change difference. Optionally, the pose change difference includes a position difference and an attitude difference, wherein the position difference is the difference between the position indicated by the first pose change and the position indicated by the second pose change, and the attitude difference is the difference between the attitude angle indicated by the first pose change and the attitude angle indicated by the second pose change. Optionally, the pose change difference is obtained by weighted summation of the position difference and the attitude difference.

[0077] The greater the difference in pose change, the higher the probability of the guide malfunctioning during movement, such as colliding with obstacles or vibrating. Therefore, the determining device first obtains a first feedback signal based on a first feedback sub-signal and a second feedback sub-signal. Then, it obtains a third energy gain based on the pose change difference and adjusts the energy of the first feedback signal based on the third energy gain to obtain a second feedback signal, where the third energy gain is positively correlated with the pose change difference. Finally, based on the second feedback signal, the remote control device outputs the first feedback information, allowing the user to more clearly perceive any abnormalities in the guide, thereby improving the safety of remotely controlling the guide's movement using the remote control device.

[0078] In one optional implementation, before performing the step "obtaining the third energy gain of the energy of the first feedback signal based on the pose change difference," the determining device further performs the following steps: acquiring a reference feedback signal, wherein the reference feedback signal is collected by a first torque sensor when the guide has not collided with an obstacle and the guide has not jittered. Determining the difference between the energy of the reference feedback signal and the energy of the first feedback signal to obtain a target energy difference. After obtaining the target energy difference, during the process of performing the step "obtaining the third energy gain of the energy of the first feedback signal based on the pose change difference," the determining device performs the following steps: if the target energy difference is greater than a third threshold, obtaining a third energy gain based on the pose change difference.

[0079] In this implementation, since the reference feedback signal is a signal collected by the first torque sensor when the guide is moving without any abnormalities, the greater the difference between the energy of the first feedback signal and the energy of the reference feedback signal, the greater the probability that the guide will malfunction during its movement in response to the first control command. Therefore, after determining the target energy difference by measuring the difference between the energy of the reference feedback signal and the energy of the first feedback signal, the determining device obtains a third energy gain based on the pose change difference if the target energy difference is greater than a third threshold. This allows for the determination of the third energy gain when the probability of abnormalities in the guide's movement is high, and the adjustment of the feedback intensity based on the third energy gain, thereby improving the feedback effect.

[0080] As an optional implementation, the determining device performs the following steps during the execution of step "obtaining a second feedback sub-signal based on the second torque sub-signal": obtaining a second energy gain based on the frequency of the second torque sub-signal. Specifically, when the frequency of the second torque sub-signal is less than a first threshold, the second energy gain is less than 1; when the frequency of the second torque sub-signal is greater than or equal to the first threshold, the second energy gain is greater than or equal to 1. Based on the second energy gain, the energy of the second torque sub-signal is adjusted to obtain the second feedback sub-signal.

[0081] In this implementation, determining the energy gain corresponding to the torque sub-signal can be divided into two cases: In scenario 1, the frequency of the torque sub-signal is less than the first threshold, indicating that the frequency of the torque sub-signal is low and its change is slow. Because the change of this torque sub-signal is slow, its existence is equivalent to constantly present white noise in relation to the torque experienced by the user-sensing guide during movement. Therefore, in this case, determining that the energy gain of the determining device is less than 1 can reduce the feedback strength of the torque sub-signal, thereby reducing the interference of the torque sub-signal on the torque experienced by the user-sensing guide and improving the feedback effect.

[0082] In scenario 2, if the frequency of the torque sub-signal is greater than or equal to the first threshold, it indicates that the frequency of the torque sub-signal is relatively high and changes rapidly. In this case, the determining device determines that the energy gain is greater than or equal to 1, which can enhance or maintain the feedback strength of the torque sub-signal, thereby allowing the user to perceive the torque sub-signal more clearly and thus improving the feedback effect.

[0083] Therefore, the determining device, based on cases 1 and 2, uses the frequency of the second torque sub-signal to determine the second energy gain, thereby improving the accuracy of the second energy gain.

[0084] Alternatively, the determining device may also determine the first energy gain based on Case 1 and Case 2, using the frequency of the first sub-torque signal.

[0085] In one optional embodiment, the target object is equipped with a second torque sensor for detecting the torque acting on the target object. The guide is equipped with a drive device for driving the target object to move; exemplarily, the drive device is a drive motor, which can drive the target object disposed in the placement slot to move.

[0086] The determining device further performs the following steps: acquiring a second control command, wherein the second control command originates from the teleoperation device and instructs the driving device to drive the target object to move based on a target path. In response to the second control command, controlling the movement of the guide and acquiring a second torque signal, wherein the second torque signal is a signal collected by a second torque sensor during the movement of the target object within the target body membrane. Determining a third torque sub-signal from the second torque signal, the third torque sub-signal having the same frequency as the second torque sub-signal. Based on the third torque sub-signal, obtaining a third feedback sub-signal, wherein the third feedback sub-signal has the same frequency as the third torque sub-signal, and its energy is obtained based on the product of the energy of the third torque sub-signal and a third energy gain, the third energy gain being greater than the second energy gain. Based on the third feedback sub-signal, controlling the teleoperation device to output second feedback information.

[0087] In this embodiment, the teleoperation device can also be used to control the drive device to move the target object within the target membrane. A second torque sensor can detect the torque experienced by the target object during its movement within the target membrane. When the target object enters the target membrane, compared to when it is not inside, the user needs to be able to more clearly perceive the torque experienced. This reduces damage to the target membrane and improves the safety of controlling the target object to move to its target position within the membrane. Therefore, for sub-torque signals of the same frequency, the feedback intensity when the target object enters the membrane should be greater than the feedback intensity when it is not inside.

[0088] Based on this, after determining the third torque sub-signal from the second torque signal, the determining device obtains a third feedback sub-signal based on the third torque sub-signal. The third energy gain, used to determine the magnitude of the energy of the third torque sub-signal, is greater than the second energy gain. Then, based on the third feedback sub-signal, the remote control device outputs second feedback information, which improves safety. The second feedback information is used to provide feedback on the torque experienced by the target object during its movement within the target body.

[0089] Optionally, during the movement of the target object within the target membrane, the torque experienced by the target object includes at least one of the following: a torque generated by the target object's gravity, a torque generated by the target object's amplitude of motion, a torque generated by the target object's velocity, a torque generated by the target object's acceleration, a torque generated by the target object's angular velocity, and a torque generated by the collision between the target object and an obstacle. For example, a torque acting on the target object is generated when the target object collides with a skeletal model within the target membrane.

[0090] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0091] If the technical solution of this application involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, while using clear signs / information to inform users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, personal information processing may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0092] The methods of the embodiments of this application have been described in detail above, and the apparatus of the embodiments of this application is provided below.

[0093] Please see Figure 2 , Figure 2 This is a schematic diagram of a device for determining the pose of a guide provided in an embodiment of this application. The device 1 for determining the pose of a guide includes: an acquisition unit 11 and a processing unit 12, wherein: The acquisition unit 11 is used to acquire the target path, the preset distance range, and the minimum movement distance; the target path is the path by which the target object set on the guide moves from the skin area of ​​the target body membrane to the target position inside the target body membrane; the preset distance range is the range of distances from the guide to the skin area; and the minimum movement distance is the minimum distance that the target object moves in a single movement. Processing unit 12 is configured to obtain a first distance based on the length of the target path, the preset distance range, and the minimum movement distance. The first distance is the distance from the guide to the skin region. The first distance is within the distance range. The sum of the first distance and the length of the target path is an integer multiple of the minimum movement distance. The acquisition unit 11 is further configured to acquire the pose of the optical tracking device, which is configured to determine the pose of the guide by locating the optical markers disposed on the guide. The processing unit 12 is further configured to obtain a first pose based on the target path, the first distance, and the pose of the optical tracking device; when the pose of the guide is the first pose, the movement path of the target object is aligned with the target path, and the angle between the target normal vector and the optical axis of the optical tracking device is less than a preset angle, wherein the target normal vector is the normal vector of the plane formed by at least three optical markers set on the guide.

[0094] In any embodiment of this application, the optical markers disposed on the guide include n marker groups, wherein each marker group includes at least three optical markers, and the planes formed by the optical markers in different marker groups are not parallel, and n is an integer greater than 1; The process of obtaining the first pose based on the target path, the first distance, and the pose of the optical tracking device includes: Based on the target path and the first distance, the second pose of the guide is obtained; when the pose of the guide is the second pose, the movement path of the target object is aligned with the target path; Based on the second pose, n reference vectors are obtained. The reference vector is the normal vector of the plane formed by the optical markers in the n marker sets when the pose of the guide is the second pose. The reference vector corresponds one-to-one with the marker set. Determine the angle between the optical axis and the n reference vectors to obtain n reference angles, and the reference angles correspond one-to-one with the reference vectors; If none of the n reference angles is smaller than the preset angle, the difference between the minimum value of the n reference angles and the preset angle is determined to obtain the reference angle difference. Based on the reference angle difference, the second pose is corrected to obtain the first pose. The first pose is obtained by rotating the second pose around the target path by a target angle, and the target angle is obtained based on the reference angle difference.

[0095] In any embodiment of this application, the guide is provided with a first torque sensor, and the acquisition unit 11 is further configured to acquire a first control command, the first control command being from a remote operating device, the first control command instructing the guide to adjust its pose to the first pose through movement; The processing unit 12 is further configured to respond to the first control command, control the movement of the guide, and acquire a first torque signal, wherein the first torque signal is a signal collected by the first torque sensor during the movement of the guide; The processing unit 12 is further configured to determine a first torque sub-signal and a second torque sub-signal from the first torque signal, wherein the frequency of the first torque sub-signal is less than the frequency of the second torque signal; The processing unit 12 is further configured to obtain a first feedback sub-signal based on the first torque sub-signal, wherein the frequency of the first feedback sub-signal is the same as the frequency of the first torque sub-signal, and the energy of the first feedback sub-signal is obtained based on the product of the energy of the first torque sub-signal and a first energy gain, wherein the first energy gain is a positive number. The processing unit 12 is further configured to obtain a second feedback sub-signal based on the second torque sub-signal, wherein the frequency of the second feedback sub-signal is the same as the frequency of the second torque sub-signal, and the energy of the second feedback sub-signal is obtained based on the product of the energy of the second torque sub-signal and the second energy gain, wherein the first energy gain is less than the second energy gain. The processing unit 12 is further configured to control the teleoperation device to output first feedback information based on the first feedback sub-signal and the second feedback sub-signal.

[0096] In any embodiment of this application, the processing unit 12 is further configured to obtain a first pose change amount based on the first control instruction, wherein the first pose change amount is the pose change amount that the guide should achieve as indicated by the first control instruction. The acquisition unit 11 is further configured to acquire a second pose change amount, wherein the second pose change amount is the actual pose change amount achieved by the guide based on the first control command; The processing unit 12 is also used to determine the difference between the first pose change and the second pose change, and to obtain the pose change difference. The processing unit 12 is further configured to obtain a first feedback signal based on the first feedback sub-signal and the second feedback sub-signal; The processing unit 12 is further configured to obtain a third energy gain of the energy of the first feedback signal based on the pose change difference, wherein the pose change difference is positively correlated with the third energy gain. The processing unit 12 is further configured to adjust the energy of the first feedback signal based on the third energy gain to obtain a second feedback signal, wherein the frequency of the first feedback signal is the same as the frequency of the second feedback signal, and the energy of the second feedback signal is obtained based on the product of the third energy gain and the energy of the first feedback signal. The processing unit 12 is further configured to control the teleoperation device to output the first feedback information based on the second feedback signal.

[0097] In any embodiment of this application, the acquisition unit 11 is further configured to acquire a reference feedback signal, which is collected by the first torque sensor when the guide does not collide with an obstacle and the guide does not vibrate. The processing unit 12 is further configured to determine the difference between the energy of the reference feedback signal and the energy of the first feedback signal, and obtain the target energy difference; The processing unit 12 is further configured to obtain the third energy gain based on the pose change difference when the target energy difference is greater than the third threshold.

[0098] In any embodiment of this application, the processing unit 12 is further configured to obtain the second energy gain based on the frequency of the second torque sub-signal; when the frequency of the second torque sub-signal is less than a first threshold, the second energy gain is less than 1; when the frequency of the second torque sub-signal is greater than or equal to the first threshold, the second energy gain is greater than or equal to 1. The processing unit 12 is further configured to adjust the energy of the second torque sub-signal based on the second energy gain to obtain the second feedback sub-signal.

[0099] In any embodiment of this application, the target object is provided with a second torque sensor, the guide is provided with a driving device, the driving device is used to drive the target object to move, and the acquisition unit 11 is also used to acquire a second control command, the second control command comes from the teleoperation device, and the second control command instructs the driving device to drive the target object to move based on the target path; The processing unit 12 is also configured to respond to the second control command, control the movement of the guide, and acquire a second torque signal, wherein the second torque signal is a signal collected by the second torque sensor during the movement of the target object within the target body membrane; The processing unit 12 is further configured to determine a third torque sub-signal from the second torque signal, wherein the frequency of the third torque sub-signal is the same as the frequency of the second torque sub-signal; The processing unit 12 is further configured to obtain a third feedback sub-signal based on the third torque sub-signal, wherein the frequency of the third feedback sub-signal is the same as the frequency of the third torque sub-signal, and the energy of the third feedback sub-signal is obtained based on the product of the energy of the third torque sub-signal and the third energy gain, wherein the third energy gain is greater than the second energy gain. The processing unit 12 is also used to control the teleoperation device to output second feedback information based on the third feedback sub-signal.

[0100] In this embodiment, the determining device first determines the first distance from the guide to the skin region of the target membrane based on the length of the target path, a preset distance range, and the minimum moving distance. This ensures that when the target object is positioned on the guide, the distance from the target object to the target location within the target membrane is an integer multiple of the minimum moving distance, thus facilitating control of the target object's movement to the target location. Then, based on the target path, the first distance, and the pose of the optical tracking device, the guide's first pose is obtained. This first pose satisfies the following conditions: when the guide's pose is the first pose, the target object's movement path is aligned with the target path, and the angle between the target normal vector and the optical axis of the optical tracking device is less than a preset angle. This improves the accuracy of the first pose.

[0101] In some embodiments, the functions or modules of the apparatus provided in this application can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0102] Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device 2 includes a processor 21 and a memory 22. Optionally, the electronic device 2 also includes an input device 23 and an output device 24. The processor 21, memory 22, input device 23, and output device 24 are coupled together via connectors, which include various interfaces, transmission lines, or buses, etc., and are not limited in this embodiment. It should be understood that in the various embodiments of this application, coupling refers to mutual connection in a specific way, including direct connection or indirect connection through other devices, such as through various interfaces, transmission lines, buses, etc.

[0103] The memory 22 can be used to store computer program instructions, as well as various types of computer program code, including program code for executing the scheme of this application. Optionally, the memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), which is used for related instructions and data.

[0104] Input device 23 is used to input data and / or signals, and output device 24 is used to output data and / or signals. Input device 23 and output device 24 can be independent devices or an integrated device.

[0105] It is understood that in this embodiment of the application, the memory 22 can be used not only to store related instructions, but also to store related data. This embodiment of the application does not limit the specific data stored in the memory.

[0106] Understandable, Figure 3 This is merely a simplified design of an electronic device. In practical applications, the electronic device may also include other necessary components, including, but not limited to, any number of input / output devices, processors, memories, etc., and all electronic devices that can implement the embodiments of this application are within the protection scope of this application.

[0107] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0108] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Those skilled in the art will also readily understand that the various embodiments of this application have different focuses, and for the sake of convenience and brevity, the same or similar parts may not be repeated in different embodiments. Therefore, parts not described or not described in detail in one embodiment can be referred to the descriptions in other embodiments.

[0109] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0110] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0111] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0112] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state disks (SSDs)).

[0113] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or RAM, magnetic disks, or optical disks.

Claims

1. A method for determining the pose of a guide, characterized in that, The method includes: The system acquires the target path, a preset distance range, and a minimum movement distance. The target path is the path taken by the target object, which is set on the guide, to move from the skin area of ​​the target membrane to the target position within the target membrane. The preset distance range is the range of distances from the guide to the skin area. The minimum movement distance is the minimum distance the target object moves in a single movement. Based on the length of the target path, the preset distance range, and the minimum movement distance, a first distance is obtained. The first distance is the distance from the guide to the skin area. The first distance is within the distance range, and the sum of the first distance and the length of the target path is an integer multiple of the minimum movement distance. The pose of an optical tracking device is acquired, wherein the optical tracking device is used to determine the pose of the guide by locating an optical marker disposed on the guide; Based on the target path, the first distance, and the pose of the optical tracking device, a first pose is obtained; when the pose of the guide is the first pose, the movement path of the target object is aligned with the target path, and the angle between the target normal vector and the optical axis of the optical tracking device is less than a preset angle, wherein the target normal vector is the normal vector of the plane formed by at least three optical markers set on the guide.

2. The method according to claim 1, characterized in that, The optical markers disposed on the guide include n marker groups, wherein each marker group includes at least three optical markers, and the planes formed by the optical markers in different marker groups are not parallel, and n is an integer greater than 1; The process of obtaining the first pose based on the target path, the first distance, and the pose of the optical tracking device includes: Based on the target path and the first distance, the second pose of the guide is obtained; when the pose of the guide is the second pose, the movement path of the target object is aligned with the target path; Based on the second pose, n reference vectors are obtained. The reference vector is the normal vector of the plane formed by the optical markers in the n marker sets when the pose of the guide is the second pose. The reference vector corresponds one-to-one with the marker set. Determine the angle between the optical axis and the n reference vectors to obtain n reference angles, and the reference angles correspond one-to-one with the reference vectors; If none of the n reference angles is smaller than the preset angle, the difference between the minimum value of the n reference angles and the preset angle is determined to obtain the reference angle difference. Based on the reference angle difference, the second pose is corrected to obtain the first pose. The first pose is obtained by rotating the second pose around the target path by a target angle, and the target angle is obtained based on the reference angle difference.

3. The method according to claim 1, characterized in that, The guide is equipped with a first torque sensor, and the method further includes: A first control command is obtained, the first control command comes from a teleoperation device, and the first control command instructs the guide to adjust its pose to the first pose by movement; In response to the first control command, the guide is controlled to move, and a first torque signal is acquired, wherein the first torque signal is a signal collected by the first torque sensor during the movement of the guide; A first torque sub-signal and a second torque sub-signal are determined from the first torque signal, wherein the frequency of the first torque sub-signal is less than the frequency of the second torque signal; Based on the first torque sub-signal, a first feedback sub-signal is obtained. The frequency of the first feedback sub-signal is the same as the frequency of the first torque sub-signal. The energy of the first feedback sub-signal is obtained by multiplying the energy of the first torque sub-signal with a first energy gain, where the first energy gain is a positive number. Based on the second torque sub-signal, a second feedback sub-signal is obtained. The frequency of the second feedback sub-signal is the same as the frequency of the second torque sub-signal. The energy of the second feedback sub-signal is obtained based on the product of the energy of the second torque sub-signal and the second energy gain. The first energy gain is less than the second energy gain. Based on the first feedback sub-signal and the second feedback sub-signal, the remote operation device is controlled to output the first feedback information.

4. The method according to claim 3, characterized in that, Before controlling the teleoperation device to output first feedback information based on the first feedback sub-signal and the second feedback sub-signal, the method further includes: Based on the first control command, the first pose change amount is obtained, which is the pose change amount that the guide should achieve as indicated by the first control command. Obtain the second pose change amount, which is the actual pose change amount achieved by the guide based on the first control command; Determine the difference between the first pose change and the second pose change to obtain the pose change difference; The step of controlling the teleoperation device to output first feedback information based on the first feedback sub-signal and the second feedback sub-signal includes: Based on the first feedback sub-signal and the second feedback sub-signal, the first feedback signal is obtained; Based on the pose change difference, a third energy gain of the energy of the first feedback signal is obtained, and the pose change difference is positively correlated with the third energy gain. Based on the third energy gain, the energy of the first feedback signal is adjusted to obtain a second feedback signal. The frequency of the first feedback signal is the same as the frequency of the second feedback signal, and the energy of the second feedback signal is obtained based on the product of the third energy gain and the energy of the first feedback signal. Based on the second feedback signal, the remote operation device is controlled to output the first feedback information.

5. The method according to claim 4, characterized in that, Before obtaining the third energy gain of the energy of the first feedback signal based on the pose change difference, the method further includes: A reference feedback signal is acquired, which is collected by the first torque sensor when the guide does not collide with an obstacle and the guide does not vibrate. The difference between the energy of the reference feedback signal and the energy of the first feedback signal is determined to obtain the target energy difference; The third energy gain, obtained based on the pose change difference, of the energy of the first feedback signal includes: When the target energy difference is greater than the third threshold, the third energy gain is obtained based on the pose change difference.

6. The method according to claim 3, characterized in that, The process of obtaining the second feedback sub-signal based on the second torque sub-signal includes: The second energy gain is obtained based on the frequency of the second torque sub-signal; when the frequency of the second torque sub-signal is less than the first threshold, the second energy gain is less than 1; when the frequency of the second torque sub-signal is greater than or equal to the first threshold, the second energy gain is greater than or equal to 1. Based on the second energy gain, the energy of the second torque sub-signal is adjusted to obtain the second feedback sub-signal.

7. The method according to any one of claims 3 to 6, characterized in that, The target object is equipped with a second torque sensor, the guide is equipped with a driving device, the driving device is used to drive the target object to move, and the method further includes: Obtain a second control command, the second control command being from the teleoperation device, the second control command instructing the drive device to drive the target object to move based on the target path; In response to the second control command, the guide is controlled to move and a second torque signal is acquired. The second torque signal is a signal collected by the second torque sensor during the movement of the target object within the target membrane. A third torque sub-signal is determined from the second torque signal, wherein the frequency of the third torque sub-signal is the same as the frequency of the second torque signal; Based on the third torque sub-signal, a third feedback sub-signal is obtained. The frequency of the third feedback sub-signal is the same as that of the third torque sub-signal. The energy of the third feedback sub-signal is obtained by multiplying the energy of the third torque sub-signal with the third energy gain. The third energy gain is greater than the second energy gain. Based on the third feedback sub-signal, the remote operation device is controlled to output the second feedback information.

8. A device for determining the pose of a guide, characterized in that, The device for determining the pose of the guide includes: The acquisition unit is used to acquire the target path, the preset distance range, and the minimum movement distance; the target path is the path by which the target object set on the guide moves from the skin area of ​​the target body membrane to the target position inside the target body membrane; the preset distance range is the range of distances from the guide to the skin area; and the minimum movement distance is the minimum distance the target object moves in a single movement. The processing unit is configured to obtain a first distance based on the length of the target path, the preset distance range, and the minimum movement distance. The first distance is the distance from the guide to the skin region. The first distance is within the distance range, and the sum of the first distance and the length of the target path is an integer multiple of the minimum movement distance. The acquisition unit is further configured to acquire the pose of the optical tracking device, which is configured to determine the pose of the guide by locating the optical markers disposed on the guide. The processing unit is further configured to obtain a first pose based on the target path, the first distance, and the pose of the optical tracking device; when the pose of the guide is the first pose, the movement path of the target object is aligned with the target path, and the angle between the target normal vector and the optical axis of the optical tracking device is less than a preset angle, wherein the target normal vector is the normal vector of the plane formed by at least three optical markers set on the guide.

9. A surgical robot, characterized in that, Includes the apparatus for determining the pose of the guide as described in claim 8.

10. An electronic device, characterized in that, include: A processor and a memory, the memory being used to store computer program code, the computer program code including computer instructions, wherein, when the processor executes the computer instructions, the electronic device performs the method as described in any one of claims 1 to 7.

Images