Surgical robot control method, apparatus, device, medium and system

Abstract

Embodiments of the present application disclose a surgical robot control method, device, equipment, medium and system. The method determines the reference pose of the endoscope in the endoscope robot coordinate system when detecting the movement of the endoscope of the endoscope robot, determines the target pose of the end instrument of the instrument robot in the instrument robot coordinate system based on the reference pose of the endoscope, and then adjusts the actual pose of the end instrument based on the target pose, so that the change of the pose of the end instrument in the display field of view of the endoscope satisfies the preset change condition during the movement of the endoscope, realizing the automatic adjustment of the instrument pose during the movement of the display field of view, ensuring that the operation position of the handle corresponds to the position of the instrument in the display field of view, without the need for the operator to frequently switch between the surgical operation, the movement of the display field of view and the clutching process to adjust the instrument pose, improving the adjustment efficiency of the instrument pose and reducing the safety risk caused by the operation error when manually adjusting the instrument pose.

Description

Technical Field

[0001] The present invention relates to the field of surgical robot technology, and in particular to a surgical robot control method, device, equipment, medium and system. Background Technology

[0002] During surgery, operators frequently need to adjust the position of the endoscope to maintain a clear view of the surgical area. These adjustments include displacement and rotation. The actions to adjust the field of view are typically specific movements performed by the operator's hands in coordination. Correspondingly, the endoscope-holding robot moves the endoscope according to preset rules in the controller based on the operator's actions. When the endoscope moves, the field of view on the screen changes, causing the operator's hand movements to no longer match the instrument posture of the robot, making it difficult to continue accurate surgical procedures. Therefore, the operator needs to frequently and actively disconnect the master-slave mapping relationship, manually adjust the handles back to a convenient position, ensure that the position of both hands matches the position of the instruments in the field of view, and then return to the master-slave state, i.e., a disengagement operation. Summary of the Invention

[0003] This invention provides a surgical robot control method, device, equipment, medium, and system to achieve automatic adjustment of instrument posture when the field of view is moved, thereby improving the efficiency of instrument posture adjustment and reducing the safety risks caused by operational errors when manually adjusting instrument posture.

[0004] In a first aspect, embodiments of the present invention provide a surgical robot control method, the method comprising:

[0005] If movement of the end-effector of the end-effector robot is detected, the reference pose of the end-effector in the base coordinate system of the end-effector robot is determined.

[0006] Based on the laparoscopic reference pose, the target pose of the end effector of the instrument robot in the instrument robot base coordinate system is determined.

[0007] The actual pose of the end effector is adjusted based on the target pose so that the pose change of the end effector in the display field of the end ...

[0008] Secondly, embodiments of the present invention also provide a surgical robot control device, the device comprising:

[0009] The reference pose determination module is used to determine the reference pose of the end-effector in the base coordinate system of the lens-holding robot if movement of the end-effector of the lens-holding robot is detected.

[0010] The target pose determination module is used to determine the target pose of the end effector of the instrument robot in the instrument robot base coordinate system based on the laparoscope reference pose.

[0011] The instrument pose adjustment module is used to adjust the actual pose of the end effector based on the target pose, so that the pose change of the end effector in the display field of view of the end effector during the movement of the end end endoscope meets the preset change conditions.

[0012] Thirdly, embodiments of the present invention also provide a surgical robot control system, the system comprising a scope-holding robot, at least one instrument robot, a handle, a display, and a controller, wherein the scope-holding robot includes an endoscope, and the instrument robot includes end-effectors; wherein...

[0013] The display is used to show the field of view of the end end endoscope;

[0014] The controller is used to adjust the actual pose of the end effector based on the surgical robot control method provided in any embodiment of the present invention, so that the pose change of the end effector in the display field of view of the end effector during the movement of the end end endoscope meets the preset change conditions.

[0015] Fourthly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0016] One or more processors;

[0017] Storage device for storing one or more programs.

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the surgical robot control method provided in any embodiment of the present invention.

[0019] Fifthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the surgical robot control method provided in any embodiment of the present invention.

[0020] The embodiments of the above invention have the following advantages or beneficial effects:

[0021] When the movement of the end-effector of the end-effector robot is detected, the reference pose of the end-effector in the robot's base coordinate system is determined. Based on the reference pose, the target pose of the instrument of the instrument robot in the instrument robot's base coordinate system is determined. Then, the actual pose of the end-effector is adjusted based on the target pose, so that the change in the pose of the end-effector in the display field of view of the end-effector meets the preset change conditions during the movement of the end-effector. This realizes the automatic adjustment of the instrument pose when the field of view is moved, ensuring that the operating position of the handle corresponds to the position of the instrument in the display field of view. The operator does not need to frequently switch during surgical operations, field of view movement, and clutch engagement to adjust the instrument pose, which improves the adjustment efficiency of the instrument pose and reduces the safety risks caused by operational errors when manually adjusting the instrument pose. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of exemplary embodiments of the present invention, the accompanying drawings used in describing the embodiments are briefly introduced below. Obviously, the accompanying drawings described are only a portion of the drawings of the embodiments to be described in this invention, and not all of the drawings. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0023] Figure 1A This is a flowchart illustrating a surgical robot control method provided in Embodiment 1 of the present invention;

[0024] Figure 1B This is a schematic diagram of the movement of an end-effector provided in Embodiment 1 of the present invention;

[0025] Figure 2 This is a flowchart illustrating a surgical robot control method provided in Embodiment 2 of the present invention;

[0026] Figure 3A This is a schematic diagram of the structure of a surgical robot control system provided in Embodiment 3 of the present invention;

[0027] Figure 3B This is a schematic diagram of another surgical robot control system provided in Embodiment 3 of the present invention;

[0028] Figure 4 This is a schematic diagram of the structure of a surgical robot control device provided in Embodiment 4 of the present invention;

[0029] Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of the present invention. Detailed Implementation

[0030] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0031] Example 1

[0032] Figure 1 is a schematic flowchart of a surgical robot control method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where, when the operator adjusts the position of the endoscope to perform surgical operations at another position, the instrument posture is automatically adjusted so that the instrument posture does not change relative to the moving display field of view. This method can be executed by a surgical robot control device, which can be implemented by hardware and / or software. The method specifically includes the following steps:

[0033] S110. If the movement of the end-effector of the lens-holding robot is detected, the reference pose of the end-effector in the lens-holding robot's base coordinate system is determined.

[0034] The end-effector robot can be a surgical robot whose end-effector holds a laparoscope. Specifically, the end-effector of the end-effector robot can image the surgical site. During this imagery, the end-effector's field of view typically also includes the end-effector of a surgical robot, which can be a surgical robot whose end-effector holds surgical instruments. This allows the operator to observe the surgical site and perform surgical procedures by controlling the end-effector of the surgical robot. The end-effector robot and the surgical robot can be separate robots or integrated together.

[0035] For example, the operator controls the end effector to perform surgery by manipulating a handle; the handle drives the robotic arm of the surgical instrument to perform master-slave motion, which in turn drives the end effector. When the operator needs to operate on another site during surgery, they can control the end-effector of the end-effector robot to move, adjusting the end-effector's field of view to the other site. For example, the movement of the end-effector includes pointing the lens in space, moving the lens position in space, and rotating the lens around the imaging axis.

[0036] In this embodiment, if the end effector of the end-effector robot moves, the end effector of the instrument robot remains stationary. Therefore, the position of the end effector within the display field of view of the end effector will change relatively. For example, when the end effector's field of view moves to the left, the position of the end effector within the field of view will move relatively to the right; when the end effector's field of view rotates clockwise, the position of the end effector within the field of view will rotate relatively counterclockwise. Accordingly, when the position of the end effector within the display field of view of the end effector changes relatively, the operating position of the handle no longer corresponds to the position of the end effector within the display field of view. Figure 1B As shown, a schematic diagram of end-effector movement is illustrated. When the end-effector moves to the upper right, the position of the end instrument in the field of view moves to the lower left. At this time, the operating position of the handle does not match the position of the end instrument in the display field of view.

[0037] Therefore, in this embodiment, if movement of the end-effector of the end-effector robot is detected, the instrument robot needs to be controlled to perform corresponding movements to ensure that the position of the end-effector in the display field of view of the end-effector does not change, thereby ensuring that the operating position of the handle corresponds to the position of the end-effector in the display field of view. Specifically, if movement of the end-effector of the end-effector of the end-effector is detected, the reference pose of the end-effector in the base coordinate system of the end-effector needs to be determined, so as to further determine the target pose of the end-effector of the instrument robot in the base coordinate system of the instrument robot based on the reference pose of the end-effector. Specifically, the base coordinate system of the end-effector can be a base coordinate system centered on the end-effector robot; the reference pose of the end-effector can be described by the homogeneous transition matrix of the end-effector in the base coordinate system of the end-effector.

[0038] S120. Based on the laparoscopic reference pose, determine the target pose of the end effector of the instrument robot in the instrument robot base coordinate system.

[0039] In this context, the base coordinate system of the instrument robot can be a coordinate system centered on the instrument robot; the target pose can be the pose that the end effector needs to be adjusted to, specifically, the target pose can be the transformation relationship from the end effector to the instrument robot. In this embodiment, the transformation relationship from the end effector to the instrument robot can be determined based on the transformation relationship from the pose of the end endoscope to the pose of the end-effector robot.

[0040] In one optional implementation, determining the target pose of the end effector of the instrument robot in the instrument robot base coordinate system based on the endoscope reference pose includes: determining a first transformation relationship of the instrument robot base coordinate system in the endoscope-holding robot base coordinate system, a second transformation relationship of the pose of the preset target point of the endoscope in the endoscope mounting coordinate system to the pose of the preset target point in the display image of the endoscope, and the instrument display pose of the end effector of the instrument robot in the display field of view coordinate system; and determining the target pose of the end effector in the instrument robot base coordinate system based on the first transformation relationship, the second transformation relationship, the instrument display pose, and the endoscope reference pose.

[0041] The first transformation relationship between the instrument robot's base coordinate system and the lens-holding robot's base coordinate system can be a transformation relationship from the instrument robot to the lens-holding robot. The preset target point can be randomly set or determined by the lens's internal parameters. Specifically, the second transformation relationship can be a transformation relationship from the pose of the preset target point in the end-effector mounting coordinate system to the pose of the preset target point in the displayed image. The display field of view coordinate system can be a coordinate system centered on the display interface displaying the field of view of the end-effector lens.

[0042] In this optional implementation, the target pose of the end effector in the instrument robot's base coordinate system can be determined based on the end effector's reference pose in the instrument robot's base coordinate system, the transformation relationship from the instrument robot to the instrument robot (first transformation relationship), the instrument display pose of the instrument robot's end effector in the display field coordinate system, and the transformation relationship from the pose of the preset target point in the end effector's mounting coordinate system to the pose of the preset target point in the displayed image (second transformation relationship). For example... Figure 1B As shown, the target pose can be Figure 1B The right-hand figure shows the desired pose of the end effector.

[0043] For example, the target pose of the end effector in the robot base coordinate system is determined based on the coordinate system pose, the endoscope lens pose, the instrument display pose, and the endoscope reference pose, satisfying the following formula:

[0044]

[0045] in, This represents the target pose of the end effector in the robot's base coordinate system. Indicates the instrument's displayed pose, T camera T represents the second transformation relation. scope Indicates the laparoscope reference pose, T registration This represents the first transformation relationship. Both the coordinates and transformations described above can be described using a homogeneous transformation matrix.

[0046] Specifically, in the above formula, the target pose of the end effector in the robot's base coordinate system can be obtained by multiplying the instrument display pose, the transpose matrix of the second transformation relationship, the endoscope reference pose, and the transpose matrix of the first transformation relationship. This method allows for accurate determination of the target pose, ensuring that the end effector's pose in the display field of view remains unchanged before and after the endoscope's movement, thus ensuring that the handle's operating position matches the end effector's position in the display field of view.

[0047] S130. Adjust the actual pose of the end effector based on the target pose so that the pose change of the end effector in the display field of view of the end effector during the movement of the end end endoscope meets the preset change conditions.

[0048] In this embodiment, after determining the target pose of the end effector in the robot's base coordinate system, the actual pose of the end effector can be adjusted based on the target pose so that the change in pose of the end effector in the display field of view of the end endoscope meets a preset change condition. The preset change condition can be that the pose change value is zero, or that the pose change value does not exceed a set threshold. Specifically, this embodiment can aim to ensure that the pose of the end effector in the display field of view does not change before and after the movement of the end endoscope, and adjust the actual pose of the end effector based on the target pose.

[0049] It should be noted that the number of instrument robots in this embodiment can be one or more, such as two instrument robots. If there are multiple instrument robots, when the end effector lens is detected to move, the actual pose of the end effector of each instrument robot can be adjusted according to the target pose of each instrument robot's end effector, so that the pose change of each end effector in the display field of view meets the preset change conditions, which facilitates the operator to control the end effector through the operating handle.

[0050] The technical solution of this embodiment determines the reference pose of the end-effector of the end-effector in the robot's base coordinate system when the movement of the end-effector of the end-effector is detected. Based on the reference pose, the target pose of the end-effector of the instrument robot in the instrument robot's base coordinate system is determined. Then, the actual pose of the end-effector is adjusted based on the target pose. This ensures that the change in the pose of the end-effector in the display field of view of the end-effector meets preset change conditions during the movement of the end-effector, guaranteeing that the operating position of the handle corresponds to the position of the instrument in the display field of view. This achieves automatic adjustment of the instrument pose when the field of view is moved, eliminating the need for the operator to frequently switch between surgical operations, field of view movements, and clutch engagement to adjust the instrument pose. This improves the efficiency of instrument pose adjustment and reduces the safety risks caused by operational errors when manually adjusting the instrument pose.

[0051] Example 2

[0052] Figure 2 This is a flowchart illustrating a surgical robot control method according to Embodiment 2 of the present invention. Based on the above embodiments, this embodiment optionally further includes: if movement of a preset component on the handle is detected, then determining that movement of the end-effector of the end-effector robot has been detected, wherein the preset component is used to control the end-effector of the end-effector robot. Explanations of terms that are the same as or corresponding to those in the above embodiments will not be repeated here. See also Figure 2 The surgical robot control provided in this embodiment includes the following steps:

[0053] S210. If the movement of a preset component on the handle is detected, it is determined that the end-effector of the lens-holding robot has been moved, wherein the preset component is used to control the end-effector of the lens-holding robot.

[0054] The preset component can be an operating joystick or an adjustment button on the handle. Specifically, in this embodiment, the preset component can be a component used to control the robotic arm of the endoscope-holding robot. When the operator operates the preset component, the robotic arm of the endoscope-holding robot can drive the endoscope to move accordingly.

[0055] For example, the operator controls the end-effector robot to move by operating two joysticks on the handle with their fingers. The movement of the end-effector includes the movement of the lens in space, the movement of the lens position in space, and the rotation of the lens around the shooting axis. The degrees of freedom contained in the above methods are distributed on the operation of the two joysticks. For example, the left joystick is turned to correspond to the spatial orientation of the end-effector's field of view, and the right joystick is turned to correspond to the movement of the end-effector along a single direction and the rotation of the end-effector around the axis of that movement direction.

[0056] Optionally, the process by which the preset component controls the movement of the end-effector of the end-effector robot satisfies the following formula:

[0057]

[0058] Among them, T scope T represents the pose of the end-effector in the base coordinate system of the end-effector robot. registration This represents the first transformation relationship between the coordinate system of the instrument robot and the coordinate system of the mirror-holding robot. This indicates the pose of the end effector in the robot's base coordinate system. T represents the pose of the end effector in the display field of view coordinate system. control T indicates the control space direction of the preset component. camera This represents a second transformation relationship between the pose of a preset target point of the end-effector in the end-effector mounting coordinate system and the pose of the preset target point in the displayed image of the end-effector. For example, if the preset component is an operating joystick, T... control It can be used to control the spatial orientation by operating the joystick.

[0059] According to the above formula, when a user-controlled preset component movement is detected, the pose of the end-effector in the base coordinate system of the end-effector robot can be calculated based on the movement (control space orientation) of the preset component. That is, the pose at which the end-effector needs to move, and the end-effector can be controlled to move to that pose. Through this optional implementation method, precise movement of the end-effector controlled by the preset component can be achieved.

[0060] S220. If the movement of the end-effector of the end-effector is detected, the reference pose of the end-effector in the base coordinate system of the end-effector is determined.

[0061] S230. Based on the laparoscopic reference pose, determine the target pose of the end effector of the instrument robot in the instrument robot base coordinate system.

[0062] S240. Adjust the actual pose of the end effector based on the target pose so that the pose change of the end effector in the display field of view of the end effector during the movement of the end end endoscope satisfies the preset change conditions.

[0063] Optionally, adjusting the actual pose of the end effector based on the target pose includes: generating a control signal based on the target pose, and sending the control signal to the robot to control the robot to adjust the actual pose of the end effector based on the control signal.

[0064] The control signal may include information on the degrees of freedom of the robotic arm. Specifically, the robotic arm can use this control signal to move the end effector, thereby adjusting the actual pose of the end effector.

[0065] In one optional implementation, after adjusting the actual pose of the end effector based on the target pose, the method further includes: determining whether the position of the control robot corresponding to the handle has changed; if not, activating a master-slave control mode between the handle and the robot; wherein the control robot is used to control the pose of the handle, and in the master-slave control mode, the handle is the master device, the robot is the slave device, and the master device controls the slave device.

[0066] The control robotic arm can be a mechanical device used to control the movement of a handle; it can be mounted on the handle's operating platform. Specifically, the robotic hand on the control robotic arm can grasp the handle. If the operator unintentionally triggers the handle during the movement of the endoscope, the handle will drive the movement of the control robotic arm connected to it, and the control robotic arm can collect the handle's movement information. At this time, the handle's operating position no longer corresponds to the position of the endoscope in the display field of view. The actual pose of the handle needs to be readjusted to make the handle's operating position correspond to the position of the endoscope in the display field of view. Alternatively, the position of the endoscope in the display field of view can be adjusted according to the actual pose of the handle to make the handle's operating position correspond to the position of the endoscope in the display field of view. Of course, if the position of the control robotic arm corresponding to the handle does not change, it indicates that the handle's position has not changed during the movement of the endoscope and adjustment of the endoscope's instrument pose. In this case, the adjusted position of the endoscope in the display field of view matches the handle's operating position, and the master-slave control mode between the handle and the instrument robot can be initiated; in this mode, the handle is the master device, and the instrument robot is the slave device, with the master device controlling the slave device. It should be noted that the surgical robot control method provided in this embodiment can disconnect the master-slave control mode between the handle and the instrument robot before adjusting the actual pose of the end effector based on the target pose, until the adjustment of the actual pose of the end effector is completed.

[0067] In this optional implementation, by determining whether the position of the control robot corresponding to the handle has changed, it is ensured that the position of the handle has not changed when adjusting the actual pose of the end effector. This ensures that the position of the adjusted end effector in the display field corresponds to the operating position of the handle, improving the accuracy of the device pose adjustment and avoiding the mismatch between the position of the handle and the end effector caused by the operator's misoperation of the handle. This reduces the safety risk of operational errors caused by mismatch in position.

[0068] In this embodiment, if the movement of a preset component on the handle is detected, it is determined that the end-effector of the end-effector robot has moved. Then, the reference pose of the end-effector in the robot's base coordinate system is determined. Based on the reference pose, the target pose of the end-effector of the instrument robot in the instrument robot's base coordinate system is determined. Finally, the actual pose of the end-effector is adjusted based on the target pose, ensuring that the change in the end-effector's pose within the end-effector's display field of view during movement satisfies preset change conditions. This achieves automatic adjustment of the instrument's pose during field of view movement, eliminating the need for frequent operator switching during surgical procedures, field of view movement, and disengagement to adjust the instrument's pose. This improves the efficiency of instrument pose adjustment and reduces the safety risks associated with manual adjustment errors.

[0069] Example 3

[0070] Figure 3A This is a schematic diagram of a surgical robot control system provided in Embodiment 3 of the present invention. This embodiment is applicable to situations where the operator adjusts the field of view during surgery to operate on other surgical sites or tissues. The system specifically includes a scope-holding robot 31, at least one instrument robot 32, a handle 33, a display 34, and a controller 35. The scope-holding robot 31 includes an endoscope 310, and the instrument robot 32 includes an end instrument 320. The display 34 is used to display the field of view of the endoscope 310. The controller 35 is used to adjust the actual pose of the end instrument 320 based on the surgical robot control methods provided in the above embodiments, so that the change in pose of the end instrument 320 in the field of view of the endoscope 310 during the movement of the endoscope 340 satisfies a preset change condition.

[0071] Optionally, this embodiment also provides another surgical robot control system, such as... Figure 3B The diagram shown illustrates the structure of another surgical robot control system provided in this embodiment. (Combined with...) Figure 3B The surgical robot system includes an instrument robotic arm 1, surgical instruments 1-1, instrument robotic arm 2, surgical instruments 2-1, endoscope robotic arm 3, endoscope 3-1, handle console 4, control robotic arm 5, handle 6, handle joystick 6-1, controller 7, and display device 8.

[0072] Surgical instrument 1-1 is mounted at the end of surgical arm 1, surgical instrument 1-2 is mounted at the end of surgical arm 2, and endoscope 3-1 is mounted at the end of endoscope-holding arm 3. Control arm 5 is mounted on handle console 4 to control handle 6; there can be two control arms 5. There can be two handle joysticks 6-1, each mounted on handle 6, operated by the operator by shaking them with their fingers. Controller 7 can be installed independently in other devices or on handle console 4. Display device 8 can be mounted on handle console 4 to display the field of view of endoscope 3-1.

[0073] Specifically, during the operation, the operator moves the robotic arms 1 and 2 by operating the handle and using the master-slave control mode of the handle. The operator's hand movements on the handle can drive the control robotic arm 5 to move. The six degrees of freedom of the operator's hand on the handle 6 are collected by the control robotic arm 5. The collected movements of the two hands are used as motion commands and sent to the controller 7. The controller 7 performs kinematic coordinate transformation according to the motion commands, and drives the robotic arms 1 and 2, as well as the surgical instruments 1-1 and 2-1 installed at the end effector, to perform corresponding movements in space.

[0074] When the operator needs to move the endoscope's field of view, the operator can actively deactivate the master-slave control mode of the handle 6, meaning the robotic arm 5 will no longer drive the instrument robotic arms 1 and 2 in master-slave motion. Simultaneously, the operator can control the endoscope 3-1 to move the field of view by manipulating the two joysticks 6-1 on the handle. At this time, the controller 7 can detect the movement of the joysticks 6-1 and determine the endoscope reference pose of the endoscope 3-1 in the coordinate system of the endoscope-holding robotic arm 3. Based on this reference pose, it further determines the target pose of surgical instrument 1-1 in the coordinate system of the instrument robot 1, or the target pose of surgical instrument 2-1 in the coordinate system of the instrument robot 2. Furthermore, based on this target pose, it sends a pose adjustment command to instrument robot 1 or instrument robot 2, causing instrument robot 1 or instrument robot 2 to control the robotic arm to move surgical instrument 1-1 or surgical instrument 1-2.

[0075] The surgical robot control system provided in this embodiment can automatically adjust the instrument position when the field of vision is moved, eliminating the need for the operator to frequently switch between surgical operations, field of vision movement, and clutch engagement to adjust the instrument position. This improves the efficiency of instrument position adjustment and reduces the safety risks caused by operational errors.

[0076] Example 4

[0077] Figure 4This is a schematic diagram of a surgical robot control device provided in Embodiment 4 of the present invention. This embodiment can be applied to situations where the operator adjusts the position of the endoscope to perform surgical operations at another position, and automatically adjusts the instrument posture so that the instrument does not undergo relative posture changes in the moving display field of view. The device specifically includes: a reference posture determination module 410, a target posture determination module 420, and an instrument posture adjustment module 430.

[0078] The reference pose determination module 410 is used to determine the reference pose of the end-effector in the base coordinate system of the lens-holding robot if the movement of the end-effector of the lens-holding robot is detected.

[0079] The target pose determination module 420 is used to determine the target pose of the end effector of the instrument robot in the instrument robot base coordinate system based on the endoscopic reference pose.

[0080] The instrument pose adjustment module 430 is used to adjust the actual pose of the end-effector based on the target pose, so that the pose change of the end-effector in the display field of view of the end-effector meets the preset change conditions during the movement of the end-effector.

[0081] Optionally, the target pose determination module 420 includes a first determination unit and a second determination unit, wherein the first determination unit is used to determine a first transformation relationship of the instrument robot base coordinate system in the mirror-holding robot base coordinate system, a second transformation relationship of the pose of the preset target point of the end-effector in the end-effector mounting coordinate system to the pose of the preset target point in the display image of the end-effector, and the instrument display pose of the end-effector of the instrument robot in the display field of view coordinate system; the second determination unit is used to determine the target pose of the end-effector in the instrument robot base coordinate system based on the first transformation relationship, the second transformation relationship, the instrument display pose, and the end-effector reference pose.

[0082] Optionally, the second determining unit is specifically used to determine the target pose of the end effector in the robot's base coordinate system according to the following formula:

[0083]

[0084] in, This represents the target pose of the end effector in the robot's base coordinate system. Indicates the instrument's displayed pose, T camera T represents the second transformation relation. scope Indicates the laparoscope reference pose, T registration This indicates the first transformation relationship.

[0085] Optionally, the device includes a laparoscope movement detection module, which is used to determine that the end-effector of the laparoscope-holding robot has been detected moving if a movement of a preset component on the handle is detected, wherein the preset component is used to control the end-effector of the laparoscope-holding robot.

[0086] Optionally, the instrument pose adjustment module 430 is specifically used to generate a control signal based on the target pose, and send the control signal to the instrument robot, so as to control the instrument robot to adjust the actual pose of the end effector based on the control signal.

[0087] Optionally, the device further includes a handle detection module, which is used to determine whether the position of the control robot corresponding to the handle has changed after the actual pose of the end effector is adjusted based on the target pose. If not, the module initiates a master-slave control mode between the handle and the robot. In this mode, the control robot is used to control the pose of the handle, and the handle is the master device, the robot is the slave device, and the master device controls the slave device.

[0088] Optionally, the process by which the preset component controls the movement of the end-effector of the end-effector robot satisfies the following formula:

[0089]

[0090] Among them, T scope T represents the pose of the end-effector in the base coordinate system of the end-effector robot. registration This represents the first transformation relationship between the coordinate system of the instrument robot and the coordinate system of the mirror-holding robot. This indicates the pose of the end effector in the robot's base coordinate system. T represents the pose of the end effector in the display field of view coordinate system. control T indicates the control space direction of the preset component. camera This represents a second transformation relationship between the pose of the preset target point of the end-effector in the end-effector mounting coordinate system and the pose of the preset target point in the display image of the end-effector.

[0091] In this embodiment, the reference pose determination module determines the reference pose of the end-effector of the end-effector in the robot's base coordinate system when the movement of the end-effector of the end-effector is detected. The target pose determination module determines the target pose of the end-effector of the instrument robot in the instrument robot's base coordinate system based on the reference pose of the end-effector. Then, the instrument pose adjustment module adjusts the actual pose of the end-effector based on the target pose, so that the change in the pose of the end-effector in the display field of view of the end-effector meets the preset change conditions during the movement of the end-effector. This realizes the automatic adjustment of the instrument pose when the field of view is moved, eliminating the need for the operator to frequently switch between surgical operations, field of view movement, and clutch engagement to adjust the instrument pose, thus improving the adjustment efficiency of the instrument pose and reducing the safety risks caused by operational errors when manually adjusting the instrument pose.

[0092] The surgical robot control device provided in the embodiments of the present invention can execute the surgical robot control method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0093] It is worth noting that the various units and modules included in the above system are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of the present invention.

[0094] Example 5

[0095] Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of the present invention. Figure 5 A block diagram is shown of an exemplary electronic device 12 suitable for implementing embodiments of the present invention. Figure 5 The illustrated electronic device 12 is merely an example and should not be construed as limiting the functionality or scope of the embodiments of the present invention. Device 12 is typically an electronic device that performs control functions for a surgical robot.

[0096] like Figure 5 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, memory 28, and bus 18 connecting different components (including memory 28 and processing unit 16).

[0097] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0098] Electronic device 12 typically includes a variety of computer-readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0099] Memory 28 may include computer device readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer storage media. By way of example only, storage device 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 5 Not shown; usually referred to as a "hard drive"). Although Figure 5 Not shown, disk drives for reading and writing to removable non-volatile disks (e.g., "floppy disks") and optical disc drives for reading and writing to removable non-volatile optical discs (e.g., Compact Disc-Read Only Memory (CD-ROM), Digital Video Disc-Read Only Memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product 40 having a set of program modules 42 configured to perform the functions of the embodiments of the present invention. Program product 40 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0100] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, mouse, camera, etc., and monitor), and with one or more devices that enable a user to interact with electronic device 12, and / or with any device that enables electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., Local Area Network (LAN), Wide Area Network (WAN) and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, Redundant Arrays of Independent Disks (RAID) devices, tape drives, and data backup storage devices.

[0101] Processor 16 executes various functional applications and data processing by running programs stored in memory 28, such as implementing the surgical robot control method provided in the above embodiments of the present invention, including:

[0102] If movement of the end-effector of the end-effector robot is detected, the reference pose of the end-effector in the base coordinate system of the end-effector robot is determined.

[0103] Based on the laparoscopic reference pose, the target pose of the end effector of the instrument robot in the instrument robot base coordinate system is determined.

[0104] The actual pose of the end effector is adjusted based on the target pose so that the pose change of the end effector in the display field of the end ...

[0105] Of course, those skilled in the art will understand that the processor can also implement the technical solutions of the surgical robot control method provided in any embodiment of the present invention.

[0106] Example 6

[0107] Embodiment 6 of the present invention also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the program implements the steps of the surgical robot control method provided in any embodiment of the present invention, the method comprising:

[0108] If movement of the end-effector of the end-effector robot is detected, the reference pose of the end-effector in the base coordinate system of the end-effector robot is determined.

[0109] Based on the laparoscopic reference pose, the target pose of the end effector of the instrument robot in the instrument robot base coordinate system is determined.

[0110] The actual pose of the end effector is adjusted based on the target pose so that the pose change of the end effector in the display field of the end ...

[0111] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0112] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0113] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0114] Computer program code for performing the operations of embodiments of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A surgical robot control method, characterized in that, The method includes: If movement of the end-effector of the end-effector robot is detected, the reference pose of the end-effector in the base coordinate system of the end-effector robot is determined. Based on the laparoscopic reference pose, the target pose of the end effector of the instrument robot in the instrument robot base coordinate system is determined. The actual pose of the end effector is adjusted based on the target pose so that the pose change of the end effector in the display field of view of the end ...

2. The method according to claim 1, characterized in that, The step of determining the target pose of the end effector of the robotic instrument in the robotic instrument base coordinate system based on the laparoscope reference pose includes: The first transformation relationship of the instrument robot base coordinate system under the lens-holding robot base coordinate system, the second transformation relationship of the pose of the preset target point of the end end endoscope in the end end end endoscope mounting coordinate system to the pose of the preset target point in the display image of the end ... Based on the first transformation relationship, the second transformation relationship, the instrument display pose, and the endoscope reference pose, the target pose of the end effector in the instrument robot base coordinate system is determined.

3. The surgical robot control method according to claim 2, characterized in that, Based on the first transformation relationship, the second transformation relationship, the instrument display pose, and the endoscope reference pose, the target pose of the end effector in the instrument robot base coordinate system is determined, satisfying the following formula: ； in, This represents the target pose of the end effector in the robot's base coordinate system. This indicates the position and orientation of the instrument. Indicates the second transformation relation. Indicates the reference pose of the endoscope. This indicates the first transformation relationship.

4. The surgical robot control method according to claim 1, characterized in that, Also includes: If the movement of a preset component on the handle is detected, it is determined that the end-effector of the end-effector robot has been moved, wherein the preset component is used to control the end-effector of the end-effector robot.

5. The surgical robot control method according to claim 1, characterized in that, The adjustment of the actual pose of the end effector based on the target pose includes: A control signal is generated based on the target pose and sent to the robotic instrument so that the robotic instrument can be controlled to adjust the actual pose of the end effector based on the control signal.

6. The surgical robot control method according to claim 4, characterized in that, After adjusting the actual pose of the end effector based on the target pose, the method further includes: Determine whether the position of the control robot corresponding to the handle has changed; if not, activate the master-slave control mode between the handle and the robot. The control robotic arm is used to control the position and posture of the handle. In the master-slave control mode, the handle is the master device and the robotic robot is the slave device. The master device controls the slave device.

7. The surgical robot control method according to claim 4, characterized in that, The process by which the preset component controls the movement of the end-effector of the end-effector robot satisfies the following formula: ； in, This indicates the pose of the end-effector in the coordinate system of the end-effector robot. This represents the first transformation relationship between the coordinate system of the instrument robot and the coordinate system of the mirror-holding robot. This indicates the pose of the end effector in the robot's base coordinate system. This indicates the pose of the end effector in the display field of view coordinate system. This indicates the control space orientation of the preset component. This represents a second transformation relationship between the pose of the preset target point of the end-effector in the end-effector mounting coordinate system and the pose of the preset target point in the display image of the end-effector.

8. A surgical robot control device, characterized in that, The device includes: The reference pose determination module is used to determine the reference pose of the end-effector in the base coordinate system of the lens-holding robot if movement of the end-effector of the lens-holding robot is detected. The target pose determination module is used to determine the target pose of the end effector of the instrument robot in the instrument robot base coordinate system based on the laparoscope reference pose. The instrument pose adjustment module is used to adjust the actual pose of the end effector based on the target pose, so that the pose change of the end effector in the display field of view of the end effector during the movement of the end end endoscope meets the preset change conditions, wherein the preset change conditions include the pose change value being zero or not exceeding a set threshold.

9. A surgical robot control system, characterized in that, The system includes a scope-holding robot, at least one instrument robot, a handle, a display, and a controller. The scope-holding robot includes an endoscope, and the instrument robot includes end-effectors. The display is used to show the field of view of the end end endoscope; The controller is used to adjust the actual pose of the end effector based on the surgical robot control method according to any one of claims 1-7, so that the pose change of the end effector in the display field of view of the end effector during the movement of the end end endoscope meets the preset change conditions.

10. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the surgical robot control method as described in any one of claims 1-7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the surgical robot control method as described in any one of claims 1-7.

Images