Pose adjustment method, device, equipment, medium and surgical robot system

Abstract

Embodiments of the present application disclose a pose adjustment method, device, equipment, medium and surgical robot system. The method determines a first pose of an end lens of a mirror holding robot in a base coordinate system of the mirror holding robot when detecting movement of the end lens, determines a second pose of an end instrument of an instrument robot in a display field coordinate system based on the first pose, determines a target pose of a handle according to the second pose, and adjusts an actual pose of the handle based on the target pose, so that a relative positional relationship between the adjusted actual pose of the handle and the second pose satisfies a preset spatial mapping relationship, thereby achieving automatic adjustment of the pose of the handle when the field is moved, eliminating the need for an operator to frequently switch between surgical operation, field movement and clutching to adjust the pose of the handle, improving the adjustment efficiency of the pose of the handle, and reducing the safety risks caused by operation errors.

Description

Technical Field

[0001] The present invention relates to the field of surgical robots, and more particularly to a pose adjustment method, device, equipment, medium and surgical robot system. Background Technology

[0002] During surgery, operators frequently need to adjust the position of the endoscope to maintain a clear view of the surgical area. These adjustments include displacement and rotation. The actions involved in adjusting the field of view are typically specific movements performed by the operator's hands. Correspondingly, the endoscope-holding robot moves the endoscope according to preset rules in the controller based on the operator's actions. When the endoscope moves, the field of view on the screen changes, causing the operator's hand movements to no longer match the robot's instrument posture, making it difficult to continue accurate surgical procedures. Current technology typically guides the operator to re-match their hand rotation posture to the robot's instrument posture, but this method cannot match the endoscope's displacement position. Therefore, the operator needs to frequently and actively disconnect the master-slave mapping relationship, adjust the handles back to a convenient position, ensure the hand positions match the instrument's position in the field of view, and then return to the master-slave state. This disengagement operation requires the operator to frequently switch between surgical procedures, field of view movements, and disengagement, increasing learning difficulty, reducing efficiency, and increasing the safety risks associated with operational errors. Summary of the Invention

[0003] This invention provides a pose adjustment method, device, equipment, medium, and surgical robot system to achieve automatic adjustment of the handle pose when the field of view is moved, improve the efficiency of handle pose adjustment, and reduce the safety risks caused by operational errors.

[0004] In a first aspect, embodiments of the present invention provide a pose adjustment method, the method comprising:

[0005] If movement of the end-effector lens of the camera-holding robot is detected, then the first pose of the end-effector lens in the base coordinate system of the camera-holding robot is determined;

[0006] Based on the first pose, the second pose of the end effector of the robot in the display field coordinate system is determined, and the target pose of the handle is determined according to the second pose, wherein the end effector is displayed in the display field of view of the end lens.

[0007] The actual pose of the handle is adjusted based on the target pose so that the relative positional relationship between the adjusted actual pose of the handle and the second pose satisfies a preset spatial mapping relationship.

[0008] Secondly, embodiments of the present invention also provide a pose adjustment device, the device comprising:

[0009] The first pose determination module is used to determine the first pose of the end lens in the base coordinate system of the camera-holding robot if the movement of the end lens of the camera-holding robot is detected.

[0010] The target pose determination module is used to determine the second pose of the end effector of the robotic robot in the display field coordinate system based on the first pose, and to determine the target pose of the handle based on the second pose, wherein the end effector is displayed in the display field of view of the end lens.

[0011] The handle pose adjustment module is used to adjust the actual pose of the handle based on the target pose, so that the relative positional relationship between the adjusted actual pose of the handle and the second pose satisfies a preset spatial mapping relationship.

[0012] Thirdly, embodiments of the present invention also provide a surgical robot system, the system comprising a scope-holding robot, at least one instrument robot, a controller, a handle, and a display module, wherein the scope-holding robot includes an end effector lens, and the instrument robot includes end effectors; wherein...

[0013] The display module is used to display the field of view of the end lens;

[0014] The controller is used to adjust the actual pose of the handle according to the pose adjustment method provided in any embodiment of the present invention, so that the relative positional relationship between the adjusted actual pose of the handle and the second pose satisfies a preset spatial mapping relationship.

[0015] Fourthly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0016] One or more processors;

[0017] Storage device for storing one or more programs.

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the pose adjustment method as provided in any embodiment of the present invention.

[0019] Fifthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the pose adjustment method as provided in any embodiment of the present invention.

[0020] The embodiments of the above invention have the following advantages or beneficial effects:

[0021] When the movement of the end effector lens of the end-effector robot is detected, the first pose of the end effector lens in the base coordinate system of the end effector robot is determined. Based on the first pose, the second pose of the end effector of the instrument robot in the display field of view coordinate system is determined. The target pose of the handle is determined according to the second pose. The actual pose of the handle is adjusted based on the target pose so that the relative positional relationship between the adjusted actual pose of the handle and the second pose satisfies the preset spatial mapping relationship. This realizes the automatic adjustment of the handle pose when the field of view is moved, eliminating the need for the operator to frequently switch to adjust the handle pose during surgical operations, field of view movement, and clutch engagement. This improves the efficiency of handle pose adjustment and reduces the safety risks caused by operational errors. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of exemplary embodiments of the present invention, the accompanying drawings used in describing the embodiments are briefly introduced below. Obviously, the accompanying drawings described are only a portion of the drawings of the embodiments to be described in this invention, and not all of the drawings. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0023] Figure 1A This is a flowchart illustrating a pose adjustment method provided in Embodiment 1 of the present invention.

[0024] Figure 1B This is a schematic diagram of an end-lens movement provided in Embodiment 1 of the present invention;

[0025] Figure 2 This is a schematic flowchart of a pose adjustment method provided in Embodiment 2 of the present invention;

[0026] Figure 3 This is a flowchart illustrating a pose adjustment method provided in Embodiment 3 of the present invention;

[0027] Figure 4A This is a schematic diagram of the structure of a surgical robot system provided in Embodiment 4 of the present invention;

[0028] Figure 4B This is a schematic diagram of another surgical robot system provided in Embodiment 4 of the present invention;

[0029] Figure 5 This is a schematic diagram of the structure of a pose adjustment device provided in Embodiment 5 of the present invention;

[0030] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment Six of the present invention. Detailed Implementation

[0031] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0032] Example 1

[0033] Figure 1A This is a flowchart illustrating a pose adjustment method according to Embodiment 1 of the present invention. This embodiment is applicable to situations where the pose of the handle is adjusted accordingly when the end effector of a camera-holding robot moves, so that the pose of the handle and the pose of the robot in the display field of view satisfy a preset spatial mapping relationship. This method can be executed by a pose adjustment device, which can be implemented by hardware and / or software. The method specifically includes the following steps:

[0034] S110. If the movement of the end lens of the lens-holding robot is detected, the first pose of the end lens in the lens-holding robot's base coordinate system is determined.

[0035] The end-effector robot can be a surgical robot whose end effector holds a laparoscope; specifically, the end-effector lens of the end-effector robot can be a laparoscope lens. The end-effector lens of the end-effector robot can capture images of the surgical site. Specifically, during the image capture of the surgical site, if the end-effector instrument of the surgical instrument robot is near the surgical site, the end-effector lens's field of view also includes the end-effector instrument; the surgical instrument robot can be a surgical robot whose end effector holds surgical instruments. Optionally, the end-effector robot and the surgical instrument robot can be separate robots or integrated into a single robot.

[0036] In this embodiment, the operator controls the end effector to perform surgery by manipulating a handle; the handle can drive the robotic arm of the instrument robot to perform master-slave motion, thereby moving the end effector. When the operator needs to further observe the details of the surgical site or the condition of other tissues adjacent to the surgical site during the operation, they can control the end effector lens of the end-effector robot to adjust the field of view. For example, the movement of the end effector lens includes pointing movement of the lens in space, positional movement of the lens in space, and rotation of the lens around the imaging axis.

[0037] Specifically, when the end effector of the camera-holding robot moves, the position of the end effector of the robotic arm within its field of view changes relative to the viewer's perspective. For example, when the end effector's field of view moves to the left, the end effector's position within the field of view moves relatively to the right; when the end effector's field of view rotates clockwise, the end effector's position rotates relatively counterclockwise. Correspondingly, when the end effector's position within the field of view changes relative to its position, the operating position of the handle no longer corresponds to the end effector's position within the field of view. For example... Figure 1B As shown, a schematic diagram of an end-effector movement is illustrated. When the end-effector moves to the left, the position of the end-effector in the field of view moves to the right relative to it. At this time, the operating position of the handle does not match the position of the end-effector in the field of view.

[0038] Therefore, in this embodiment, if movement of the end effector lens of the lens-holding robot is detected, it is necessary to determine the first pose of the end effector lens in the lens-holding robot's base coordinate system, so as to further determine the second pose of the end effector of the instrument robot in the display field of view coordinate system based on the first pose. Specifically, the lens-holding robot's base coordinate system can be a base coordinate system centered on the lens-holding robot; the first pose can be described by the homogeneous transition matrix of the end effector lens in the lens-holding robot's base coordinate system.

[0039] S120. Based on the first pose, determine the second pose of the end effector of the robot in the display field coordinate system, and determine the target pose of the handle according to the second pose, wherein the end effector is displayed in the display field of view of the end lens.

[0040] The display field of view coordinate system can be a coordinate system centered on the display interface of the end-effector's field of view. Specifically, the second pose of the end-effector in the display field of view coordinate system can be determined by the first pose of the end-effector in the base coordinate system of the lens-holding robot; that is, the transformation relationship from the pose of the end-effector to the pose of the lens-holding robot can be determined based on the transformation relationship from the pose of the end-effector to the pose of the lens-holding robot.

[0041] In one optional implementation, determining the second pose of the end effector of the robotic device in the display field of view coordinate system based on the first pose includes: determining the device pose of the end effector of the robotic device in the robotic device base coordinate system, a first transformation relationship of the robotic device base coordinate system in the lens-holding robot base coordinate system, and a second transformation relationship of the pose of the preset target point of the end lens in the end lens mounting coordinate system to the pose of the preset target point in the display image of the end lens; and determining the second pose of the end effector in the display field of view coordinate system based on the first pose, the device pose, the first transformation relationship, and the second transformation relationship.

[0042] The instrument robot's base coordinate system can be a coordinate system centered on the instrument robot; the instrument pose can be a transformation relationship from the end effector to the instrument robot. The first transformation relationship of the instrument robot's base coordinate system under the lens-holding robot's base coordinate system can be a transformation relationship from the instrument robot to the lens-holding robot. The preset target point can be randomly set or determined by the lens's internal parameters. Specifically, the second transformation relationship can be a transformation relationship from the pose of the preset target point in the end effector lens mounting coordinate system to the pose of the preset target point in the displayed image.

[0043] In this optional implementation, the transformation relationship from the end effector to the display field of view coordinate system, i.e., the second pose, can be determined based on the transformation relationship from the end effector to the robot (device pose), the transformation relationship from the end lens to the lens-holding robot (first pose), the transformation relationship from the robot to the lens-holding robot (first transformation relationship), and the transformation relationship from the pose of the preset target point in the end lens mounting coordinate system to the pose of the preset target point in the displayed image (second transformation relationship). For example, Figure 1B As shown, the second pose can be Figure 1B The right-hand image shows the pose of the end effector in the display field coordinate system.

[0044] In one specific implementation, the second pose of the end effector in the display field coordinate system is determined based on the first pose, the device pose, the first transformation relationship, and the second transformation relationship, satisfying the following formula:

[0045]

[0046] in, T represents the second pose of the end effector in the display field coordinate system. camera T represents the second transformation relationship between the pose of the preset target point of the end lens in the end lens mounting coordinate system and the pose of the preset target point in the display image of the end lens. scope T represents the first pose of the end-effector in the base coordinate system of the camera-holding robot. registration This represents the first transformation relationship between the coordinate system of the instrument robot and the coordinate system of the mirror-holding robot. This indicates the device pose of the end effector in the robot's base coordinate system.

[0047] The aforementioned coordinates and transformations can all be described by homogeneous transformation matrices. Specifically, in the above formula, the second pose of the end effector in the display field coordinate system can be obtained by multiplying the second transformation relation, the transpose of the first pose, the first transformation relation, and the device pose. This method allows for the accurate determination of the second pose, thereby ensuring the accuracy of the final adjusted handle pose.

[0048] In this embodiment, after determining the second pose of the end effector in the display field coordinate system, in order to ensure that the second pose of the end effector in the display field coordinate system corresponds to the operating position of the handle, as follows: Figure 1B The correspondence between the handle operation position and the instrument display position shown in the middle left figure allows us to further determine the target position of the handle based on the second position, and then adjust the actual position of the handle based on the target position so that the actual position of the handle corresponds to the second position of the end effector in the display field coordinate system.

[0049] Specifically, determining the target pose of the handle based on the second pose can be achieved by: determining the target pose of the handle based on the second pose and the spatial relative relationship between the display field of view coordinate system and the robotic system. For example, determining the target pose of the handle based on the second pose and the spatial relative relationship between the display field of view coordinate system and the robotic system can satisfy the following formula:

[0050]

[0051] Among them, T gripper For the target pose, This indicates the spatial relative relationship between the display field coordinate system and the robotic robot. This indicates the second pose.

[0052] It should be noted that, in this embodiment, even if the display field of view of the end lens does not have a corresponding display interface, that is, there is no display device to display the display field of view of the end lens, the second pose of the end device in the display field coordinate system and the target pose of the handle can still be determined based on the above method.

[0053] S130. Adjust the actual pose of the handle based on the target pose, so that the relative positional relationship between the adjusted actual pose of the handle and the second pose satisfies a preset spatial mapping relationship.

[0054] In this embodiment, after determining the target pose to which the handle needs to be adjusted, the actual pose of the handle can be adjusted according to the target pose so that the adjusted actual pose corresponds to the second pose. It should be noted that before adjusting the actual pose of the handle, the master-slave mapping relationship between the handle and the robotic arm needs to be broken; that is, the handle no longer drives the robotic arm to perform active movement. Specifically, the relative positional relationship between the adjusted actual pose and the second pose satisfies a preset spatial mapping relationship, which can be the transformation relationship between the pose of the end effector in the display coordinate system and the pose of the handle in the handle mounting base coordinate system.

[0055] Optionally, the relative positional relationship between the actual pose of the handle after adjustment and the second pose satisfies a preset spatial mapping relationship, satisfying the following formula:

[0056]

[0057] Among them, T Gripper The actual position of the handle after adjustment. This is the second pose. This represents the third transformation relationship between the pose of the end effector in the display coordinate system and the pose of the handle in the handle mounting base coordinate system. Through this optional implementation, precise and automatic adjustment of the actual pose of the handle can be achieved, ensuring that the operating position of the handle corresponds to the displayed position of the end effector, thus guaranteeing the accuracy of the surgical procedure.

[0058] It should be noted that the number of instrument robots in this embodiment can be one or more, such as two instrument robots. If there are multiple instrument robots, when the end effector lens is detected to move, the actual pose of the handle can be adjusted according to the pose of the end effector of the instrument robot that has a master-slave control relationship with the handle in the display field coordinate system, so that the actual pose of the handle corresponds to the display position of the end effector of the instrument robot, which facilitates the operator to control the end effector by operating the handle.

[0059] The technical solution of this embodiment determines the first pose of the end-effector of the instrument robot in the display field of view coordinate system when the movement of the end-effector of the instrument robot is detected. Based on the first pose, the second pose of the end-effector of the instrument robot in the display field of view coordinate system is determined. The target pose of the handle is determined according to the second pose. The actual pose of the handle is adjusted based on the target pose so that the relative positional relationship between the adjusted actual pose of the handle and the second pose satisfies the preset spatial mapping relationship. This realizes the automatic adjustment of the handle pose when the field of view is moved, eliminating the need for the operator to frequently switch to adjust the handle pose during surgical operations, field of view movement, and clutch engagement. This improves the adjustment efficiency of the handle pose and reduces the safety risks caused by operational errors.

[0060] Example 2

[0061] Figure 2This is a flowchart illustrating a pose adjustment method according to Embodiment 2 of the present invention. Based on the above embodiments, optionally, the method further includes: if movement of a preset component on the handle is detected, then determining that movement of the end effector lens of the lens-holding robot has been detected, wherein the preset component is used to control the end effector lens of the lens-holding robot; correspondingly, adjusting the actual pose of the handle based on the target pose includes: generating a pose adjustment command based on the target pose and sending it to the control robotic arm corresponding to the handle, so as to control the control robotic arm to adjust the actual pose of the handle based on the pose adjustment command. Explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here. See also Figure 2 The pose adjustment provided in this embodiment includes the following steps:

[0062] S210. If the movement of a preset component on the handle is detected, it is determined that the end lens of the lens-holding robot has been moved, wherein the preset component is used to control the end lens of the lens-holding robot.

[0063] The preset component can be an operating joystick or an adjustment button on the handle. Specifically, in this embodiment, the preset component can be a component used to control the robotic arm of the lens-holding robot. When the operator operates the preset component, the end effector lens of the lens-holding robot will move accordingly.

[0064] For example, the operator controls the end-lens robot to move the lens by operating two joysticks on the handle with their fingers. The movement of the end-lens includes the movement of the lens's direction in space, the movement of the lens's position in space, and the rotation of the lens around the shooting axis. The degrees of freedom contained in the above methods are distributed on the operation of the two joysticks. For example, the left joystick is turned to correspond to the spatial direction of the end-lens's field of view, and the right joystick is turned to correspond to the movement of the end-lens along a single direction and the rotation of the lens around the axis of that movement direction.

[0065] Optionally, the process by which the preset component controls the movement of the end lens satisfies the following formula:

[0066]

[0067] Among them, T scope This indicates the pose of the end-effector in the base coordinate system of the camera-holding robot, that is, the pose in which the end-effector needs to move. registration This represents the first transformation relationship between the coordinate system of the instrument robot and the coordinate system of the mirror-holding robot. This indicates the pose of the end effector in the robot's base coordinate system. T represents the pose of the end effector in the display field of view coordinate system. controlT indicates the control space direction of the preset component. camera This represents a second transformation relationship between the pose of the preset target point of the end-lens camera in the end-lens camera mounting coordinate system and the pose of the preset target point in the displayed image of the end-lens camera. For example, if the preset component is an operating joystick, T... control It can be used to control the spatial orientation by operating the joystick.

[0068] According to the above formula, when a user-controlled preset component movement is detected, the pose of the end effector in the base coordinate system of the camera-holding robot can be calculated based on the movement (control space pointing) of the preset component. That is, the pose that the end effector needs to move to, and the end effector can be controlled to move to that pose. Through this optional implementation method, the precise movement of the end effector controlled by the preset component can be achieved.

[0069] S220. If the movement of the end lens of the lens-holding robot is detected, the first pose of the end lens in the lens-holding robot's base coordinate system is determined.

[0070] S230. Based on the first pose, determine the second pose of the end effector of the robotic device in the display field coordinate system, and determine the target pose of the handle based on the second pose.

[0071] S240. Based on the target pose, a pose adjustment command is generated and sent to the control robot arm corresponding to the handle, so as to control the control robot arm to adjust the actual pose of the handle based on the pose adjustment command, so that the relative positional relationship between the actual pose of the handle after adjustment and the second pose satisfies the preset spatial mapping relationship.

[0072] Specifically, in this embodiment, a preset component on the handle is used to move the field of view of the end effector lens; the handle is used to control the movement of the end effector of the robotic robot. The control robotic arm can be a mechanical device for controlling the movement of the handle; the control robotic arm can be mounted on the handle's operating platform. The pose adjustment command can include information on the degrees of freedom that the control robotic arm needs to move. Specifically, the robotic hand on the control robotic arm can grasp the handle, and during the process of adjusting the actual pose of the handle, the robotic hand on the control robotic arm drives the handle to move; the operator's hand grips the handle, so that the handle can drive the operator's hand to move accordingly. Optionally, an impedance control method or an admittance control method can be used to control the control robotic arm to adjust the actual pose of the handle based on the pose adjustment command, so that the control robotic arm can smoothly and gently drive the handle to ensure the operator's comfort and safety.

[0073] It should be noted that after the robot arm adjusts the actual pose of the handle based on the pose adjustment command, it can be checked whether the actual pose of the handle matches the second pose of the end effector in the display field of view. If so, the master-slave control mode between the handle and the robot can be restarted.

[0074] In this embodiment, when the movement of a preset component on the handle is detected, the movement of the end effector lens of the lens-holding robot is detected. At this time, the first pose of the end effector lens in the base coordinate system of the lens-holding robot is determined. Based on the first pose, the second pose of the end effector of the instrument robot in the display field coordinate system is determined, and the target pose of the handle is determined based on the second pose. A pose adjustment command is generated based on the target pose and sent to the control robot arm corresponding to the handle. The robot arm is controlled to adjust the actual pose of the handle based on the pose adjustment command, thereby realizing automatic adjustment of the handle pose, improving the adjustment efficiency of the handle pose, and reducing the safety risks caused by operational errors.

[0075] Example 3

[0076] Figure 3 This is a flowchart illustrating a pose adjustment method according to Embodiment 3 of the present invention. Optionally, based on the above embodiments, the method further includes: if a change in the current pose on the handle is detected, then determining that movement of the end effector lens of the lens-holding robot has been detected, wherein the handle is used to control the end effector lens of the lens-holding robot. Explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here. See also... Figure 3 The pose adjustment provided in this embodiment includes the following steps:

[0077] S310. If a change in the current pose of the handle is detected, it is determined that the end effector of the lens-holding robot has been moved, wherein the handle is used to control the end effector of the lens-holding robot.

[0078] In this embodiment, the handle can be used to control the movement of the end effector lens of the camera-holding robot. The movement of the end effector lens includes the pointing of the lens in space, the position of the lens in space, and the rotation of the lens around the shooting axis. The operator's hand movements are subject to the following two constraints: i. The relative relationship between the hands; the maintenance of this relative relationship is as follows: if one hand is chosen, in the fixed coordinate system attached to that hand, the position and posture of the other hand remain stationary; ii. Simultaneously, the operator's hand movements are restricted by the camera-holding robot's degrees of freedom, and can only move in the directions that the camera-holding robot can execute and within the range of motion reachable by the camera-holding robot.

[0079] During the movement of the end effector lens of the camera-holding robot, the instrument robot remains stationary. Conversely, as the field of view of the end effector lens of the camera-holding robot shifts, the position of the stationary end effector within the shifted field of view changes relative to its position. For example, when the field of view moves to the left, the end effector moves relatively to the right; when the field of view rotates clockwise, the end effector rotates relatively counterclockwise.

[0080] It should be noted that in this embodiment, because the stationary end effector undergoes relative motion in the opposite direction to the direction of movement when the end effector lens moves, the direction of movement of the handle can be opposite to the direction of movement of the end effector lens it controls, so that the display position of the handle after movement still corresponds to the display position of the end effector. For example, when the handle moves to the left, the end effector lens it controls moves to the right. At this time, the position of the end effector in the field of view of the end effector lens moves to the left relatively, and the operating position of the handle corresponds to the position of the end effector in the field of view of the end effector lens.

[0081] Optionally, the process by which the handle controls the movement of the end lens satisfies the following formula:

[0082]

[0083] Among them, T scope This indicates the pose of the end-effector in the base coordinate system of the camera-holding robot, that is, the pose in which the end-effector needs to move. registration This represents the first transformation relationship between the coordinate system of the instrument robot and the coordinate system of the mirror-holding robot. This indicates the pose of the end effector in the robot's base coordinate system. T represents the pose of the end effector in the display field of view coordinate system. control Indicates the control space direction of the handle, T camera The second transformation relationship represents the pose of the preset target point of the end lens in the end lens mounting coordinate system to the pose of the preset target point in the display image of the end lens.

[0084] According to the above formula, when a user's control handle movement is detected, the pose of the end effector in the robot's base coordinate system can be calculated based on the handle's movement (control space pointing), i.e., the pose where the end effector needs to move, and the end effector can be controlled to move to that pose. Through this optional implementation, precise movement of the end effector controlled by the handle can be achieved.

[0085] S320. If the movement of the end lens of the lens-holding robot is detected, the first pose of the end lens in the lens-holding robot's base coordinate system is determined.

[0086] S330. Based on the first pose, determine the second pose of the end effector of the robotic device in the display field coordinate system, and determine the target pose of the handle based on the second pose.

[0087] S340. Adjust the actual pose of the handle based on the target pose, so that the relative positional relationship between the adjusted actual pose of the handle and the second pose satisfies a preset spatial mapping relationship.

[0088] In this embodiment, the handle can be used to control the movement of the end-effector lens. Since the direction of handle movement is opposite to the direction of end-effector lens movement, the actual operating position of the handle still corresponds to the position of the end-effector device in the display field of view after the end-effector's field of view moves. It should be noted that during handle movement, the target pose obtained using the above method can restrict the handle's movement. That is, it is still necessary to calculate the second pose and the target pose based on the above method, and adjust the actual pose of the handle using the target pose so that the actual pose of the handle corresponds to the second pose of the end-effector device in the display field of view coordinate system. In other words, during handle movement in this embodiment, a preset spatial mapping relationship must be satisfied with the second pose.

[0089] After adjusting the field of view of the end-effector lens based on the handle, if the actual position of the handle matches the position of the end-effector in the display field of view, the handle can be set to master-slave control mode to control the end-effector.

[0090] In the technical solution of this embodiment, the handle can be used to control the end effector of the end effector robot. During the movement of the handle, it needs to satisfy a preset spatial mapping relationship with the second pose, so that the actual pose of the handle corresponds to the second pose of the end effector in the display field coordinate system. This realizes the automatic correspondence between the handle operation position and the instrument display position when the field of view is moved. The operator does not need to frequently switch to adjust the handle pose during surgical operation, field of view movement and disengagement, which improves the adjustment efficiency of the handle pose and reduces the safety risks caused by operation errors.

[0091] Example 4

[0092] Figure 4AThis is a schematic diagram of a surgical robot system provided in Embodiment 4 of the present invention. This embodiment is applicable to situations where operators adjust their field of vision during surgery to observe adjacent tissues of the surgical site or observe the surgical site from different angles. The system specifically includes a scope-holding robot 41, at least one instrument robot 42, a controller 43, a handle 44, and a display module 45. The scope-holding robot 41 includes an end effector 410, the instrument robot 42 includes an end effector 420, and the display module 45 is used to display the field of vision of the end effector 410. The controller 43 is used to adjust the actual pose of the handle 44 according to the pose adjustment method provided in the above embodiment, so that the relative positional relationship between the adjusted actual pose of the handle 44 and the second pose satisfies a preset spatial mapping relationship.

[0093] Optionally, this embodiment also provides another surgical robot system, such as Figure 4B The diagram shown illustrates the structure of another surgical robot system provided in this embodiment. (Combined with...) Figure 4B The surgical robot system includes an instrument robotic arm 1, surgical instruments 11, an instrument robotic arm 2, surgical instruments 21, a scope-holding robotic arm 3, an endoscope 31, a handle console 4, a control robotic arm 5, a handle 6, a handle joystick 61, a controller 7, and a display device 8.

[0094] Surgical instrument 11 is mounted at the end of surgical arm 1, surgical instrument 12 is mounted at the end of surgical arm 2, and endoscope 31 is mounted at the end of endoscope-holding arm 3. Control arms 5 are mounted on the handle control console 4 to control handle 6; there can be two control arms 5. There can also be two handle joysticks 61, each mounted on handle 6, operated by the operator using their fingers. The controller 7 can be installed independently in other devices or on the handle control console 4. A display device 8 can be mounted on the handle control console 4 to display the field of view of the endoscope 31.

[0095] Specifically, during the operation, the operator moves the robotic arms 1 and 2 by operating the handle and using the master-slave control mode of the handle. The operator's hand movements on the handle can drive the control robotic arm 5 to move. The six degrees of freedom of the operator's hand on the handle 6 are collected by the control robotic arm 5. The collected movements of the two hands are used as motion commands and sent to the controller 7. The controller 7 performs kinematic coordinate transformation according to the motion commands, and drives the robotic arms 1 and 2, as well as the surgical instruments 11 and 21 installed at the end effector, to perform corresponding movements in space.

[0096] When the operator needs to move the endoscope's field of view, the operator can actively deactivate the master-slave control mode of the handle 6. This means the robotic arm 5 will no longer drive the instrument robotic arms 1 and 2 in master-slave motion. Simultaneously, the operator can control the endoscope 31 to move the field of view by manipulating the two joysticks 61 on the handle. At this time, the controller 7 can detect the movement of the joysticks 61 and determine the first pose of the endoscope 31 in the base coordinate system of the holding robotic arm 3. Based on this first pose, it further determines the second pose of the surgical instrument 11 or 21 in the display field of view coordinate system. Further, based on the second pose, it determines the target pose of the handle 6 and sends a pose adjustment command to the control robotic arm 5, causing the control robotic arm 5 to move the handle. After the movement, the actual pose of the handle and the second pose satisfy a preset spatial mapping relationship.

[0097] The surgical robot system provided in this embodiment can automatically adjust the handpiece position when the field of view is moved, eliminating the need for the operator to frequently switch between surgical operations, field of view movements, and clutch engagement / disengagement processes to adjust the handpiece position. This improves the efficiency of handpiece position adjustment and reduces the safety risks caused by operational errors.

[0098] Example 5

[0099] Figure 5 This is a schematic diagram of a pose adjustment device provided in Embodiment 5 of the present invention. This embodiment can be applied to the situation where the pose of the handle is adjusted accordingly when the end lens of the lens-holding robot moves, so that the pose of the handle and the pose of the robot in the display field of view meet the preset spatial mapping relationship. The device specifically includes: a first pose determination module 510, a target pose determination module 520, and a handle pose adjustment module 530.

[0100] The first pose determination module 510 is used to determine the first pose of the end lens in the base coordinate system of the lens-holding robot if the movement of the end lens of the lens-holding robot is detected.

[0101] The target pose determination module 520 is used to determine the second pose of the end effector of the robotic robot in the display field coordinate system based on the first pose, and to determine the target pose of the handle according to the second pose, wherein the end effector is displayed in the display field of view of the end lens.

[0102] The handle pose adjustment module 530 is used to adjust the actual pose of the handle based on the target pose, so that the relative positional relationship between the adjusted actual pose of the handle and the second pose satisfies a preset spatial mapping relationship.

[0103] Optionally, the target pose determination module 520 includes a first determination unit and a second determination unit, wherein the first determination unit is used to determine the pose of the end effector of the robotic device in the robotic device base coordinate system, a first transformation relationship of the robotic device base coordinate system in the lens-holding robot base coordinate system, and a second transformation relationship of the pose of the preset target point of the end lens in the end lens mounting coordinate system to the pose of the preset target point in the display image of the end lens; the second determination unit is used to determine the second pose of the end effector in the display field coordinate system based on the first pose, the device pose, the first transformation relationship and the second transformation relationship.

[0104] Optionally, the second determining unit is specifically used to determine the second pose of the end effector in the display field coordinate system according to the following formula:

[0105]

[0106] in, T represents the second pose of the end effector in the display field coordinate system. camera T represents the second transformation relationship between the pose of the preset target point of the end lens in the end lens mounting coordinate system and the pose of the preset target point in the display image of the end lens. scope T represents the first pose of the end-effector in the base coordinate system of the camera-holding robot. registration This represents the first transformation relationship between the coordinate system of the instrument robot and the coordinate system of the mirror-holding robot. This indicates the device pose of the end effector in the robot's base coordinate system.

[0107] Optionally, the device further includes a first movement determination module, which is used to determine that the end effector of the lens-holding robot has moved if the movement of a preset component on the handle is detected, wherein the preset component is used to control the end effector of the lens-holding robot; correspondingly, the handle pose adjustment module is specifically used to generate a pose adjustment command based on the target pose and send it to the control robotic arm corresponding to the handle, so as to control the control robotic arm to adjust the actual pose of the handle based on the pose adjustment command.

[0108] Optionally, the device further includes a second movement determination module, which is used to determine that movement of the end effector of the lens-holding robot is detected if a change in the current pose of the handle is detected, wherein the handle is used to control the end effector of the lens-holding robot.

[0109] Optionally, the relative positional relationship between the actual pose of the handle after adjustment and the second pose satisfies a preset spatial mapping relationship, satisfying the following formula:

[0110]

[0111] Among them, T Gripper The actual position of the handle after adjustment. This is the second pose. This represents the third transformation relationship between the pose of the end effector in the display coordinate system and the pose of the handle in the handle mounting base coordinate system.

[0112] Optionally, the process by which the preset component or the handle controls the movement of the end lens satisfies the following formula:

[0113]

[0114] Among them, T scope T represents the pose of the end-effector lens in the base coordinate system of the lens-holding robot. registration This represents the first transformation relationship between the coordinate system of the instrument robot and the coordinate system of the mirror-holding robot. This indicates the pose of the end effector in the robot's base coordinate system. T represents the pose of the end effector in the display field of view coordinate system. control Indicates the control space orientation of the preset component or the handle, T camera The second transformation relationship represents the pose of the preset target point of the end lens in the end lens mounting coordinate system to the pose of the preset target point in the display image of the end lens.

[0115] In this embodiment, the first pose determination module determines the first pose of the end-effector of the end-effector in the base coordinate system of the end-effector robot when the movement of the end-effector of the end-effector is detected. The target pose determination module determines the second pose of the end-effector of the instrument robot in the display field coordinate system based on the first pose. The target pose of the handle is determined based on the second pose. The handle pose adjustment module adjusts the actual pose of the handle based on the target pose so that the relative positional relationship between the adjusted actual pose of the handle and the second pose satisfies the preset spatial mapping relationship. This realizes the automatic adjustment of the handle pose when the field of view is moved, eliminating the need for the operator to frequently switch to adjust the handle pose during surgical operations, field of view movement, and clutch engagement, thus improving the adjustment efficiency of the handle pose and reducing the safety risks caused by operational errors.

[0116] The pose adjustment device provided in the embodiments of the present invention can execute the pose adjustment method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method.

[0117] It is worth noting that the various units and modules included in the above system are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of the present invention.

[0118] Example 6

[0119] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment Six of the present invention. Figure 6 A block diagram is shown of an exemplary electronic device 12 suitable for implementing embodiments of the present invention. Figure 6 The electronic device 12 shown is merely an example and should not be construed as limiting the functionality or scope of the embodiments of the present invention. Device 12 is typically an electronic device that performs handle pose adjustment functions.

[0120] like Figure 6 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, memory 28, and bus 18 connecting different components (including memory 28 and processing unit 16).

[0121] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0122] Electronic device 12 typically includes a variety of computer-readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0123] Memory 28 may include computer device readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer storage media. By way of example only, storage device 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 6 Not shown; usually referred to as a "hard drive"). Although Figure 6 Not shown, disk drives for reading and writing to removable non-volatile disks (e.g., "floppy disks") and optical disc drives for reading and writing to removable non-volatile optical discs (e.g., Compact Disc-Read Only Memory (CD-ROM), Digital Video Disc-Read Only Memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product 40 having a set of program modules 42 configured to perform the functions of the embodiments of the present invention. Program product 40 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0124] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, mouse, camera, etc., and monitor), and with one or more devices that enable a user to interact with electronic device 12, and / or with any device that enables electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., Local Area Network (LAN), Wide Area Network (WAN) and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, Redundant Arrays of Independent Disks (RAID) devices, tape drives, and data backup storage devices.

[0125] Processor 16 executes various functional applications and data processing by running programs stored in memory 28, such as implementing the pose adjustment method provided in the above embodiments of the present invention, including:

[0126] If movement of the end-effector lens of the camera-holding robot is detected, then the first pose of the end-effector lens in the base coordinate system of the camera-holding robot is determined;

[0127] Based on the first pose, the second pose of the end effector of the robot in the display field coordinate system is determined, and the target pose of the handle is determined according to the second pose, wherein the end effector is displayed in the display field of view of the end lens.

[0128] The actual pose of the handle is adjusted based on the target pose so that the relative positional relationship between the adjusted actual pose of the handle and the second pose satisfies a preset spatial mapping relationship.

[0129] Of course, those skilled in the art will understand that the processor can also implement the technical solutions of the pose adjustment method provided in any embodiment of the present invention.

[0130] Example 7

[0131] Embodiment 7 of the present invention also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the program implements the pose adjustment method steps provided in any embodiment of the present invention, the method comprising:

[0132] If movement of the end-effector lens of the camera-holding robot is detected, then the first pose of the end-effector lens in the base coordinate system of the camera-holding robot is determined;

[0133] Based on the first pose, the second pose of the end effector of the robot in the display field coordinate system is determined, and the target pose of the handle is determined according to the second pose, wherein the end effector is displayed in the display field of view of the end lens.

[0134] The actual pose of the handle is adjusted based on the target pose so that the relative positional relationship between the adjusted actual pose of the handle and the second pose satisfies a preset spatial mapping relationship.

[0135] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0136] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0137] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0138] Computer program code for performing the operations of embodiments of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0139] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A pose adjustment method, characterized in that, The method includes: If movement of the end-effector lens of the camera-holding robot is detected, then the first pose of the end-effector lens in the base coordinate system of the camera-holding robot is determined; Based on the first pose, the second pose of the end effector of the robot in the display field coordinate system is determined, and the target pose of the handle is determined according to the second pose, wherein the end effector is displayed in the display field of view of the end lens. The actual pose of the handle is adjusted based on the target pose so that the relative positional relationship between the adjusted actual pose of the handle and the second pose satisfies a preset spatial mapping relationship.

2. The method according to claim 1, characterized in that, Based on the first pose, the second pose of the end effector of the robotic robot in the display field coordinate system is determined, including: Determine the device pose of the end effector of the robotic system in the robotic system base coordinate system, the first transformation relationship of the robotic system base coordinate system in the lens-holding robot base coordinate system, and the second transformation relationship of the pose of the preset target point of the end lens in the end lens mounting coordinate system to the pose of the preset target point in the display image of the end lens. Based on the first pose, the device pose, the first transformation relationship, and the second transformation relationship, the second pose of the end effector in the display field coordinate system is determined.

3. The pose adjustment method according to claim 2, characterized in that, The second pose of the end effector in the display field coordinate system is determined based on the first pose, the device pose, the first transformation relationship, and the second transformation relationship, satisfying the following formula: in, This indicates the second pose of the end effector in the display field of view coordinate system. This represents a second transformation relationship between the pose of the preset target point of the end lens in the end lens mounting coordinate system and the pose of the preset target point in the display image of the end lens. This indicates the first pose of the end-effector lens in the base coordinate system of the lens-holding robot. This represents the first transformation relationship between the instrument robot's base coordinate system and the mirror-holding robot's base coordinate system. This indicates the device pose of the end effector in the robot's base coordinate system.

4. The pose adjustment method according to claim 1, characterized in that, The method further includes: If the movement of a preset component on the handle is detected, it is determined that the end lens of the lens-holding robot has been moved, wherein the preset component is used to control the end lens of the lens-holding robot; Accordingly, adjusting the actual pose of the handle based on the target pose includes: Based on the target pose, a pose adjustment command is generated and sent to the control robotic arm corresponding to the handle, so as to control the control robotic arm to adjust the actual pose of the handle based on the pose adjustment command.

5. The pose adjustment method according to claim 1, characterized in that, The method further includes: If a change in the current pose of the handle is detected, it is determined that movement of the end effector of the lens-holding robot has been detected, wherein the handle is used to control the end effector of the lens-holding robot.

6. The pose adjustment method according to claim 1, characterized in that, The relative positional relationship between the actual pose of the adjusted handle and the second pose satisfies a preset spatial mapping relationship, as shown in the following formula: in, The actual position of the handle after adjustment. This is the second pose. This represents the third transformation relationship between the pose of the end effector in the display field coordinate system and the pose of the handle in the handle mounting base coordinate system.

7. The pose adjustment method according to claim 4 or 5, characterized in that, The process by which the preset component or the handle controls the movement of the end lens satisfies the following formula: in, This indicates the pose of the end-effector lens in the base coordinate system of the lens-holding robot. This represents the first transformation relationship between the instrument robot's base coordinate system and the mirror-holding robot's base coordinate system. This indicates the pose of the end effector in the robot's base coordinate system. This indicates the pose of the end effector in the display field of view coordinate system. This indicates the control space orientation of the preset component or the handle. The second transformation relationship represents the pose of the preset target point of the end lens in the end lens mounting coordinate system to the pose of the preset target point in the display image of the end lens.

8. A posture adjustment device, characterized in that, The device includes: The first pose determination module is used to determine the first pose of the end lens in the base coordinate system of the camera-holding robot if the movement of the end lens of the camera-holding robot is detected. The target pose determination module is used to determine the second pose of the end effector of the robotic robot in the display field coordinate system based on the first pose, and to determine the target pose of the handle based on the second pose, wherein the end effector is displayed in the display field of view of the end lens; The handle pose adjustment module is used to adjust the actual pose of the handle based on the target pose, so that the relative positional relationship between the adjusted actual pose of the handle and the second pose satisfies a preset spatial mapping relationship.

9. A surgical robot system, characterized in that, The system includes a lens-holding robot, at least one instrument robot, a controller, a handle, and a display module. The lens-holding robot includes an end effector lens, and the instrument robot includes an end effector. The display module is used to display the field of view of the end lens; The controller is configured to adjust the actual pose of the handle according to any one of the pose adjustment methods described in claims 1-7, so that the relative positional relationship between the adjusted actual pose of the handle and the second pose satisfies a preset spatial mapping relationship.

10. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the pose adjustment method as described in any one of claims 1-7.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the pose adjustment method as described in any one of claims 1-7.

Images