Method for end effector replacement, surgical robot system, and related apparatus

Abstract

Provided are a method and apparatus for end effector replacement, an electronic device, and a computer-readable storage medium. The method comprises: determining, in a surgical robot system (A), a target effector holding arm (211a) requiring end effector replacement, and determining an end effector currently mounted on the target effector holding arm (211a) and a target end effector to be mounted onto the target effector holding arm (211a) for replacement; and controlling a second mechanical arm (61) to perform end effector replacement on the target effector holding arm (211a). The end effector replacement comprises at least one of the following: controlling the second mechanical arm (61) to remove the currently mounted end effector from the target effector holding arm (211a); and / or controlling the second mechanical arm (61) to grasp the target end effector and mount the target end effector onto the target effector holding arm (211a). The method and apparatus for end effector replacement can improve surgical efficiency and reduce labor costs.

Description

Surgical instrument replacement methods, surgical robot systems and related devices Cross-reference to related applications

[0001] This application is based on and claims priority to Chinese Patent Application No. 202411894562.5, filed on December 20, 2024, the entire contents of which are hereby incorporated herein by reference. Technical Field

[0002] This application relates to the field of surgical instrument installation, and more particularly to a method for replacing surgical instruments, a surgical robot system, electronic devices, and computer-readable storage media. Background Technology

[0003] Surgical robot systems are widely used in various surgeries. They offer advantages such as accurate positioning, stable operation, high dexterity, large working range, and immunity to radiation and infection. These advantages help improve the precision of surgical procedures, allowing surgeons to perform operations in a more comfortable state. This is of great value in improving surgical success rates and reducing patient suffering. A surgical robot system may include at least one robotic arm capable of mounting surgical instruments, and the surgical instruments mounted on the robotic arm.

[0004] In related technologies, during surgery using a surgical robot system, the surgeon controlling the robot frequently needs to change surgical instruments mounted on the robotic arm. However, this surgeon is typically located at a control panel far from the robotic arm, rather than next to the arm itself where the instruments need changing. Usually, the surgeon at the control panel issues voice instructions, which are then manually changed by a physician near the patient. This method requires the physician to be constantly present at the operating table to respond to the surgeon's instrument-changing requests, resulting in low surgical efficiency and high labor costs. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a method for replacing surgical instruments, a surgical robot system, an electronic device, and a computer-readable storage medium. The technical solutions are as follows:

[0006] According to a first aspect of this application, a method for replacing surgical tools is provided for replacing surgical tools in a surgical robot system, the surgical robot system including at least one first robotic arm and a second robotic arm, the first robotic arm including a holding arm, the surgical tool being detachably mounted on the holding arm and capable of performing surgical operations based on the control of the first robotic arm; the method includes: determining, in the surgical robot system, a target holding arm for which a surgical tool needs to be replaced, and determining the surgical tool currently mounted on the target holding arm, and a target surgical tool to be replaced to the target holding arm; controlling the second robotic arm to replace the surgical tool on the target holding arm, the replacement of the surgical tool including at least one of the following: controlling the second robotic arm to remove the currently mounted surgical tool from the target holding arm; and / or, controlling the second robotic arm to grasp the target surgical tool and install the target surgical tool onto the target holding arm.

[0007] According to a second aspect of this application, a surgical robot system is provided, the system comprising: a first robotic arm capable of mounting surgical tools and performing surgical operations in response to a master control device controlling the surgical tools; and a second robotic arm configured to grasp surgical tools and mount them onto the first robotic arm or remove surgical tools from the first robotic arm.

[0008] According to a third aspect of this application, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the method as described in the first aspect.

[0009] According to a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method as described in the first aspect.

[0010] The technical solution provided in this application, after identifying the target robotic arm from which the surgical tool needs to be replaced, the surgical tool currently installed on the target robotic arm, and the target surgical tool to be replaced in the target robotic arm within at least one first robotic arm, can control a second robotic arm to remove the currently installed surgical tool from the target robotic arm, and control the second robotic arm to grasp the target surgical tool and install it onto the target robotic arm. Utilizing the automatically operating second robotic arm, the surgical tool on the robotic arm can be automatically replaced during the surgery, replacing the manual replacement operation by the doctor beside the patient. This eliminates the need for the doctor to be constantly present at the operating table, thus achieving automated replacement of surgical tools, improving surgical efficiency, and reducing labor costs.

[0011] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0013] Figure 1a is a schematic diagram of the structure of a surgical robot system according to an embodiment of this application;

[0014] Figure 1b is a schematic diagram of the structure of a first robot according to an embodiment of this application;

[0015] Figure 2 is a schematic diagram of the structure of a surgical robot system according to an embodiment of this application;

[0016] Figure 3 is a structural schematic diagram of a surgical robot system according to another embodiment of this application;

[0017] Figure 4 is a structural schematic diagram of a surgical robot system according to another embodiment of this application;

[0018] Figure 5 is a schematic diagram of the structure of a surgical tool according to an embodiment of this application;

[0019] Figure 6 is a flowchart illustrating a surgical tool replacement method according to this application;

[0020] Figure 7 is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art should fall within the scope of protection of this application.

[0022] Figure 1a shows a schematic diagram of a surgical robot system A. Referring to Figure 1a, the surgical robot system A may include a console 1 and a first robot 2. The console 1 may include a display device for displaying the surgical instrument environment and a processor. The display device has an observation window (also called a stereoscopic display) for the doctor to observe, and the processor is used for information processing and program execution. In addition, the console 1 may also include some or all of the following: an operation control mechanism, armrests, and control switches (not shown in the figure). The actions of the operation control mechanism correspond to the actions of the surgical instruments. By controlling the operation control mechanism, the position of the surgical instruments can be adjusted. The armrests are used to support the doctor's arms. The control switches can be switches that are easy to touch or press with the hand or foot to perform various functional operations and complete human-machine interaction.

[0023] The first robot 2 includes at least one first robotic arm 21. The first robotic arm 21 includes several connecting arms, with adjacent connecting arms moving relative to each other with specific degrees of freedom. This allows the end effector of the first robotic arm 21 to achieve multiple degrees of freedom (e.g., 7 degrees of freedom, depending on the surgical tool). The end effector of the first robotic arm 21 is a holding arm 211, on which a manipulator assembly for driving the surgical tool and the surgical tool (e.g., a surgical instrument or an image acquisition device) are mounted. The surgical tool is detachably mounted on the holding arm 211 and can perform surgical operations based on the control of the first robotic arm 21. Taking the surgical tool as an image acquisition device as an example, the image acquisition device can be an endoscope (e.g., a 3D endoscope) and is detachably mounted on the manipulator assembly of the holding arm 211. The images acquired by the image acquisition device (e.g., surgical images) can be displayed on the display device of the control console 1. It is understood that the image acquisition device disclosed herein can also be of other types, such as a laparoscope.

[0024] At least one first robotic arm 21 may be mounted on one or more bases, and at least one first robotic arm 21 may be mounted on each base. For ease of description, each robotic arm on each base may be defined as a robot.

[0025] Optionally, the surgical robot system A described above may further include an image processing system 3. The image processing system 3 includes a display device (e.g., a screen) that can display images acquired by the image acquisition device. The image acquisition device can acquire images and send them to the image processing system 3, so that the display device included in the image processing system 3 can display the acquired images. The image acquisition device can send the acquired images directly or via the console 1 to the image processing system 3. The image acquisition device can perform image processing (e.g., decoding) on ​​the acquired images using its own processor or via the processor of the image processing system 3. The processed images can also be sent to other image processing devices for further image processing (e.g., noise reduction, contrast enhancement, sharpness improvement, etc.).

[0026] Taking a specific implementation of the first robot 2 shown in Figure 1b as an example, the first robot 2 may include a plurality of first robotic arms 21, each first robotic arm 21 including a holding arm 211, a manipulator assembly 212 disposed on the holding arm 211, and a surgical tool 4 that can be mounted on the manipulator assembly 212. The first robot 2 may also include an inlet assembly 5, through which the surgical tool 4 can enter the patient's body for surgery. In some embodiments, the first robot 2 may include one first robotic arm 21, and the first robotic arm 21 may include at least two holding arms 211, each holding arm 211 being respectively mounted with a manipulator assembly 212 and a surgical tool 4, that is, a first robotic arm 21 may be mounted with multiple surgical tools 4.

[0027] In related technologies, during surgery using a surgical robot system, the surgeon controlling the robot frequently needs to change surgical instruments mounted on the robotic arm. However, this surgeon is typically located at a control console (main control device) far from the robotic arm, rather than directly next to the arm itself. Usually, the surgeon issues voice instructions from the control console, and a physician near the patient manually changes the surgical instruments on the robotic arm based on these instructions. This method requires the physician near the patient to be constantly present at the operating table to respond to the surgeon's instrument-changing requests, resulting in low surgical efficiency and high labor costs. A specific scenario will illustrate this below. Please refer to Figures 1a and 1b. In this specific scenario, the same first robotic arm 21 may need to be equipped with different surgical tools 4 corresponding to different surgical goals, as the surgical target may change at any time. Therefore, for this first robotic arm 21, the surgeon has the need to change the surgical tools 4 at any time. The surgeon is usually at the control console 1 (i.e., the main control device for the first robotic arm 21) to control the first robotic arm 21. The control console 1 is often far away from the first robotic arm 21 (e.g., in different indoor spaces). Next to the operating table, that is, near the first robotic arm 21, there is often a doctor by the patient's side. The doctor by the patient's side can respond to the surgeon's need to change the surgical tools 4 and manually change the surgical tools 4 on the first robotic arm 21.

[0028] It is worth noting that the above description of a scenario for changing surgical instruments is merely an illustrative example. In actual applications, there may be other scenarios for changing surgical instruments, which are not specifically limited here.

[0029] In the specific scenario described above, when the surgeon at console 1 needs to change different surgical tools 4 during the operation, he may issue multiple change instructions to the doctor next to the patient. The doctor next to the patient needs to respond to each instruction and perform the corresponding manual change operation. The speed and accuracy of the manual change operation are affected by the doctor next to the patient's proficiency, and there is a possibility that the doctor next to the patient may misunderstand the instructions or make operational errors. The surgical risk is high, the change process is relatively inefficient, and the labor cost is also high.

[0030] Based on this, referring to Figure 2, the first robotic arm 21 of this application can mount surgical tools and control the surgical tools to perform surgical operations in response to the main control device (control console 1); the second robotic arm 61 of this application is configured to grasp surgical tools and mount them onto the first robotic arm 21 or remove surgical tools from the first robotic arm 21. Referring to Figure 3, the control device 11 of this application can be used to determine the target holding arm 211a that needs to replace surgical tools in the surgical robot system, and determine the surgical tools currently mounted on the target holding arm 211a, as well as the target surgical tools that need to be replaced to the target holding arm 211a, and control the second robotic arm 61 to replace the surgical tools on the target holding arm 211a. The replacement of surgical tools includes: controlling the second robotic arm 61 to remove the currently mounted surgical tools from the target holding arm 211a, and / or controlling the second robotic arm 61 to grasp the target surgical tools, and controlling the second robotic arm 61 to mount the target surgical tools onto the target holding arm 211a.

[0031] The technical solution provided in this application, after determining the target surgical arm that needs to have its surgical tools replaced, the surgical tools currently installed on the target surgical arm, and the target surgical tools that need to be replaced on the target surgical arm in the surgical robot system, can control the second robotic arm to remove the currently installed surgical tools from the target surgical arm, and control the second robotic arm to grasp the target surgical tools and install them onto the target surgical arm. By using the automatically controlled second robotic arm, the surgical tools on the robotic arm can be automatically replaced during the operation, replacing the manual replacement operation by the doctor next to the patient. This eliminates the need for the doctor to be constantly at the operating table, thereby improving surgical efficiency and reducing labor costs.

[0032] It is understood that the number of first robotic arms 21 available for mounting surgical tools 4 in a surgical robot system can be one or more. For example, in a multi-port surgical robot system, there may be multiple first robotic arms 21, each of which may include a holding arm 211, and each holding arm 211 may mount a surgical tool 4. As an example, referring to FIG4, a multi-port surgical robot system includes multiple first robotic arms 21, each of which includes a holding arm 211. The target holding arm 211a may be the holding arm among the multiple holding arms 211 that needs to be replaced with surgical tool 4 in the current surgical stage. Alternatively, for example, in a single-port surgical robot system, there may be one first robotic arm 21, which may include multiple holding arms 211, and correspondingly, each holding arm 211 may mount a surgical tool 4.

[0033] It is understood that the first robotic arm 21 and the second robotic arm 61 can be set on different bases, that is, the first robotic arm 21 and the second robotic arm 61 can be set on different robots. As shown in Figure 4, the first robotic arm 21 can be set on the first robot 2 and the second robotic arm 61 can be set on the second robot 6. The first robot 2 and the second robot 6 are robots on different bases.

[0034] The second robot 6 can have various specific implementations. As an example, the second robot 6 can be a single-arm robot or a collaborative robot, that is, the second robot 6 includes only one robotic arm (i.e., a second robotic arm 61). As another example, the second robot 6 can also include multiple robotic arms, wherein some or all of the multiple robotic arms can be the aforementioned second robotic arm 61.

[0035] Surgical tools can take many specific forms. As an example, a surgical tool may include surgical instruments or image acquisition devices. As another example, an image acquisition device may include devices such as an endoscope or a laparoscope.

[0036] The control device 11 can be configured in various ways. As an example, the control device 11 can be integrated into the console 1 shown in Figure 1a, or it can be integrated into other devices, such as the first robot 2 shown in Figure 1a, or the second robot 6 to which the second robotic arm 61 belongs. Of course, the implementation of this application is not limited to these.

[0037] There are multiple ways for the control device 11 to determine the target surgical arm 211a, the surgical tool currently installed on the target surgical arm 211a, and the target surgical tool. As an example, taking the control device 11 integrated into the console 1 shown in Figure 1a, the control device 11 can determine the target surgical arm 211a, the surgical tool currently installed on the target surgical arm 211a, and the target surgical tool based on a surgical tool replacement command input by the user. As an example, the user inputting the surgical tool replacement command could be the surgeon at the console 1. It is worth noting that the above description of the methods for determining the target surgical arm 211a, the surgical tool currently installed on the target surgical arm 211a, and the target surgical tool is merely an illustrative example. In practical applications, other determination methods may exist, and no specific limitations are imposed on these.

[0038] As an example, when the surgical robot system includes at least two first robotic arms 21, each first robotic arm 21 including a holding arm 211, the control device 11 can determine the target holding arm 211a from the at least two first robotic arms 21. As another example, when the surgical robot includes one first robotic arm 21, which includes at least two holding arms 211, the control device 11 can determine the target holding arm 211a from the at least two holding arms 211.

[0039] The surgical tool replacement command input by the user can have a variety of specific implementations. As an example, the surgical tool replacement command input by the user may include information about the first robotic arm 21 containing the target robotic arm 211a selected by the user or information about the target robotic arm 211a itself, as well as information about the surgical tool currently installed on the target robotic arm 211a (i.e. the surgical tool that needs to be replaced), and information about the target surgical tool that needs to be replaced to the target robotic arm 211a.

[0040] As an example, a user-inputted instruction to change surgical tools could be a voice command.

[0041] As another example, the console 1 may include a touchscreen for user interaction. This touchscreen can display the various robotic arms included in the surgical robot system, the surgical tools currently installed on each robotic arm, and surgical tools that the user can currently select but are not currently installed on, but which can be replaced by the robotic arms. The user can select the robotic arm from which the surgical tool needs to be replaced and select the surgical tool to be replaced on that robotic arm. The surgical tool replacement command input by the user can be generated based on the user's selection. As another example, the surgical robot system may also include a tray for holding surgical tools. When the user selects a surgical tool to be replaced on the robotic arm on the touchscreen, the control device 11 can confirm whether the selected surgical tool exists on the tray based on the tray's instrument recognition system. If it exists, the surgical tool replacement command is generated; if it does not exist, a prompt indicating that the surgical tool does not exist is generated. It is worth noting that the above description of the specific implementation of the user-input surgical tool replacement command is only an exemplary demonstration. In practical applications, other specific implementations are possible, and no specific limitation is made.

[0042] It is understandable that before the control device 11 controls the second robotic arm 61 to remove the surgical tool currently mounted on the target holding arm 211a from the target holding arm 211a, the position of the second robotic arm 61 may already be relatively close to the target holding arm 211a, and the second robotic arm 61 can remove the surgical tool from the target holding arm 211a without moving; there may also be a certain distance between the position of the second robotic arm 61 and the target holding arm 211a, in which case the second robotic arm 61 needs to move to get closer to the target holding arm 211a in order to remove the surgical tool from the target holding arm 211a.

[0043] Considering the distance between the target robotic arm 211a and the second robotic arm 61, the second robotic arm 61 needs to move closer to the target robotic arm 211a. To address this issue, the control device 11 can control the second robotic arm 61 to move towards the target robotic arm 211a in several ways. As an example, the second robotic arm 61 may include sensors. Based on a pre-constructed three-dimensional map of the surgical space and information collected in real time by the sensors of the second robotic arm 61, the movement path of the second robotic arm 61 relative to the target robotic arm 211a can be determined, and the second robotic arm 61 can be controlled to move towards the target robotic arm 211a according to the determined movement path. The aforementioned three-dimensional map of the surgical space may at least include the pose information of each of the first robotic arms, each surgical tool, and the second robotic arm 61 included in the surgical robot system within that surgical space.

[0044] It is understood that the surgical space can be the indoor space where the various first robotic arms, various surgical tools and the second robotic arm 61 of the surgical robot system are located, as shown in Figure 4. The indoor space may also include elements such as the tray 7, the operating table 8, the patient 9 and medical staff (not shown in the figure).

[0045] The aforementioned three-dimensional map of the surgical space can be constructed in various ways. As an example, it can be based on the three-dimensional models corresponding to the first robotic arm 21, surgical tool 4, and second robotic arm 61 included in the surgical robot system, and based on the information collected in the surgical space by the sensors corresponding to the first robotic arm 21, surgical tool 4, and second robotic arm 61 included in the surgical robot system, to determine the global feature information of the surgical space. The global feature information includes at least the feature information and pose information of the three-dimensional models corresponding to the first robotic arm 21, surgical tool 4, and second robotic arm 61 included in the surgical robot system. Then, based on the global feature information of the surgical space, a three-dimensional map of the surgical space is constructed.

[0046] As an example, the three-dimensional models of the first robotic arm 21, surgical tool 4 and second robotic arm 61 included in the surgical robot system can be pre-generated based on the existing structural information of the first robotic arm 21, surgical tool 4 and second robotic arm 61.

[0047] It is understandable that since the second robotic arm 61 may include several connecting arms, the pose information corresponding to the second robotic arm 61 may include the motion posture information of each connecting arm of the second robotic arm 61, which can be determined in a variety of ways. As an example, among the several connecting arms included in the second robotic arm 61, adjacent connecting arms are connected by joints. The joints can drive the connecting arms to move under the drive of the joint motors. The sensors corresponding to the second robotic arm 61 may include at least the sensors of each joint. The sensors can collect the motion posture information of the connecting arms connected to the joints. Based on the information collected by the sensors of each joint, the motion posture information of each connecting arm of the second robotic arm 61 can be determined.

[0048] In addition to the sensors at each joint, the sensors corresponding to the second robotic arm 61 can also include other types of sensors. As an example, the sensors corresponding to the second robotic arm 61 can also include LiDAR, collision sensors, and vision sensors. Based on the information collected in real time by the sensors at each joint of the second robotic arm 61, LiDAR, collision sensors, and vision sensors in the surgical space, the motion path of the second robotic arm 61 relative to the target holding arm 211a can be determined. This motion path can avoid collisions between the second robotic arm 61 and other objects. Furthermore, the information collected in real time by the aforementioned sensors can also be used to determine the pose information corresponding to the second robotic arm 61. This pose information can be used to determine the aforementioned global feature information of the surgical space.

[0049] It is understood that the sensors (such as lidar, collision sensors, and vision sensors) installed on the second robotic arm 61 can also be installed on the second robot 6 that carries the second robotic arm 61 to achieve the functions described above, such as determining the motion path of the second robot 6 relative to the target robotic arm 211a, which can prevent the second robot 6 from colliding with other objects, and the information collected in real time by each of the above sensors can also be used to determine the pose information corresponding to the second robot 6, which can be used to determine the above-mentioned global feature information of the surgical space.

[0050] The sensors corresponding to the first robotic arm 21 can also be implemented in various ways. As an example, among the several connecting arms included in the first robotic arm 21, adjacent connecting arms are also connected by joints. The joints can drive the connecting arms to move under the drive of the joint motors. The sensors corresponding to the first robotic arm 21 can include at least the sensors of each joint. The sensors can collect the motion posture information of the connecting arms connected to the joints. Based on the information collected by the sensors of each joint, the motion posture information of each connecting arm of the first robotic arm 21 can be determined. The motion posture information can also be included in the above-mentioned pose information corresponding to the first robotic arm 21.

[0051] In addition to the sensors at each joint, the sensors corresponding to the first robotic arm 21 may also include other types of sensors. For example, the sensors corresponding to the first robotic arm 21 may also include LiDAR, collision sensors, and vision sensors. The information collected in real time by the sensors at each joint of the first robotic arm 21, LiDAR, collision sensors, and vision sensors in the surgical space can be used to determine the pose information of the first robotic arm 21. This pose information can be used to determine the aforementioned global feature information of the surgical space.

[0052] It is understood that the sensors (such as lidar, collision sensors, and vision sensors) set on the first robotic arm 21 can also be set on the first robot 2 that carries the first robotic arm 21 to achieve the functions described above. For example, the information collected in real time by each of the above sensors can be used to determine the pose information corresponding to the first robot 2, and the pose information can be used to determine the global feature information of the surgical space.

[0053] Referring to Figure 4, as an example, the surgical space may further include an operating table 8 and a tray 7 for placing surgical instruments. LiDAR, collision sensors, and vision sensors may also be installed on the operating table 8 and the tray 7, respectively. Based on the information collected in real time by the LiDAR, collision sensors, and vision sensors on the operating table and tray within the surgical space, the pose information of the operating table and tray can be determined. This pose information can also be used to determine the aforementioned global feature information of the surgical space. That is, the three-dimensional map of the surgical space constructed based on the global feature information of the surgical space can also include the pose information of the operating table and tray within the surgical space. As another example, the surgical space may also include a patient 9 and medical staff, and the three-dimensional map of the surgical space can also include the pose information of the patient 9 and medical staff within the surgical space.

[0054] There are several ways in which the control device 11 controls the second robotic arm 61 to remove the surgical instruments currently mounted on the target holding arm 211a from the target holding arm 211a. As an example, the surgical robot system may include at least two first robotic arms 21. Among the at least two first robotic arms 21, the control device 11 can control the first robotic arm 21 containing the target holding arm 211a to adjust to the target pose, and under the target pose of the target holding arm 211a, control the second robotic arm 61 to remove the surgical instruments currently mounted on the target holding arm 211a from the target holding arm 211a. The surgical tool is removed from the target holding arm 211a. In this target pose, the distance between the first robotic arm 21 containing the target holding arm 211a and other first robotic arms 21 that do not contain the target holding arm 211a is greater than or equal to a first preset distance. The distance between the surgical tool currently installed on the target holding arm 211a and the top position of the target holding arm 211a is less than or equal to a second preset distance. Similarly, in this target pose, the control device 11 can control the second robotic arm 61 to install the target surgical tool onto the target holding arm 211a. When the first robotic arm 21, including the target holding arm 211a, is in the target pose, it can be as far away from other robotic arms as possible to avoid collisions. Furthermore, the surgical tool currently mounted on the target holding arm 211a is as close as possible to its top position, thus leaving sufficient space between the currently mounted surgical tool and the entry assembly for linear axial removal from the target holding arm 211a. This ensures that both the target holding arm 211a and the currently mounted surgical tool are in a position conducive to removal and installation, thereby reducing the operational complexity of the second robotic arm 61 and further improving the efficiency of surgical tool replacement. As an example, the aforementioned second preset distance can also be 0, meaning the surgical tool currently mounted on the target holding arm 211a is at the top position of its holding arm, thus maximizing the space between the surgical tool entry assemblies. As another example, the number of first robotic arms 21 included in the surgical robot system can also be one. In this case, the first robotic arm 21 can include at least two holding arms 211, and the target holding arm 211a is one of the at least two holding arms 211. The control device 11 can directly control the first robotic arm 21 or the target holding arm 211a to adjust to the target pose, and control the second robotic arm 61 to remove the currently installed surgical tool from the target holding arm 211a in the target pose. In the target pose, the distance between the currently installed surgical tool and the top position of the target holding arm 211a is less than or equal to a second preset distance.

[0055] As an example, before the control device 11 controls the first robotic arm 21 containing the target surgical tool arm 211a to adjust to the target pose, the current pose information of the first robotic arm 21 containing the target surgical tool arm 211a can be recorded. If it is detected that the target surgical tool has been installed on the target surgical tool arm 211a, the first robotic arm 21 containing the target surgical tool arm 211a can be controlled to return to the recorded current pose. After each change of surgical tool, the first robotic arm 21 containing the target surgical tool arm 211a can be controlled to return to the working posture before the change of surgical tool, so as to continue the surgical operation before the change of surgical tool, making the surgical operation process smoother.

[0056] As an example, the control device 11 may control the second robotic arm 61 to remove the surgical tool currently mounted on the target robotic arm 211a from the target robotic arm 211a in the following ways: using the sensors of the second robotic arm 61 to identify feature points of the first robotic arm 21 containing the target robotic arm 211a, and identifying feature points of the surgical tool currently mounted on the target robotic arm 211a; based on the identified target robotic arm 211a, the feature points of the first robotic arm 21 containing the target robotic arm 211a, and the feature points of the surgical tool currently mounted on the target robotic arm 211a, the control device 11 controls the second robotic arm 61 to remove the surgical tool currently mounted on the target robotic arm 211a from the target robotic arm 211a.

[0057] The feature points of the target robotic arm 211a can be implemented in various ways. As an example, the feature points of the target robotic arm 211a can be feature points of key parts of the target robotic arm 211a, such as feature points of each joint of the first robotic arm 21 containing the target robotic arm 211a, or feature points of the target robotic arm 211a itself. Of course, the implementation of this application is not limited to this.

[0058] The feature points of the surgical tool currently mounted on the target robotic arm 211a can be implemented in various ways. As an example, the feature points of the surgical tool currently mounted on the target robotic arm 211a can also be feature points of key parts of the surgical tool, such as key points of the part of the surgical tool that can dock with the second robotic arm 61, or key points that can reflect the orientation of the surgical tool in the axial direction. Of course, the implementation of this application is not limited to these.

[0059] By identifying key feature points of the target robotic arm 211a and the currently installed surgical tools, the posture of the second robotic arm 61 in removing the surgical tools can be optimized to ensure a smooth removal process without jamming or collision.

[0060] There are various ways for the control device 11 to control the second robotic arm 61 to grasp the target surgical tool. As an example, the surgical robot system may also include a tray 7 for placing the surgical tool. Before controlling the second robotic arm 61 to grasp the target surgical tool, the sensor of the second robotic arm 61 can be used to identify the position of the tray 7. Based on the identified position of the tray 7, the second robotic arm 61 is controlled to place the surgical tool currently installed on the target holding arm 211a on the tray 7. Then, the sensor of the second robotic arm 61 is used to identify the feature points of the target surgical tool in the tray 7. Based on the identified feature points of the target surgical tool, the second robotic arm 61 is controlled to grasp the target surgical tool in the tray.

[0061] It is understood that the tray on which the surgical tool currently installed on the target robotic arm 211a is placed and the tray on which the target surgical tool is placed can be the same tray or different trays (i.e., controlling the second robotic arm 61 to place the currently installed surgical tool in one tray and to pick up the target surgical tool from another tray containing the target surgical tool), and there is no specific limitation on this.

[0062] It is understandable that before the control device 11 controls the second robotic arm 61 to place the surgical tool currently installed on the target holding arm 211a onto the tray, the position of the second robotic arm 61 may already be relatively close to the tray, and the second robotic arm 61 can place the surgical tool currently installed on the target holding arm 211a onto the tray without moving; or there may be a certain distance between the position of the second robotic arm 61 and the tray, in which case the second robotic arm 61 needs to move to get closer to the tray in order to place the surgical tool currently installed on the target holding arm 211a onto the tray.

[0063] Considering the distance between the tray and the second robotic arm 61, the second robotic arm 61 needs to move closer to the tray. To address this issue, the control device 11 can control the second robotic arm 61 to move towards the tray in several ways. As an example, based on the pre-constructed 3D map of the surgical space and the information collected in real time by the sensors of the second robotic arm 61 within that surgical space, the movement path of the second robotic arm 61 relative to the tray can be determined, and the second robotic arm 61 can be controlled to move towards the tray based on the determined movement path.

[0064] Considering that after the control device 11 controls the second robotic arm 61 to grasp the target surgical tool from the tray, there may be a certain distance between the second robotic arm 61 and the target holding arm 211a, the second robotic arm 61 needs to move again to get closer to the target holding arm 211a, thereby installing the target surgical tool onto the target holding arm 211a. To address this issue, as an example, the movement path of the second robotic arm 61 relative to the target holding arm 211a can still be determined based on the pre-constructed 3D map of the surgical space and the information collected in real time by the sensors of the second robotic arm 61 in the surgical space. Based on the determined movement path, the second robotic arm 61 can be controlled to move to the target holding arm 211a.

[0065] It is understood that the process of the second robotic arm 61 moving as described above can be the process of the second robotic arm 61 moving on its own, and / or the process of the second robot 6 moving to drive the second robotic arm 61 to move, and there is no specific limitation on this.

[0066] Referring to Figure 5, as an example, the surgical tool (surgical tool 4, the target surgical tool in surgical tool 4, and the currently installed surgical tool in surgical tool 4) described in any of the embodiments above has a portion capable of docking with the second robotic arm 61. This docking portion can be an interface for docking with the second robotic arm 61, or a portion for the gripper of the second robotic arm 61 to grasp the surgical tool. Correspondingly, controlling the second robotic arm 61 to remove the currently installed surgical tool can be either controlling the second robotic arm 61 to dock with the currently installed surgical tool through the interface to remove the currently installed surgical tool, or controlling the second robotic arm 61 to grasp the currently installed surgical tool through its gripper to remove the currently installed surgical tool. Similarly, controlling the second robotic arm 61 to grasp the target surgical tool can be either controlling the second robotic arm 61 to dock with the target surgical tool through the interface to grasp the target surgical tool, or controlling the second robotic arm 61 to grasp the target surgical tool through its gripper to grasp the target surgical tool.

[0067] There are several ways in which the control device 11 controls the second robotic arm 61 to mount the target surgical tool onto the target holding arm 211a. As an example, the sensors of the second robotic arm 61 can be used to identify feature points of the target holding arm 211a and the first robotic arm 21 containing the target holding arm 211a. Based on the identified feature points of the target holding arm 211a and the first robotic arm 21 containing the target holding arm 211a, the second robotic arm 61 is controlled to mount the target surgical tool onto the target holding arm 211a. As an example, the feature points of the target holding arm 211a may include feature points at the interface position between the target holding arm 211a and the surgical tool, as well as feature points at each joint that can reflect the posture of the first robotic arm 21 containing the target holding arm 211a.

[0068] As an example, if the control device 11 detects an obstacle in the movement path of the second robotic arm 61 based on information collected by the sensors of the second robotic arm 61, it controls the second robotic arm 61 to stop moving, or controls the second robotic arm 61 to change its current movement path. By monitoring the dynamic changes in the surgical space in real time through the sensors of the second robotic arm 61, the control device 11 adjusts the movements of the second robotic arm 61 in a timely manner to avoid interference from obstacles.

[0069] As an example, a display device in the surgical robot system can be used to display at least the pose information of the second robotic arm 61, and the pose information of any obstacles in the movement path of the second robotic arm 61. As an example, the display device can also display the pose information of other objects in the surgical space (such as the first robot 2, the first robotic arm 21, the tray 7, the operating table 8, the patient 9, the second robot 6, etc.). As an example, the display device can be a display device on the control console 1, allowing the surgeon next to the control console 1 to monitor the status of the second robotic arm 61 and other objects in the surgical space in real time.

[0070] The first robot 2 described in any of the above embodiments can be of various types. As an example, the first robot 2 can be a single-port surgical robot. As another example, the first robot 2 can also be a multi-port surgical robot. Therefore, the specific type of the first robot 2 is not limited.

[0071] Corresponding to the above-described embodiments of the surgical robot system, this application also provides a method for replacing surgical tools in a surgical robot system. The surgical robot system includes at least one first robotic arm and a second robotic arm. The first robotic arm includes a holding arm, and the surgical tool is detachably mounted on the holding arm and can perform surgical operations based on the control of the first robotic arm. Referring to Figure 6, the method includes the following steps:

[0072] S601. In the surgical robot system, determine the target surgical arm that needs to have its surgical tools replaced, and determine the surgical tools currently installed on the target surgical arm, as well as the target surgical tools that need to be replaced to the target surgical arm.

[0073] S602, Control the second robotic arm to change the surgical tools of the target robotic arm.

[0074] The replacement of surgical instruments includes at least one of the following: controlling a second robotic arm to remove the currently installed surgical instrument from the target holding arm, and / or controlling the second robotic arm to grasp the target surgical instrument and install the target surgical instrument onto the target holding arm.

[0075] As an example, the second robotic arm includes sensors, and the method further includes: determining the motion path of the second robotic arm relative to the target holding arm based on a pre-constructed three-dimensional map of the surgical space and information collected in real time by the sensors of the second robotic arm, and controlling the second robotic arm to move toward the target holding arm according to the motion path; wherein, the three-dimensional map of the surgical space includes at least the pose information of the first robotic arm and the second robotic arm in the surgical space.

[0076] As an example, the three-dimensional map of the surgical space is constructed in the following way: based at least on the three-dimensional models corresponding to the first robotic arm and the second robotic arm, and based on the information collected in the surgical space by the sensors corresponding to the first robotic arm and the second robotic arm, the global feature information of the surgical space is determined; wherein, the global feature information includes at least the feature information and pose information of the three-dimensional models corresponding to the first robotic arm and the second robotic arm; based on the global feature information of the surgical space, the three-dimensional map of the surgical space is constructed.

[0077] As an example, a surgical robot system includes at least two first robotic arms, each of which includes a holding arm. Determining the target holding arm that requires replacement of surgical tools in the surgical robot system includes: determining the target holding arm from at least two first robotic arms.

[0078] As an example, before controlling the second robotic arm to replace the surgical tool on the target robotic arm, the method further includes: controlling the first robotic arm containing the target robotic arm to adjust to a target pose; wherein, in the target pose, the distance between the first robotic arm containing the target robotic arm and the first robotic arm not containing the target robotic arm is greater than or equal to a first preset distance, and the distance between the currently installed surgical tool and the top position of the target robotic arm is less than or equal to a second preset distance.

[0079] As an example, a surgical robot system includes a first robotic arm, the first robotic arm including at least two holding arms, and determining the target holding arm that needs to be replaced in the surgical robot system includes: determining the target holding arm from at least two holding arms.

[0080] As an example, before controlling the second robotic arm to replace the surgical tool on the target holding arm, the method further includes: controlling the first robotic arm or the target holding arm to adjust to a target pose; wherein, in the target pose, the distance between the currently installed surgical tool and the top position of the target holding arm is less than or equal to a second preset distance.

[0081] As an example, the method further includes: recording information about the current pose of the first robotic arm containing the target robotic arm before controlling the first robotic arm containing the target robotic arm to adjust to the target pose; if the target surgical tool is detected to be installed on the target robotic arm, controlling the first robotic arm containing the target robotic arm to return to the current pose.

[0082] As an example, controlling the second robotic arm to remove the currently installed surgical tool from the target robotic arm includes: using the sensors of the second robotic arm to identify feature points of the target robotic arm and the first robotic arm containing the target robotic arm, and identifying feature points of the currently installed surgical tool; based on the identified feature points of the target robotic arm and the first robotic arm containing the target robotic arm, and the feature points of the currently installed surgical tool, controlling the second robotic arm to remove the currently installed surgical tool from the target robotic arm.

[0083] As an example, the surgical robot system also includes a tray for holding surgical instruments; before controlling the second robotic arm to acquire the target surgical instrument, the method further includes: using sensors of the second robotic arm to identify the pose of the tray, and based on the identified pose of the tray, controlling the second robotic arm to place the currently mounted surgical instrument on the tray; controlling the second robotic arm to grasp the target surgical instrument includes: using sensors of the second robotic arm to identify feature points of the target surgical instrument in the tray; and based on the identified feature points of the target surgical instrument, controlling the second robotic arm to grasp the target surgical instrument in the tray.

[0084] As an example, the method further includes: if an obstacle is detected in the movement path of the second robotic arm based on information collected by the sensors of the second robotic arm, then the second robotic arm is controlled to stop moving, or the second robotic arm is controlled to change its current movement path.

[0085] As an example, the surgical robot system also includes a display device, which is used to display at least the pose information of the second robotic arm, and the pose information of the obstacle when an obstacle is present.

[0086] This application also provides an electronic device, as shown in FIG7, which includes:

[0087] Processor 701;

[0088] Memory 702 is used to store processor-executable instructions;

[0089] The processor 701 is configured to implement the surgical tool replacement method described in any of the embodiments above.

[0090] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the surgical tool replacement method described in any of the embodiments above.

[0091] The above description is only a specific embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for replacing surgical tools in a surgical robot system, the surgical robot system comprising at least one first robotic arm and a second robotic arm, the first robotic arm including a holding arm, the surgical tool being detachably mounted on the holding arm and capable of performing surgical operations based on control of the first robotic arm, the method comprising: In the surgical robot system, a target surgical arm that requires replacement of surgical tools is identified, and the surgical tools currently installed on the target surgical arm and the target surgical tools that need to be replaced to the target surgical arm are also identified. The second robotic arm is controlled to change the surgical tools on the target robotic arm, and the replacement of the surgical tools includes at least one of the following: Control the second robotic arm to remove the currently mounted surgical tool from the target robotic arm; and / or, The second robotic arm is controlled to grasp the target surgical tool and install the target surgical tool onto the target holding arm.

2. The method of claim 1, wherein, The second robotic arm includes sensors, and the method further includes: Based on a pre-constructed 3D map of the surgical space and information collected in real time by the sensors of the second robotic arm, the motion path of the second robotic arm relative to the target holding arm is determined, and the second robotic arm is controlled to move toward the target holding arm according to the motion path. The three-dimensional map of the surgical space includes at least the pose information of the first robotic arm and the second robotic arm in the surgical space.

3. The method of claim 2, wherein, The three-dimensional map of the surgical space is constructed in the following way: Based at least on the three-dimensional models corresponding to the first robotic arm and the second robotic arm respectively, and based on the information collected in the surgical space by the sensors corresponding to the first robotic arm and the second robotic arm respectively, global feature information of the surgical space is determined; wherein, the global feature information includes at least feature information and pose information of the three-dimensional models corresponding to the first robotic arm and the second robotic arm respectively; A three-dimensional map of the surgical space is constructed based on the global feature information of the surgical space.

4. The surgical tool replacement method according to any one of claims 1 to 3, wherein The surgical robot system includes at least two first robotic arms, each first robotic arm including a holding arm, wherein determining the target holding arm requiring replacement of surgical tools in the surgical robot system includes: The target holding arm is determined from at least two of the first robotic arms.

5. The method of replacing a surgical tool according to claim 4, wherein, Before controlling the second robotic arm to change the surgical tool on the target robotic arm, the method further includes: Control the first robotic arm, which includes the target holding arm, to adjust to the target pose; In the target pose, the distance between the first robotic arm containing the target holding arm and the first robotic arm not containing the target holding arm is greater than or equal to a first preset distance, and the distance between the currently installed surgical tool and the top position of the target holding arm is less than or equal to a second preset distance.

6. The surgical tool replacement method according to any one of claims 1 to 3, wherein The surgical robot system includes a first robotic arm, the first robotic arm including at least two holding arms, wherein determining the target holding arm requiring replacement of surgical tools in the surgical robot system includes: The target arm is determined from the at least two arm-holding arms.

7. The method of claim 6, wherein, Before controlling the second robotic arm to change the surgical tool on the target robotic arm, the method further includes: Control the first robotic arm or the target holding arm to adjust to the target pose; In the target pose, the top position of the currently installed surgical tool is less than or equal to a second preset distance from the target surgical arm.

8. The method of claim 5 or 7, wherein, The method further includes: Before controlling the first robotic arm containing the target holding arm to adjust to the target pose, record the current pose information of the first robotic arm containing the target holding arm; If the target surgical tool is detected to be installed on the target holding arm, the first robotic arm containing the target holding arm is controlled to return to the current pose.

9. The method according to any one of claims 1 to 8, wherein, The control of the second robotic arm to remove the currently mounted surgical tool from the target robotic arm includes: The second robotic arm's sensors are used to identify feature points of the target robotic arm and the first robotic arm containing the target robotic arm, and feature points of the currently installed surgical tool are also identified. Based on the identified feature points of the target robotic arm and the first robotic arm containing the target robotic arm, as well as the feature points of the currently installed surgical tool, the second robotic arm is controlled to remove the currently installed surgical tool from the target robotic arm.

10. The method of any one of claims 1-9, wherein, The surgical robot system further includes a tray for holding surgical instruments; the method further includes, prior to controlling the second robotic arm to grasp the target surgical instrument: The second robotic arm uses sensors to identify the position and orientation of the tray, and based on the identified position and orientation of the tray, controls the second robotic arm to place the currently installed surgical tool onto the tray; The control of the second robotic arm to grasp the target surgical tool includes: The sensor of the second robotic arm identifies the feature points of the target surgical tool in the tray; Based on the identified feature points of the target surgical tool, the second robotic arm is controlled to grasp the target surgical tool from the tray.

11. The method of any one of claims 2-10, wherein, The method further includes: If, based on information collected by the sensors of the second robotic arm, an obstacle is detected in the movement path of the second robotic arm, then the second robotic arm is controlled to stop moving, or the second robotic arm is controlled to change its current movement path.

12. The method of claim 11, wherein, The surgical robot system also includes a display device, which is used to display at least the pose information of the second robotic arm and the pose information of the obstacle when the obstacle is present.

13. A surgical robot system, comprising: A first robotic arm, capable of mounting surgical tools and responding to a master control device to control the surgical tools to perform surgical operations; A second robotic arm is configured to grasp surgical instruments mounted on or remove surgical instruments from the first robotic arm.

14. The surgical robotic system of claim 13, wherein, The surgical robot system includes: A first robot, wherein the first robotic arm is mounted on the first robot; The second robot, with the second robotic arm mounted on it.

15. The surgical robot system according to claim 13 or 14, wherein, The surgical robot system includes at least two first robotic arms, each first robotic arm including a holding arm, on which surgical instruments are mounted, or... The surgical robot system includes a first robotic arm, which includes at least two holding arms, each of which can be fitted with the surgical tool.

16. The surgical robot system according to any one of claims 13 to 15, wherein, The surgical robot system also includes: A control device is configured to determine, within the surgical robot system, a target surgical arm for which a surgical tool needs to be replaced, and to determine the surgical tool currently mounted on the target surgical arm, and a target surgical tool to be replaced onto the target surgical arm; and to control a second robotic arm to replace the surgical tool on the target surgical arm, wherein the replacement of the surgical tool includes at least one of the following: controlling the second robotic arm to remove the currently mounted surgical tool from the target surgical arm, and / or controlling the second robotic arm to grasp the target surgical tool and install the target surgical tool onto the target surgical arm.

17. An electronic device comprising: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1 to 12.

18. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any one of claims 1 to 12.

Images