Computer-readable storage medium, electronic device, and surgical robot system

Abstract

The present invention provides a computer-readable storage medium, an electronic device, and a surgical robot system. The computer-readable storage medium stores a program that, when executed, performs the following steps: establishing a three-dimensional model of the surgical area based on first image information of the surgical area; acquiring second image information of a local area of the surgical area; and planning the motion path of the surgical instrument based on the three-dimensional model, the second image information, the initial position, and the target position of the surgical instrument. The computer-readable storage medium is applied to surgical operations performed by the surgical robot system, which combines the first image information of the surgical area with the real-time second image information captured by an endoscope to plan the motion path of the surgical instrument, thereby reducing the reliance on the operator's experience and operating skills and improving the controllability and safety of the surgery.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and in particular to a computer-readable storage medium, an electronic device, and a surgical robot system. Background Art

[0002] During surgery using an endoscopic surgical robot, the operator interprets the image feed from the endoscope and controls the robot to perform the surgical procedure. Due to the limited viewing angle of the endoscope, the operator must constantly adjust the endoscope's position to maximize the viewing angle and use their experience to determine how to manipulate the surgical instruments.

[0003] That is to say, in the prior art, in surgeries performed using surgical robots and endoscopes, the execution of the surgical operation is highly dependent on the operator's personal experience and operating status, resulting in insufficient safety of the surgery. Summary of the Invention

[0004] The purpose of the present invention is to provide a computer-readable storage medium, an electronic device and a surgical robot system, aiming to reduce the dependence of surgery performed using a surgical robot and an image acquisition device on the operator's personal experience and improve surgical safety.

[0005] To achieve the above object, the present invention provides a computer-readable storage medium having a program stored thereon. When the program is executed, the following steps are performed:

[0006] establishing a three-dimensional model of the surgical area according to the first image information of the surgical area;

[0007] acquiring second image information of a local area of the surgical area;

[0008] The motion path of the surgical instrument is planned according to the three-dimensional model, the second image information, the initial position and target position of the surgical instrument.

[0009] Optionally, the step of planning the motion path of the surgical instrument according to the three-dimensional model, the second image information, the initial posture and the target posture of the surgical instrument includes:

[0010] planning a global motion path of the surgical instrument according to the three-dimensional model, the initial position and the target position of the surgical instrument;

[0011] determining a target environment in the three-dimensional model that matches the second image information;

[0012] The portion of the global motion path located within the target environment is corrected according to the second image information, the target environment, and the surrounding environment of the target environment to obtain a target motion path.

[0013] Optionally, the global motion path includes all expected poses of the surgical instrument between the initial pose and the target pose.

[0014] Optionally, the second image information displays the surgical instrument and tissue;

[0015] The step of correcting the portion of the global motion path located within the target environment according to the second image information, the target environment, and the surrounding environment of the target environment comprises:

[0016] acquiring a relative positional relationship between the surgical instrument and the tissue according to the second image information;

[0017] Based on the relative positional relationship between the surgical instrument and the tissue, the target environment and the surrounding environment of the target environment, it is determined whether the surgical instrument can move according to all the expected postures within the target environment. If not, at least part of the expected postures within the target environment are corrected.

[0018] Optionally, the expected posture includes an expected position and an expected posture; and correcting at least part of the expected posture in the target environment includes correcting at least part of the expected position and / or at least part of the expected posture in the target environment.

[0019] Optionally, the program further performs the following steps:

[0020] The motion trajectory of the surgical instrument is planned according to the motion path.

[0021] Optionally, the surgical instrument is driven to move along the motion trajectory by a tool arm of a surgical robot system;

[0022] When planning the motion trajectory of the surgical instrument, at least one of the working space of the tool arm, the dynamics of the tool arm, and the transmission chain performance of the robot is used as a constraint condition.

[0023] Optionally, the surgical instrument is connected to the end of a tool arm of a surgical robot system; and the program further performs the following steps:

[0024] The tool arm is controlled to move so as to drive the surgical instrument to move along the motion path.

[0025] Optionally, the program further performs the following steps:

[0026] The motion path is sent to a display device for display.

[0027] Optionally, the program plans more than two motion paths; the program further performs the following steps: generating prompt information to prompt the user to perform an intervention operation to select a path.

[0028] To achieve the above object, the present invention further provides an electronic device, comprising a processor and the computer-readable storage medium as described above, wherein the processor is configured to execute a program stored on the computer-readable storage medium.

[0029] To achieve the above objectives, the present invention further provides a surgical robot system, comprising:

[0030] an imaging arm, a distal end of which is connected to an image acquisition device, the image acquisition device being used to acquire second image information of a local area of the surgical area; and

[0031] A control unit is communicatively connected to the image acquisition device to receive the second image information, and the control unit is further configured to execute the program stored on the computer-readable storage medium as described above.

[0032] Optionally, the surgical robot system further includes a tool arm, the end of which is connected to a surgical instrument; the tool arm is communicatively connected to the control unit, and the tool arm moves under the control of the control unit to drive the surgical instrument to move along the motion path.

[0033] Compared with the prior art, the computer-readable storage medium, electronic device, and surgical robot system of the present invention have the following advantages:

[0034] The aforementioned computer-readable storage medium stores a program, and when the program is executed, the following steps are performed: establishing a three-dimensional model of the surgical area based on the first image information of the surgical area; acquiring second image information of a local area of the surgical area; and planning the motion path of the surgical instrument based on the three-dimensional model, the second image information, the initial position and target position of the surgical instrument. Here, the second image information can be acquired by an image acquisition device such as an endoscope that extends into the surgical area. When the computer-readable storage medium is applied to surgical operations performed by a surgical robot system, the motion path of the surgical instrument can be planned in combination with the first image information of the surgical area and the real-time second image information acquired by the endoscope, rather than deciding how to operate the surgical instrument based on the surgical field of view provided by the endoscope and the operator's experience and operating level. This reduces the dependence on the operator's experience and operating level, and improves the controllability and safety of the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings are provided for a better understanding of the present invention and are not intended to limit the present invention.

[0036] Figure 1 is a schematic diagram of an application scenario of a surgical robot system provided according to one embodiment of the present invention;

[0037] Figure 2 is a schematic diagram of a doctor-side control device of a surgical robot system according to one embodiment of the present invention;

[0038] Figure 3 is a schematic diagram of a fixed image display device of a surgical robot system according to one embodiment of the present invention;

[0039] Figure 4 is a schematic diagram of a surgical operating device of a surgical robot system according to one embodiment of the present invention;

[0040] Figure 5 is a schematic structural diagram of a surgical instrument connected to a surgical robot system provided according to one embodiment of the present invention;

[0041] Figure 6 yes Figure 5 An enlarged schematic diagram of point A of a surgical instrument connected to the surgical robot system is shown;

[0042] Figure 7 This is an overall flow chart of a control unit of a surgical robot system planning a motion path of a surgical instrument according to one embodiment of the present invention;

[0043] Figure 8 2 is a schematic diagram of a method for acquiring first image information used by a control unit of a surgical robot system provided according to an embodiment of the present invention;

[0044] Figure 9 is a schematic diagram of a method for acquiring second image information used by a surgical robot system provided according to one embodiment of the present invention;

[0045] Figure 10 This is a schematic diagram of a control unit of a surgical robot system according to one embodiment of the present invention acquiring an operating area, a disease location, and performing global motion path planning;

[0046] Figure 11 This is a flow chart of a control unit of a surgical robot system according to one embodiment of the present invention performing path planning using an RRT algorithm;

[0047] Figure 12 2. It is a schematic diagram of a control unit of a surgical robot system according to one embodiment of the present invention using an RRT algorithm for path planning;

[0048] Figure 13is a flow chart of a control unit of a surgical robot system according to an embodiment of the present invention correcting a portion of a global motion path located within the target environment;

[0049] Figure 14 is a schematic diagram of a control unit of a surgical robot system planning a motion trajectory of a surgical instrument according to one embodiment of the present invention;

[0050] Figure 15 is a schematic diagram of a surgical instrument of a surgical robot system according to an embodiment of the present invention moving along a planned target path, wherein a control unit plans a target path;

[0051] Figure 16 is a schematic diagram of target paths planned by a control unit of a surgical robot system according to one embodiment of the present invention, wherein two target paths are shown, and the surgical instrument will move along the target path with the shortest time;

[0052] Figure 17 2 is a schematic diagram of a target mold planned by a control unit of a surgical robot system according to an embodiment of the present invention, wherein two target paths are shown, and the surgical instrument will move along the safest target path;

[0053] Figure 18 Schematic diagram of the interaction between a surgical robot system and a surgeon according to one embodiment of the present invention;

[0054] Figure 19 This is a schematic diagram of a surgical robot system provided according to one embodiment of the present invention guiding the movement of surgical instruments during use. DETAILED DESCRIPTION

[0055] The following describes the embodiments of the present invention through specific specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in this embodiment only illustrate the basic concept of the present invention in a schematic manner, and the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components during actual implementation. During actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0056] In addition, each embodiment described below has one or more technical features. However, this does not mean that users of the present invention must implement all technical features in any embodiment at the same time, or that they can only implement some or all technical features in different embodiments separately. In other words, as long as implementation is possible, those skilled in the art can, based on the disclosure of the present invention and depending on design specifications or implementation requirements, selectively implement some or all technical features in any embodiment, or selectively implement a combination of some or all technical features in multiple embodiments, thereby increasing the flexibility of the implementation of the present invention.

[0057] As used in this specification, the singular forms "a", "an", and "the" include plural objects, and the plural form "a plurality" includes more than two objects, unless the content clearly indicates otherwise. As used in this specification, the term "or" is generally used in a sense including "and / or", unless the content clearly indicates otherwise, and the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection. It can be a mechanical connection or an electrical connection. It can be directly connected or indirectly connected through an intermediate medium, and it can be a connection between the internal parts of two elements or an interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0058] To make the objects, advantages, and features of the present invention more apparent, the present invention is further described below in detail with reference to the accompanying drawings. It should be noted that the drawings are greatly simplified and not to exact scale, and are intended solely to facilitate and clearly illustrate the embodiments of the present invention. The same or similar reference numerals in the drawings represent the same or similar components.

[0059] Figure 1 shows a schematic diagram of the application scenario of the surgical robot system, Figure 2 、 Figure 3 and Figure 4 Schematic diagrams showing some components of the surgical robot system are shown respectively. Figures 1 to 4As shown, the surgical robot system includes a control end and an execution end, and the control end includes a doctor-end control device 10. The execution end includes a patient-end control device 20, a surgical operation device 30, a display device 40 and other equipment. Among them, the patient-end control device 20 can be integrated with the surgical operation device 30. An image arm 31 and a tool arm 32 are mounted on the surgical operation device. The image arm 31 is used to mount a first image acquisition device, and the first image acquisition device is used to acquire image information inside the patient's body (such as the second image information described later), and the first image acquisition device is, for example, an endoscope 50. The tool arm 32 is used to mount a surgical instrument 60, and the surgical instrument 60 is used to extend into the surgical area inside the patient's body to perform surgical operations. The embodiment of the present invention does not particularly limit the specific type of the surgical instrument 60, which can be such as Figure 5 and Figure 6 The clamp assembly shown may also be other surgical instruments. The doctor-side control device 10 may include an immersive display device 11, and the immersive display device 11 and the display device 40 may be communicatively connected to the endoscope 50 to receive and display the image information (such as the second image information) collected by the endoscope 50. In addition, a main operator (not shown in the figure) is also provided on the doctor-side control device 10, and a predetermined mapping relationship is formed between the main operator and the tool arm 32 and the surgical instrument 60, so that the tool arm 32 and the surgical instrument 60 can follow the movement of the main operator.

[0060] The surgical robot system further includes a control unit (not shown in the figure). When the surgical robot system and the endoscope 50 are used in conjunction to perform a surgical operation, the control unit is configured to execute a path planning method for a surgical instrument to guide the movement of the surgical instrument 60 within the surgical area. The embodiment of the present invention does not limit the specific arrangement of the control unit. The control unit can be provided as a whole at the patient-side control device 20, or as a whole at the doctor-side control device 10, or partially at the patient-side control device 20 and another part at the doctor-side control device 10, or completely independent of the doctor-side control device 10 and the patient-side control device 20, as long as it can achieve the corresponding functions.

[0061] like Figure 7 As shown, the path planning method of the surgical instrument includes the following steps:

[0062] Step S10: establishing a three-dimensional model of the operating area according to the first image information of the operating area.

[0063] Step S20: Acquire second image information of a local area of the surgical area.

[0064] And, step S30: planning the motion path of the surgical instrument 60 according to the three-dimensional model, the second image information, the initial posture and the target posture of the surgical instrument 60.

[0065] Thereafter, the surgical instrument 60 may be controlled to move along the motion path.

[0066] The surgical area is determined according to the specific surgery. For example, in laparoscopic surgery, the surgical area is the abdominal cavity, and in thoracoscopic surgery, the surgical area is the thoracic cavity. The following text takes laparoscopic surgery as an example, that is, the surgical area is the abdominal cavity. The first image information can be a medical image obtained by the second image acquisition device 70 before the surgery. The second image acquisition device 70 is, for example, a CT (such as Figure 8 The second image information is generally obtained during the operation by the endoscope 50 inserted into the operation area (as shown), MRI, B-ultrasound, etc. Figure 9 shown).

[0067] In an embodiment of the present invention, the movement path of the surgical instrument 60 is planned by using the first image information of the surgical area before the operation and the real-time second image information collected during the operation, rather than deciding how to operate the surgical instrument 60 based on the surgical field of view provided by the endoscope 50 and the operator's experience and operation level. This reduces the dependence on the operator's experience and operation level and improves the controllability and safety of the operation.

[0068] Next, this article describes in detail the various steps of the path planning method for the surgical instrument.

[0069] Please refer to Figure 10In step S10, the first image information is subjected to image recognition by AI image recognition technology, thereby establishing the three-dimensional model. Specific AI image recognition technologies that can be used include, but are not limited to, neural network algorithms, KNN algorithms (K nearest neighbor algorithms), and the like. In a specific embodiment, the KNN algorithm can be used to perform step S10. In the process of establishing the three-dimensional model, step S11 can be first performed: the surgical area is found by image classification, and then step S12 can be performed: the lesion location is determined by target detection, and the three-dimensional model is established accordingly. Those skilled in the art will appreciate that when performing image recognition using the KNN algorithm, it is necessary to determine training samples, samples to be classified, sample distances, and K values, and adjust the K value according to the recognition accuracy, which refers to the correct probability of identifying objects within the surgical area during image recognition. For those skilled in the art, determining training samples, samples to be classified, sample distances, and K values, and adjusting the K value according to the recognition accuracy are all well-known contents and will not be described in detail here. In addition, in this embodiment, the sample distance is preferably the Euclidean distance, that is, the sample A[a0, a1, a2, ... a] of the basic image features during image recognition. n ] and sample B[b0,b1,b2,…b n The distance d between (A,B) satisfy:

[0070] It should be noted that the first image information obtained before the operation can be pre-stored in the control unit, or transmitted to the control unit via a wired or wireless manner, or manually input into the control unit by the operator.

[0071] As previously mentioned, the second image information in step S20 is captured by the endoscope 50. The endoscope 50 has a limited viewing angle, so during the surgical procedure, the operator must adjust the endoscope 50's orientation multiple times based on actual conditions to capture second image information of different local areas for observation of the patient's internal conditions. The second image information may display partial tissue within the abdominal cavity and may also display the surgical instrument 60 used to perform the procedure.

[0072] Please continue to refer to Figure 7 The step S30 includes a step S31: planning a global motion path of the surgical instrument 60 according to the three-dimensional model, the initial posture and the target posture of the surgical instrument 60.

[0073] The initial position and target position of the surgical instrument 60 can be determined according to actual conditions. It can be understood that the position includes position and posture, so the initial position includes the initial position and initial posture, and the target position includes the target position and target posture. In some specific embodiments, the initial position is, for example, the position of the hole on the patient's body surface, and the initial posture may refer to the state in which the surgical instrument 60 does not touch the tissue. The target position is, for example, the position of the lesion, and the target posture may refer to the posture in which the surgical instrument 60 does not touch the tissue and the lesion. In addition, the global motion path should be within the constrained space, and the constrained space refers to the space where the surgical area is located, which is the abdominal cavity in this embodiment.

[0074] Please refer to Figure 10 The global motion path is planned using the AI path planning method. Optional planning methods include but are not limited to graph search method, RRT algorithm (rapidly expanding random tree algorithm), artificial potential field method, BUG algorithm, etc. In a non-limiting embodiment, the RRT algorithm is used to plan the global motion path. The flow chart is as follows: Figure 11 As shown, the following steps are included:

[0075] Step S311: Determine the starting position point Xinit of the surgical instrument 60 and add it to the search tree 100 (e.g. Figure 12 As shown in FIG, 1 is used as the initial value for the search. It is understood that the starting pose point Xinit can be a random value.

[0076] Step S312 : generating random sampling points Xrand in a free space, where the free space refers to the motion space of the tool arm 32 .

[0077] Step S313: Find the node Xnwar closest to the sampling point Xrand from the search tree 100.

[0078] Step S314: Calculate the distance Dis between the sampling point Xrand and the node Xnear.

[0079] Step S315: Determine whether the distance Dis between the sampling point Xrand and the node Xnear is greater than the step length u. If so, execute steps S316 and S317. If not, execute step S319. In this step, the distance Dis can be either Euclidean distance or Manhattan distance. When it is Manhattan distance, the calculation method is: Dis Manhattan=|x1-x2|+|y1-y2|+|z1-z2|, where (x1, y1, z1) are the coordinates of the sampling point Xrand, and (x2, y2, z2) are the coordinates of the node Xnear. The step size u can be a fixed value or can be changed according to actual needs, which is not limited in this embodiment of the present invention.

[0080] Step S316: moving a step length u from the node Xnear to the sampling point Xrand to obtain a new node Xnew.

[0081] Step S317: Determine whether there is a straight path between the node Xnew and the node Xnear. If so, execute step S318; if not, return to step S313 and start the next cycle.

[0082] Step S318: Add the node Xnew to the search tree 100. The parent node of the node Xnew is Xnear.

[0083] Step S319: Generate a new node Xnew at the node Xrand. The new node can be used for calculation in the next cycle. The next cycle starts from step S313.

[0084] The global motion path is planned based on a three-dimensional model derived from preoperative medical imaging. Theoretically, during actual surgery, if the surgical area remains consistent during surgery, the surgical instrument 60 can move along the global motion path to reach the target position without touching tissue or lesions during the entire movement process. However, in practice, various factors, such as positioning, establishment of pneumoperitoneum, and even the patient's breathing, often cause the surgical environment to change from preoperatively. This makes it very likely that the surgical instrument 60 will touch tissue and / or lesions when moving along the global motion path. In view of this, step S30 also includes step S32, which is to correct the global motion path based on the second image information. By executing step S32, the global motion path is corrected based on the actual situation of the surgical area during surgery, preventing the surgical instrument 60 from touching tissue and / or lesions during movement.

[0085] Optionally, step S32 specifically includes: first determining the target environment in the three-dimensional model that matches the second image information through alignment or any other suitable method, and then correcting the part of the global motion path located in the target environment based on the second image information, the target environment and the surrounding environment of the target environment to obtain the target motion path (that is, the target motion path is the corrected global motion path).

[0086] It is understood that the tissue model in the target environment is actually at least a portion of the model of the tissue displayed in the second image information. Thus, after determining the target environment, the local path of the global motion path at the tissue displayed in the second image information can be determined.

[0087] It can also be understood that the global path is actually composed of all expected positions of the surgical instrument 60 between the initial position and the target position. When modifying the global path, any suitable AI path planning method such as the RRT algorithm can be used.

[0088] like Figure 13 As shown, the “correcting the portion of the global motion path located within the target environment based on the second image information, the target environment, and the surrounding environment of the target environment” specifically includes:

[0089] Step S321: Acquire the relative position relationship between the surgical instrument 60 and the tissue according to the second image information.

[0090] Step S322: Determine whether the surgical instrument 60 can move according to all the expected postures in the target environment based on the relative position of the surgical instrument 60 and the tissue, the target environment, and the surrounding environment of the target environment. If not, execute step S323. If so, determine that the global motion path is the executable target motion path in the target environment. That is, when it is determined that the surgical instrument 60 can move according to all the expected postures in the target environment, the correction amount for the global motion path is zero. In this step, if the surgical instrument 60 does not touch the tissue when moving along all the expected postures in the target environment, it is considered that the surgical instrument can move according to all the expected postures in the target environment. Otherwise, it is considered that the surgical instrument 60 cannot move according to all the expected postures in the target environment.

[0091] Step S323: Correcting at least part of the expected posture in the target environment to obtain the target motion path. The expected posture includes an expected position and an expected posture. In this step, correcting at least part of the expected posture includes correcting at least part of the expected position and / or at least part of the expected posture.

[0092] Please continue to refer to Figure 13In an exemplary embodiment, step S322 includes steps S3221 and S3222, and step S323 includes steps S3231 and S3232. Step S3221 comprises determining whether the surgical instrument 60 can reach all the desired positions in the target environment. Step S3222 comprises determining whether the surgical instrument 60 can assume the corresponding desired posture at any desired position. Step S3231 comprises correcting at least some of the desired positions. Step S3232 comprises correcting at least some of the desired postures. It is understood that if the judgment result of step S3221 is "yes," step S3231 is not executed. If the judgment result of step S3221 is "no," step S3231 is executed, and then step S3221 is executed again. Similarly, when the judgment result of step S3222 is "yes", step S3232 is not executed. When the judgment result of step S3222 is "no", step S3232 is executed, and step S3222 is executed again. The execution order of step S3221, step S3222, step S3231, and step S3222 can be set as needed. For example, please continue to refer to Figure 13 In a typical implementation, step S3221 is first executed. If the judgment result is "No," step S3231 is then executed, and then step S3221 is executed again until the judgment result of step S3221 is "Yes." Next, step S3222 is executed. If the judgment result is "Yes," it is determined that the executable target motion path has been obtained, and step S3232 is not executed. If the judgment result of step S3222 is "No," step S3232 is then executed, and then step S3222 is executed again until the judgment result of step S3222 is "Yes."

[0093] For further information, please refer to Figure 7 Combined with Figure 14After obtaining the target motion path, the path planning method for the surgical instrument may further include step S40: planning the motion trajectory of the surgical instrument 60 according to the target motion path, that is, imposing a time constraint on the target motion path to obtain the relationship between the position and posture of the surgical instrument 60 and time when the surgical instrument 60 moves along the target motion path. In the surgical robot system, the surgical instrument 60 is moved by the tool arm 32. After obtaining the motion trajectory of the surgical instrument 60, the relationship between the velocity, acceleration, and position of each joint of the tool arm 32 and time can be solved according to the inverse kinematics of the robot arm, that is, the velocity information, acceleration information, and position information of the tool arm 32 can be obtained. In this way, by controlling the tool arm 32 to move according to the velocity information, acceleration information, and position information, the surgical instrument 60 can be controlled to move according to the motion trajectory.

[0094] Since the movement of the surgical instrument 60 is actually achieved through the movement of the tool arm 32, the relevant performance of the tool arm 32, such as the workspace of the tool arm 32, the performance of the robot drive train, and the dynamics of the tool arm 32, must be considered when planning the trajectory of the surgical instrument 60. In other words, in addition to using time as a constraint, at least one of the workspace of the tool arm 32, the performance of the robot drive train, and the dynamics of the tool arm 32 can also be used as a constraint when planning the trajectory of the surgical instrument 60. By using the workspace constraint of the tool arm 32, the extreme position of the surgical instrument 60 during movement can be determined, thereby implementing position constraints on the movement of the surgical instrument 60. By using the performance constraint of the robot drive train, the maximum speed of the surgical instrument 60 during movement can be determined, thereby implementing speed constraints on the surgical instrument 60. By using the dynamic constraints of the tool arm 32, the maximum torque of the surgical instrument 60 during movement can be determined, thereby implementing torque constraints on the surgical instrument 60. By combining the position constraint, velocity constraint and torque constraint of the surgical instrument 60 , the acceleration constraint of the surgical instrument 60 can be obtained.

[0095] In addition, in some cases, such as Figure 15 As shown, the control unit only plans one motion path (specifically, the target motion path), and accordingly, the control unit also only plans one motion trajectory. In this way, the tool arm 32 can be controlled to move to drive the surgical instrument 60 to move along the motion trajectory. In other cases, the control unit plans more than two motion paths, such as Figure 16 and Figure 17 As shown in the figure, two target motion paths are shown, which are the shortest path and the safest path. Figure 7 As shown, the path planning method for the surgical instrument further includes step S50, wherein the step S50 is: generating prompt information to prompt the execution of the intervention operation to select the path. According to actual needs, the operator can select the path with the shortest time (such as Figure 16 As shown in FIG), the surgical instrument 60 can be moved along the shortest path in subsequent operations, thereby shortening the operation time. Alternatively, the operator can choose the safest path (as shown in FIG). Figure 17 As shown), so that the surgical instrument 60 moves along the safest path in subsequent operations, further improving the reliability and safety of the surgical operation. Here, the safest path refers to the distance d from any point on the target path to the boundary of any tissue being greater than or equal to the safety distance dsafe, and the safety distance can be manually set by the operator. The step S50 can be performed after the step S30 and before the step S40, so that the control unit only needs to perform trajectory planning for the selected target motion path, without the need to perform trajectory planning for multiple target motion paths, thereby reducing the performance requirements for the control unit. Of course, in an alternative embodiment, the step S50 can also be performed after the step S40.

[0096] Furthermore, in an embodiment of the present invention, Figure 7 As shown, the path planning method for the surgical instrument also includes step S60: sending the motion path (specifically, the target motion path) to a display device for display. The display device mentioned here is, for example, at least one of the display device 40 and the immersive display device 11.

[0097] From the above introduction to the path planning method of the surgical instrument, it can be seen that when performing a surgical operation using the surgical robot system, human-machine interaction between the operator and the surgical robot system is also involved. Figure 1 The surgical robot system shown in FIG. 1 (the surgical robot system is a master-slave mapping surgical robot system) is taken as an example. Figure 18 As shown, the doctor-side control device 10, the display device 40 and the surgical operation device 30 can constitute an interactive unit, wherein the doctor-side control device 10 can receive instructions issued by the operator, such as path selection instructions (when the control unit has planned two movement paths), various functional operation instructions, emergency stop instructions when the operation may be dangerous, etc. The display device 40 is used to provide various visual prompts, such as prompting that the two movement paths are planned through image display, and prompting path selection. The surgical operation device 30 performs corresponding operations according to the instructions issued by the operator. At the same time, a voice reminder module can also be provided on the surgical operation device 30 to provide voice reminders.

[0098] Also, please refer to Figure 19 In the surgical robot system, the doctor-side control device 10 includes a processing module 12, in which a mapping relationship between the master operator and each joint of the tool arm 32 is preset. After obtaining the relationship between the position, velocity, and acceleration of the tool arm 32 during movement over time through the motion trajectory planning of the surgical instrument 60, it is necessary to plan the movement of the master operator in combination with the master-slave mapping relationship so that when the master operator moves, the joint controllers of each joint of the tool arm 32 can be actuated, thereby causing the surgical instrument 60 to move along the motion trajectory in response to the movement of the master operator.

[0099] It should be noted that the above description uses a master-slave mapping robot system as an example to illustrate the path planning method for the surgical instrument. However, in alternative embodiments, the surgical robot system may also be a non-master-slave mapping robot. Here, the control unit may be configured to directly control the movement of the tool arm 32 to drive the surgical instrument 60 to move along the motion trajectory.

[0100] Furthermore, an embodiment of the present invention also provides a computer-readable storage medium, on which a program is stored. When the program is executed, each step in the path planning method for a surgical instrument as described above is executed.

[0101] Furthermore, an embodiment of the present invention also provides an electronic device, which includes a processor and the aforementioned computer-readable storage medium, and the processor is configured to execute a program stored on the computer-readable storage medium.

[0102] While the present invention is disclosed above, it is not limited thereto. Those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, the present invention is intended to encompass such modifications and variations as long as they fall within the scope of the claims and their equivalents.

Claims

1. A computer-readable storage medium, characterized in that A program is stored thereon, and when the program is executed, the following steps are performed: establishing a three-dimensional model of the surgical area based on first preoperative image information of the surgical area; Acquiring real-time second image information of a local area of the surgical area during the operation; planning a global motion path of the surgical instrument according to the three-dimensional model, the initial position and target position of the surgical instrument; determining a target environment in the three-dimensional model that matches the second image information; Correcting a portion of the global motion path within the target environment according to the second image information, the target environment, and the surrounding environment of the target environment to obtain a target motion path; The global motion path includes all expected positions of the surgical instrument between the initial position and the target position; The second image information displays the surgical instrument and tissue; The step of correcting the portion of the global motion path located within the target environment according to the second image information, the target environment, and the surrounding environment of the target environment comprises: acquiring a relative positional relationship between the surgical instrument and the tissue according to the second image information; Based on the relative positional relationship between the surgical instrument and the tissue, the target environment and the surrounding environment of the target environment, it is determined whether the surgical instrument can move according to all the expected postures within the target environment. If not, at least part of the expected postures within the target environment are corrected.

2. The computer-readable storage medium according to claim 1, wherein The expected posture includes an expected position and an expected posture; and correcting at least part of the expected posture in the target environment includes correcting at least part of the expected position and / or at least part of the expected posture in the target environment.

3. The computer-readable storage medium according to claim 1, wherein The program also performs the following steps: The motion trajectory of the surgical instrument is planned according to the target motion path.

4. The computer-readable storage medium according to claim 3, wherein: The tool arm of the surgical robot system drives the surgical instrument to move along the motion trajectory; When planning the motion trajectory of the surgical instrument, at least one of the working space of the tool arm, the dynamics of the tool arm, and the transmission chain performance of the surgical robot system is used as a constraint condition.

5. The computer-readable storage medium according to claim 1, wherein The surgical instrument is connected to the end of a tool arm of a surgical robot system; the program further performs the following steps: The tool arm is controlled to move so as to drive the surgical instrument to move along the target motion path.

6. The computer-readable storage medium according to claim 1, wherein The program also performs the following steps: The target motion path is sent to a display device for display.

7. The computer-readable storage medium according to claim 1, wherein The program plans more than two target motion paths; the program further executes the following steps: generating prompt information, prompting to perform an intervention operation to perform path selection.

8. An electronic device, characterized in that: The system comprises a processor and a computer-readable storage medium according to any one of claims 1 to 7, wherein the processor is configured to execute a program stored on the computer-readable storage medium.

9. A surgical robot system, characterized in that: include: an imaging arm, a distal end of which is connected to an image acquisition device, the image acquisition device being used to acquire second image information of a local area of the surgical area; as well as, A control unit is communicatively connected to the image acquisition device to receive the second image information, and the control unit is further configured to execute the program stored on the computer-readable storage medium according to any one of claims 1 to 7.

10. The surgical robot system according to claim 9, characterized in that: The surgical robot system also includes a tool arm, the end of which is connected to a surgical instrument; the tool arm is communicatively connected to the control unit, and the tool arm moves under the control of the control unit to drive the surgical instrument to move along the target motion path.

Images