A computer-readable storage medium, an electronic device, and a surgical robot system

Abstract

The application relates to a computer readable storage medium, which stores a program, when the program is executed, the following steps are performed: a first sign image model is established according to first body surface information and lesion information of a surgical object in a first state, the first sign image model is used for planning a pre-hole position; the elastic property of soft tissue of the surgical object is valued as a deformation parameter of the first sign image model; a second sign image model is established according to second body surface information of the surgical object in a second state; the first sign image model is deformed according to the deformation parameter, and the deformed first sign image model is registered with the second sign image model, so that a target hole position corresponding to the pre-hole position is obtained on the second sign image model. The problem that the pre-hole position is inaccurate due to the body surface deformation of the surgical object in different states caused by the body position change or other changes of the surgical object is reduced, the surgical safety is improved, and the pain of the surgical object is reduced.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of medical devices, and particularly relates to a computer-readable storage medium, an electronic device and a surgical robot system. BACKGROUND

[0002] The design concept of a surgical robot is to accurately implement a complex surgical operation in a minimally invasive manner. In the face of various limitations of traditional surgical operations, surgical robots have emerged. The surgical robot breaks through the limitations of the human eye and can use stereoscopic imaging technology to present the organs inside the human body more clearly to the operator. And for some narrow areas where the hands of some people cannot reach, the surgical robot can still control the surgical instruments to complete the movement, swing, clamping and 360-degree rotation, and can avoid shaking, improve the operation accuracy, and further achieve the advantages of small incision, less bleeding, fast postoperative recovery and greatly shortened postoperative hospitalization time of the surgical object. Therefore, the surgical robot is favored by the majority of doctors and patients and is widely used in various clinical operations.

[0003] Like traditional surgery, before performing surgery by using a surgical robot, the lesion needs to be located, and then a punch site is determined on the body surface of the surgical object according to the lesion position, and then a hole is punched at the punch site, so that the surgical instrument can enter the surgical object from the punch site to perform the surgical operation. In the prior art, a doctor obtains image information of the lesion and the body surface of the surgical object by using a CT or MRI device to establish a first body sign image model of the surgical object and determine the lesion position, and then plans the punch site according to the experience of the doctor, and then punches. However, this way of determining the punch site has the following problems: the soft tissue of the human body moves in position and deforms in the transfer process or without the surgical position, which may cause the pre-planned position and the punch position to change. If these changes cannot be known in advance, it is likely to affect the convenience and safety of the surgical operation.

[0004] In view of the above analysis, in the traditional surgical process, the final determination of the punch site on the body surface of the surgical object greatly depends on the preoperative judgment, and it is difficult to reconfirm the preoperative planning before punching, which may cause unnecessary harm to the surgical object. SUMMARY

[0005] The purpose of the present application is to provide a computer-readable storage medium, an electronic device and a surgical robot system, which can identify the morphological changes of the soft tissue of the surgical object before the operation, so as to plan the punch site on the body surface of the surgical object, improve the accuracy of the punch site, and further improve the surgical efficiency, ensure the safety of the operation and reduce the pain of the patient.

[0006] To achieve the above object, the application provides a computer readable storage medium, which stores a program, when the program is executed, the following steps are performed:

[0007] A first sign image model is established according to the first body surface information and the lesion information of the surgical object in the first state, and the first sign image model is used for planning a pre-hole position;

[0008] The elastic property of the soft tissue of the surgical object is assigned as a deformation parameter of the first sign image model;

[0009] A second sign image model is established according to the second body surface information of the surgical object in the second state;

[0010] The first sign image model is deformed according to the deformation parameter, and the deformed first sign image model is registered with the second sign image model, so that a target hole position corresponding to the pre-hole position is obtained on the second sign image model.

[0011] Optionally, the deformation parameter includes at least one of an elastic coefficient and a surface tension.

[0012] Optionally, when the first sign image model is deformed, the program performs the following steps:

[0013] The first sign image model is subjected to a minimum non-rigid calculation according to the deformation parameter and the second sign image model.

[0014] Optionally, the first sign image model includes a first lesion model; the target hole position includes a first target hole position and a second target hole position; the second target hole position is used for being directed to the body surface of the surgical object to obtain a second hole position; the second target hole position is used for an image acquisition device to enter the surgical object to collect actual lesion image information of the surgical object in the second state;

[0015] The program further performs the following steps:

[0016] The actual lesion image information is received and a second lesion model is established according to the actual lesion image information;

[0017] The first sign image model is deformed according to the deformation parameter;

[0018] The first lesion model in the deformed first sign image model is registered with the second lesion model, and the first target hole position is corrected according to the registration result.

[0019] Optionally, when the first sign image model is deformed, the program performs the following steps:

[0020] Minimal non-rigid calculations are performed on the first sign image model based on the deformation parameters and the actual lesion image.

[0021] Optionally, when deforming the first vital sign image model, the program performs the following steps:

[0022] Based on the deformation parameters, perform at least one of the following operations on the first vital sign image model: position shift, posture rotation, overall reduction, overall enlargement, or point cloud state deformation adjustment.

[0023] Optionally, when planning the pre-hole locations on the first vital sign image model, the program performs the following operations:

[0024] Multiple options for the pre-hole positions are generated to select the desired pre-hole position.

[0025] Optionally, the first body surface information and lesion information are acquired through a first imaging device, which includes any one of X-ray equipment, MRI, or B-ultrasound; and / or, the second body surface information is acquired through a second imaging device, which includes any one of binocular vision camera or structured light camera.

[0026] To achieve the above objectives, the present invention also provides an electronic device including a processor and a computer-readable storage medium as described in any of the preceding claims, the processor being configured to execute a program stored on the computer-readable storage medium.

[0027] To achieve the above objectives, the present invention also provides a surgical robot system, including a control unit and a tool arm, wherein the control unit is configured to perform the steps of the program as described in any of the preceding claims; the tool arm is provided with an auxiliary device; the tool arm is communicatively connected to the control unit, and the control unit is used to control the movement of the tool arm such that the auxiliary device guides the target hole on the second vital sign image model to the surface of the surgical subject.

[0028] Optionally, the tool arm has a fixed point, and the auxiliary device includes at least two laser emitters whose lasers intersect at the fixed point; the control unit controls the movement of the tool arm such that the intersection of the lasers is indicated on the surface of the surgical subject to guide the target hole to the surface of the surgical subject.

[0029] Optionally, the target hole includes a first target hole and a second target hole, the second target hole being guided to the surface of the surgical subject to obtain the second hole; the surgical robot system also includes an image arm, the image arm being used to connect to an image acquisition device; the image acquisition device is communicatively connected to the control unit, and the image acquisition device is inserted into the surgical subject through the second hole and acquires actual lesion image information of the surgical subject in the second state, and sends the actual lesion image information to the control unit to establish a second lesion model.

[0030] Compared with the prior art, the computer-readable storage medium, electronic device, and surgical robot system of the present invention have the following advantages:

[0031] First, the aforementioned computer-readable storage medium stores a program that, when executed, establishes a first vital sign image model based on the first surface information and lesion information of the surgical subject in a first state. This first vital sign image model is used to plan pre-hole locations. The elastic properties of the soft tissue of the surgical subject are assigned values ​​as deformation parameters for the first vital sign image model. A second vital sign image model is established based on the second surface information of the surgical subject in a second state. The first vital sign image model is deformed according to the deformation parameters, and the deformed first vital sign image model is registered with the second vital sign image model to obtain a target hole location corresponding to the pre-hole location on the second vital sign image model. When the computer-readable storage medium is applied to a surgical robot system to perform operations requiring drilling on the body of a surgical subject, the accuracy of the drilling location can be improved by registering the vital sign image models of patients in different states. This avoids the problem of inaccurate pre-hole locations caused by surface distortions of the surgical subject in different states due to changes in the patient's position or other changes, thereby improving surgical safety, reducing surgical risks, and minimizing the patient's pain.

[0032] Second, the first vital sign image model includes a first lesion model; the target hole includes a first target hole and a second target hole. The second target hole is used to guide the surgical subject's body surface to obtain the second hole. After the second hole is drilled on the surgical subject's body surface, an image acquisition device is inserted into the surgical subject's body from the second hole and collects the actual lesion image information of the surgical subject. The program performs modeling based on the actual lesion image information to obtain a second lesion model, and registers the second lesion model with the first lesion model on the deformed first vital sign image model. Based on the registration result, the first target hole on the second vital sign image model is corrected. Finally, the corrected first target hole is guided to the surgical subject's body surface to further improve the accuracy of the first hole. Attached Figure Description

[0033] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0034] Figure 1 This is a schematic diagram illustrating an application scenario of the surgical robot system provided by the present invention according to an embodiment;

[0035] Figure 2 This is a schematic diagram of the surgical operation device of the surgical robot system provided by the present invention according to an embodiment;

[0036] Figure 3 This is a schematic diagram illustrating the acquisition of actual lesion image information using an endoscope in an embodiment of the present invention;

[0037] Figure 4 This is a flowchart illustrating the surgical robot system according to an embodiment of the present invention during surgical drilling.

[0038] Figure 5 This is a schematic diagram illustrating the acquisition of first body surface information and lesion information of the surgical subject using a first imaging device in an embodiment of the present invention;

[0039] Figure 6 This is a schematic diagram of a first vital sign image model established by the control unit of a surgical robot system according to an embodiment of the present invention;

[0040] Figure 7 This is a schematic diagram illustrating the acquisition of second body surface information of a surgical subject using a second imaging device in an embodiment of the present invention;

[0041] Figure 8 This is a schematic diagram illustrating the acquisition of second body surface information using another second imaging device in an embodiment of the present invention;

[0042] Figure 9 This is a schematic diagram illustrating the acquisition of second body surface information using a second imaging device according to an embodiment of the present invention;

[0043] Figure 10 This is a schematic diagram of the registration of the lesion model in the first sign image model and the actual lesion image by the control unit of the surgical robot system provided according to an embodiment of the present invention;

[0044] Figure 11 This is a schematic diagram of the registration of the lesion model in the first sign image model and the actual lesion image in the control unit of the surgical robot system provided by the present invention according to an alternative embodiment.

[0045] In the attached image:

[0046] 10-Doctor's control device, 20-Surgical operation device, 21-Image arm, 22-Tool arm, 30-Image display device;

[0047] 100 - First imaging equipment, 200 - Second imaging equipment, 300 - Endoscope. Detailed Implementation

[0048] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in this embodiment are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show components related to the present invention and are not drawn according to the actual number, shape, and size of components in the actual implementation. In the actual implementation, the type, quantity, and proportion of each component can be arbitrarily changed, and the component layout may also be more complex.

[0049] Furthermore, while each embodiment described below possesses one or more technical features, this does not imply that users of the present invention must simultaneously implement all technical features in any embodiment, or can only separately implement some or all technical features in different embodiments. In other words, provided it is feasible, those skilled in the art can, based on the disclosure of the present invention and depending on design specifications or implementation requirements, selectively implement some or all technical features in any embodiment, or selectively implement a combination of some or all technical features in multiple embodiments, thereby increasing the flexibility in implementing the present invention.

[0050] As used herein, the singular forms “a,” “an,” and “the” include plural objects, and the plural form “a plurality” includes two or more objects, unless otherwise expressly indicated. As used herein, the term “or” is generally used to include the meaning of “and / or,” unless otherwise expressly indicated, and the terms “install,” “connect,” and “link” should be interpreted broadly, for example, as a fixed connection, a detachable connection, or an integral connection. Connections can be mechanical or electrical. Connections can be direct or indirect through an intermediate medium, and can represent internal communication between two elements or an interaction between two elements. Those skilled in the art will understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0051] To make the objectives, advantages, and features of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. It should be noted that the drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clearly illustrate the objectives of the embodiments of the present invention. The same or similar reference numerals in the drawings represent the same or similar parts.

[0052] Figure 1 A schematic diagram illustrating an application scenario of the surgical robot system of the present invention is shown. Figure 2 A schematic diagram of the surgical operation device of the surgical robot system is shown. Please refer to it. Figure 1 and Figure 2 The surgical robot system includes a control end and an execution end. The control end includes a doctor's console and a doctor's control device 10 mounted on the doctor's console. The execution end includes a patient-side control device, a surgical operation device 20, an image display device 30, and other equipment. The surgical operation device 20 is equipped with a robotic arm, which includes an image arm 21 and a tool arm 22. The tool arm 22 is used to mount a punching device or surgical instrument. The punching device is used to punch a hole at a first opening on the patient's body surface, and the surgical instrument is used to insert into the surgical subject through the first opening and perform the surgical operation. The image arm 21 is used to mount an image acquisition device, which is used to acquire image information of the region or device of interest (e.g., the actual lesion image of the surgical subject in a second state, as described later). The image acquisition device is, for example, an endoscope 300 (e.g., an endoscope 300). Figure 3 (As indicated). Furthermore, the surgical robot system also includes a control unit, which is communicatively connected to the imaging arm 21, the tool arm 22, and the endoscope 300. The control unit can be located at the patient-side control device, or at the doctor-side control device, or partially at the patient-side control device and partially at the doctor-side control device. In other words, the present invention does not limit the specific arrangement of the control unit, as long as it can perform the relevant functions.

[0053] Before performing laparoscopic or thoracoscopic surgery using the surgical robot system, the control unit of the surgical robot system first obtains the perforation sites on the surface of the surgical subject. That is, the control unit is configured to establish a first vital sign image model S (e.g., based on the first surface information and lesion information of the surgical subject in a first state) Figure 6(As indicated), the first vital sign image model S is used to plan the pre-hole location and to establish a second vital sign image model based on the second surface information of the surgical object in the second state; the elastic properties of the soft tissue of the surgical object are assigned values ​​as deformation parameters of the first vital sign image model S; the first vital sign image model S is deformed according to the deformation parameters, and the deformed first vital sign image model S is registered with the second vital sign image model, and the target hole location corresponding to the pre-hole location is obtained on the second vital sign image model. Afterwards, the operator can make holes on the surface of the surgical object. It should be understood that the first vital sign image model S displays the first surface model S1 and the first lesion model S2 of the surgical object (e.g., ...). Figure 6 (as shown),

[0054] Those skilled in the art will understand that both the first state and the second state refer to the state of the surgical object itself, and that at least some of the soft tissues of the surgical object are in different positions in the first state and the second state. Taking laparoscopic surgery as an example, the first state can be the state of the surgical object before pneumoperitoneum, while the second state can be the state of the surgical object after pneumoperitoneum is established. Alternatively, the difference between the first state and the second state may be due to the fixation, transportation, or other reasons of the surgical object on the operating table, and this invention does not limit this. It should also be understood that modeling the elastic properties of the soft tissues of the surgical object is a practice familiar to those skilled in the art. For example, master's theses such as "Research on the Elastic Model of Human Soft Tissue" (Southeast University, Jiang Chuntao, 2004) and "Modification of the Elastic Model in Human Soft Tissue Modeling" (Southeast University, Sun Yanxia, ​​2005) detail the elastic models of human tissues. Those skilled in the art can use these to model the elastic models of tissues in the human body to establish elastic models of human tissues.

[0055] For ease of description, the following text will use the state of the surgical subject before pneumoperitoneum during laparoscopic surgery as the first state and the state after pneumoperitoneum is established as the second state as an example.

[0056] In this embodiment of the invention, the pre-drilling hole location is first planned based on the surface information and lesion information of the surgical subject displayed in the first vital sign image model S before pneumoperitoneum. However, after pneumoperitoneum, the vital signs of the surgical subject are deformed relative to before pneumoperitoneum, resulting in inconsistencies with the vital sign information on the first vital sign image model S (e.g., inconsistent surface information). Based on this, the control unit establishes a second vital sign image model after pneumoperitoneum and registers the second vital sign image model with the first vital sign image model S. Then, based on the registration result and the pre-drilling hole location, the target hole location is determined on the second vital sign image model and directed to the surface of the surgical subject after pneumoperitoneum. That is, this embodiment can reduce the reliance on the surgeon's experience, reduce the deviation between the planned and actual drilling sites on the surface caused by the surgical subject's surface deformities, improve the accuracy of the drilling sites on the surface, lay a good foundation for the normal conduct of the surgery, effectively reduce unnecessary drilling, shorten the operation time, reduce the surgeon's fatigue, and minimize harm to the surgical subject.

[0057] The target ports on the second vital sign image model include a first target port and a second target port. In some embodiments, the first target port can be directly guided to the surface of the surgical subject in the second state to obtain the first port, and the second target port can be guided to the surface of the surgical subject in the second state to obtain the second port. The surgical instruments are used to insert into the abdominal cavity through the first port to perform surgical operations, and the endoscope 300 is communicatively connected to the control unit and is used to insert into the abdominal cavity through the second port (e.g., ...). Figure 3 (As shown) to provide a surgical field of view.

[0058] Preferably, in other embodiments, the operator first guides the second target hole to the surface of the surgical subject in the second state to obtain the second hole, and makes a hole at the second hole. Then, the endoscope 300 is also used to acquire actual lesion image information of the surgical subject in the second state. Here, the control unit is also configured to establish a second lesion model based on the actual lesion image information, and register the second lesion model with the first lesion model S2 in the deformed first sign image model S (e.g., ...). Figure 10 and Figure 11 (As shown), the first target hole position on the second sign image model is then corrected based on the registration result. The corrected first target hole position is then guided to the surface of the surgical subject to obtain a more accurate first hole position.

[0059] Thus, in a non-limiting embodiment, the method for determining the drilling sites (including the first and second holes) on the surface of the surgical subject using the surgical robot system is as follows: Figure 4 As shown, the details are as follows:

[0060] First, perform step S1: use the first imaging device to acquire the first body surface information and lesion information of the surgical subject before pneumoperitoneum (i.e., the first state).

[0061] In this step, such as Figure 5 As shown, the first imaging device 100 includes various three-dimensional model scanning devices, such as X-ray equipment like CT scanners, or MRI, B-ultrasound, etc. In one specific embodiment, the first imaging device 100 is a CT scanner. Before performing a CT scan on the surgical subject, the surgical subject is not pneumoperitoneum established. The doctor makes a preliminary diagnosis of the cause of the surgical subject's illness, roughly determines the organ where the lesion is located, infers the possible causes, and the body position required during surgery. Subsequently, the surgical subject is scheduled for a CT scan to obtain surface information of the possible location of the lesion (this surface information is used as the first surface information) and internal image information. Then, the doctor identifies and confirms the lesion based on the obtained image information to obtain the lesion information. Those skilled in the art will understand that the first imaging device 100, while obtaining the lesion information, also obtains information about the organs or tissues surrounding the lesion.

[0062] Next, step S2 is executed: the control unit establishes a first sign image model S of the surgical subject before pneumoperitoneum based on the first body surface information and the lesion information, and assigns model values ​​to the elastic properties of the soft tissue of the surgical subject while establishing the first sign image model S. Further, the first sign image model S also displays the organs or tissues surrounding the lesion ( Figure 6 (Not shown in the image).

[0063] The control unit can employ a reconstruction algorithm based on Marching Cube rendering to establish the first vital sign image model. Specifically, the control unit constructs multiple geometric primitives in a three-dimensional volume data field composed of two-dimensional slices, based on the contour information obtained from the two-dimensional slices of the surgical object's surface and lesion. These geometric primitives are then stitched together, and a lighting model is established to construct a realistic three-dimensional model as the first vital sign image model S. Deformation parameters of the first vital sign image model S are obtained by assigning model values ​​to the elastic properties of the surgical object, serving as the basis for deformation of the first vital sign image model S in subsequent registration processes. These deformation parameters include, but are not limited to, elastic coefficients and surface tension.

[0064] Next, step S3 is performed: the pre-hole positions are planned on the first vital sign image model S.

[0065] In this step, the control unit can be used to plan the pre-hole positions. Specifically, the control unit generates the pre-hole positions based on the surgical area, the surgical instruments used, and the boundaries of the movement space of the surgical instruments during the surgical process.

[0066] Preferably, the control unit generates multiple pre-aperture positions based on different schemes. Then, the surgeon performs a simulated surgery on a simulator using these pre-aperture positions, selecting the desired position based on factors such as surgical outcome, ease of operation for the surgeon, and comfort. This not only determines the optimal pre-aperture position but also allows for the identification of potential problems during the surgery. Those skilled in the art will understand that the simulator used for the simulated surgery should have the same configuration as the surgical robot system actually performing the surgery. Optional simulators include, but are not limited to, the SEP robot simulator.

[0067] Next, step S4 is performed: the second imaging device is used to acquire the second body surface information of the surgical subject after pneumoperitoneum (i.e., the second state).

[0068] like Figure 7 to Figure 9 As shown, the second imaging device 200 has multiple options; for example, please refer to... Figure 7 In one embodiment, the second imaging device 200 includes a binocular vision camera with built-in distance detection capability. This binocular vision camera can directly acquire the relative position data of the surgical subject's body surface relative to the binocular vision camera, as the second body surface information. Alternatively, as... Figure 8 As shown, in another embodiment, the second imaging device 200 includes a binocular vision camera without distance detection capability. In this case, multiple target points 1 are set on the surface of the surgical subject. The binocular vision camera is used to identify the multiple target points 1 and acquire the coordinate point cloud of the multiple target points 1 to obtain the second surface information of the surgical subject. Alternatively, as... Figure 9 As shown, in another embodiment, the second imaging device 200 includes a structured light camera that projects structured light onto the surface of the surgical subject. The deformation or time-of-flight of the structured light is used to determine the dimensional parameters of the surgical subject's surface, thereby obtaining second surface information of the surgical subject. In the above embodiments, the second imaging device is a standalone device; however, in other embodiments, the second imaging device 200 may be mounted on the surgical subject-side control device 30.

[0069] Next, step S5 is executed: the control unit establishes a second vital sign image model of the surgical subject after pneumoperitoneum based on the second body surface information.

[0070] Next, step S6 is executed: the first vital sign image model is deformed according to the deformation parameters, and the deformed first vital sign image model is registered with the second vital sign image model to obtain the target hole position (i.e., the first target hole position and the second target hole position) on the second vital sign image model that corresponds to the pre-hole position on the first vital sign image model.

[0071] In some embodiments, the control unit performs minimum non-rigid calculations on the first vital sign image model S based on the deformation parameters and the second vital sign image model until the first surface model S1 of the first vital sign image model S is registered with the second vital sign image model. Based on this registration result, the deviation between the first vital sign image model S and the second vital sign image model is obtained, and the pre-hole position can be corrected to obtain the target hole position on the second vital sign image model.

[0072] In other embodiments, the control unit includes a display device that simultaneously displays the second vital sign image model and the first vital sign image model (not shown). Then, based on the deformation parameters, the surgeon performs at least one of the following deformation adjustments on the first vital sign image model S on the control unit: position movement, posture rotation, overall magnification, overall reduction, or adjustment to a point cloud state. Furthermore, during these operations, the surgeon compares the deformed first vital sign image model S and the second vital sign image model in real time until the first body surface model S1 of the deformed first vital sign image model S matches the second vital sign image model, achieving registration.

[0073] After the registration operation in step S6, the surface information of the surgical subject after pneumoperitoneum can be identified and introduced into the world coordinate system F0 where the surgical robot system is located.

[0074] Then, step S7 can be performed: using a suitable guiding device to guide the second target hole on the second vital sign image model to the surface of the surgical subject to obtain the second hole on the surface of the body.

[0075] Optionally, in this embodiment, the tool arm 22 and the auxiliary device mounted thereon are used as the guiding device. The tool arm 22 moves under the control of the control unit so that the auxiliary device guides the second target hole position on the second vital sign image model to the surface of the surgical subject to obtain the second hole position. Specifically, the tool arm 22 has a fixed point, and each joint of the tool arm 22 can swing / rotate around this fixed point. The auxiliary device includes two or more laser emitters, and the laser beams emitted by the two or more laser emitters intersect at the fixed point. After establishing the mapping relationship between the coordinate system of the tool arm 22 and the coordinate system of the surface of the surgical subject after pneumoperitoneum, when the tool arm 22 swings under the control of the control unit so that the intersection point of the laser beams (i.e., the fixed point) points to the surface of the surgical subject, the position of the intersection point is the hole position on the surface of the body.

[0076] As is well known to those skilled in the art, when the control unit is located at the surgical operating device 20, since the coordinate system of the tool arm 22 is the coordinate system F1 of the surgical operating device 20, and this coordinate system F1 can be mapped to the coordinate system F2 of the second imaging device 200 in the world coordinate system F0, and the mapping relationship between the coordinate system F3 of the surgical subject's body surface after pneumoperitoneum and the coordinate system F2 of the second imaging device 200 is known (the mapping relationship can be obtained when the second imaging device acquires the second body surface data of the surgical subject), a mapping relationship can be established between the coordinate system F1 of the tool arm 22 (that is, the coordinate system of the surgical operating device 20) and the coordinate system F3 of the surgical subject's body surface after pneumoperitoneum. Afterwards, the control unit can drive the tool arm 22 to move for guiding operations.

[0077] Of course, in this embodiment, other devices independent of the surgical robot system can also be used as the guiding device, as long as they can be recognized by the surgical robot system in the world coordinate system and a mapping relationship can be established with the body surface coordinate system of the surgical object after pneumoperitoneum.

[0078] Next, step S8 is performed: a hole is drilled at the second pore location on the body surface. This step can be performed manually by the practitioner or by the tool arm 22 (i.e., the drilling device is mounted on the robotic arm, and the robotic arm drives the drilling device to perform the drilling operation).

[0079] Next, step S9 is performed: an endoscope is inserted into the surgical subject through the second port and the actual lesion image information is acquired. A second lesion model is then established based on the actual lesion image information. During the acquisition of the actual lesion image information, the endoscope can be moved in all directions—forward, backward, up, down, left, and right—to more accurately display the shape and size of the lesion in the acquired lesion image.

[0080] After the control unit acquires the second lesion model of the surgical object, it executes step S10: the image terminal control device deforms the first sign image model so that the first lesion model of the first sign image model is registered with the second lesion model, and corrects the first target hole position according to the registration result.

[0081] Afterwards, the operator can guide the corrected first target hole position to the surface of the surgical subject to obtain the actual determined first hole position.

[0082] Similar to step S6, the control unit can perform a minimum stiffness algorithm (e.g., ...) on the first vital sign image model S based on the deformation parameters and the actual lesion image. Figure 10 As shown), so that the lesion model S2 of the first sign image model S is registered with the actual lesion image. Or, as Figure 11 As shown, the image display device 30 simultaneously displays the lesion model S2 of the human body model S and the actual lesion image on its display screen. Then, according to the deformation parameters, the surgeon manipulates the lesion model S2 on the image display device 30. Figure 11 The image only shows the lesion model S2, and does not show the complete first sign image model S) performing at least one of the following deformation adjustments: position movement, posture rotation, overall magnification, overall reduction, or adjustment to a point cloud state. Furthermore, during these operations, the surgeon compares the deformed first lesion model S2 and the second lesion image in real time until the deformed first lesion model S2 matches the second lesion image, achieving registration. Through this registration operation, the deviation between the first lesion model S2 and the second lesion model in the first sign image model S is obtained, and the first target hole position is corrected based on this deviation.

[0083] The surgeon can then use any suitable method to guide the corrected first target hole onto the surface of the surgical subject to obtain the first hole.

[0084] It should be noted that when the surgical robot system determines the punching sites on the surface of the surgical subject, Figure 3 The order of steps described is not fixed and can be adjusted according to actual needs. For example, steps S3 and S4 can be performed simultaneously, or step S4 can be performed before step S3.

[0085] Furthermore, the present invention also provides a computer-readable storage medium having a program stored thereon. When the program is executed, the program performs all the operations performed by the aforementioned control unit.

[0086] Furthermore, the present invention also provides an electronic device comprising a processor and the computer-readable storage medium, the processor being configured to execute a program stored on the computer-readable storage medium.

[0087] Furthermore, embodiments of the present invention also provide a hole location planning method, including the steps performed by the control unit as described above when planning target hole locations.

[0088] The technical solution provided by this invention improves the accuracy of perforation locations by applying image registration technology to the surface of the surgical subject in the second state before actual surgery, thereby increasing surgical efficiency, ensuring surgical safety, avoiding additional perforations due to inaccurate locations, and reducing the patient's pain. Specifically, this invention also acquires actual lesion image information of the surgical subject in the second state using an image acquisition device (i.e., an endoscope) inserted into the patient's body and establishes a second lesion model. The actual lesion model is then registered with a first lesion model of the surgical subject in the first state. Based on the registration result, the first target perforation location on the second lesion image model is corrected. Finally, the corrected first target perforation location is guided to the surface of the surgical subject, further improving the accuracy of the first perforation location on the patient's body surface.

[0089] While the present invention has been disclosed above, it is not limited thereto. Those skilled in the art can make various modifications and variations to the present invention without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims and their equivalents, the present invention also intends to include such modifications and variations.

Claims

1. A computer-readable storage medium having a program stored thereon, characterized in that, When the program is executed, the following steps are performed: A first sign image model is established based on the first body surface information and lesion information of the surgical object in the first state. The first sign image model is used to plan the pre-hole position and includes the first body surface model and the first lesion model. The elastic properties of the soft tissue of the surgical subject are assigned values ​​to serve as deformation parameters for the first vital sign image model; A second physical sign image model is established based on the second body surface information of the surgical subject in the second state; The first vital sign image model is deformed according to the deformation parameters, and the first body surface model of the deformed first vital sign image model is registered with the second vital sign image model to obtain the target hole position corresponding to the pre-hole position on the second vital sign image model.

2. The computer-readable storage medium according to claim 1, characterized in that, The deformation parameters include at least one of the elastic coefficient and surface tension.

3. The computer-readable storage medium according to claim 1, characterized in that, When deforming the first vital sign image model, the program performs the following steps: Minimal non-rigid calculations are performed on the first vital sign image model based on the deformation parameters and the second vital sign image model.

4. The computer-readable storage medium according to claim 1, characterized in that, The target aperture includes a first target aperture and a second target aperture; the second target aperture is used to be guided to the surface of the surgical object to obtain a second aperture, and the second aperture is used to allow an image acquisition device to enter the body of the surgical object to collect actual lesion image information of the surgical object in the second state; The procedure also performs the following steps: Receive the actual lesion image information and establish a second lesion model based on the actual lesion image information; The first vital sign image model is deformed according to the deformation parameters; The first lesion model and the second lesion model in the deformed first sign image model are registered, and the first target hole position is corrected according to the registration result.

5. The computer-readable storage medium according to claim 4, characterized in that, When deforming the first vital sign image model, the program performs the following steps: Minimal non-rigid calculations are performed on the first sign image model based on the deformation parameters and the actual lesion image.

6. The computer-readable storage medium according to claim 1 or 3, characterized in that, When deforming the first vital sign image model, the program performs the following steps: Based on the deformation parameters, perform at least one of the following operations on the first vital sign image model: position shift, posture rotation, overall reduction, overall enlargement, or point cloud state deformation adjustment.

7. The computer-readable storage medium according to claim 1, characterized in that, When planning the pre-hole positions on the first vital sign image model, the program performs the following operations: Multiple options for the pre-hole positions are generated to select the desired pre-hole position.

8. The computer-readable storage medium according to claim 1, characterized in that, The first body surface information and lesion information are acquired through a first imaging device, which includes any one of X-ray equipment, MRI or B-ultrasound; and / or, the second body surface information is acquired through a second imaging device, which includes any one of binocular vision camera or structured light camera.

9. An electronic device comprising a processor and a computer-readable storage medium as claimed in any one of claims 1-8, the processor being configured to execute a program stored on the computer-readable storage medium.

10. A surgical robot system, characterized in that, Includes a control unit and a tool arm; the control unit is configured to perform the steps of the program as described in any one of claims 1-8; The tool arm is equipped with an auxiliary device; the tool arm is communicatively connected to the control unit, and the control unit is used to control the movement of the tool arm so that the auxiliary device guides the target hole on the second vital sign image model to the surface of the surgical subject.

11. The surgical robot system according to claim 10, characterized in that, The tool arm has a fixed point, and the auxiliary device includes at least two laser emitters whose lasers intersect at the fixed point; the control unit controls the movement of the tool arm such that the intersection of the lasers is indicated on the surface of the surgical subject to guide the target hole to the surface of the surgical subject.

12. The surgical robot system according to claim 10, characterized in that, The target holes include a first target hole and a second target hole. The second target hole is guided to the surface of the surgical subject to obtain the second hole. The surgical robot system also includes an image arm, which is used to connect to an image acquisition device. The image acquisition device is communicatively connected to the control unit. The image acquisition device is inserted into the surgical subject through the second hole and acquires actual lesion image information of the surgical subject in the second state. The actual lesion image information is then sent to the control unit to establish a second lesion model.

13. The surgical robot system according to claim 10, characterized in that, The surgical robot system further includes a first imaging device and a second imaging device, both of which are communicatively connected to the control unit. The first imaging device is used to acquire the first body surface information and lesion information and send them to the control unit to establish the first sign image model. The second imaging device is used to acquire the second body surface information and send it to the control unit to establish the second sign image model.

Images