A surgical system

Abstract

The application provides a surgical system, which comprises a control device and a surgical equipment; the control device comprises a lesion recognition module, an automatic planning module and a control module; the lesion recognition module is used for acquiring real-time lesion information according to intraoperative real-time medical images and preoperative medical images; the automatic planning module is used for planning a surgical path according to the real-time lesion information to acquire a target surgical path; and the control module is used for controlling the surgical equipment to perform surgery according to the acquired surgical operation parameters and the target surgical path. The application has the advantages of accuracy, reliability, safety, high efficiency and the like, can avoid many risks brought by the fact that the surgical process completely relies on the clinical experience of doctors, can effectively reduce the workload of doctors, improve the work efficiency of doctors, and further enable the doctors to have more energy to put into the analysis of diseases and the optimization of treatment schemes.

Description

TECHNICAL FIELD

[0001] The present application relates to the medical technical field, in particular to a surgical system. BACKGROUND

[0002] Freeze ablation is mainly through low temperature instrument, and lesion tissue is controlled to experience the process of cooling, freezing and rewarming, so as to cause irreversible damage or even necrosis of cells. For example, the killing mechanism of freeze ablation on tumor is: cell dehydration and shrinkage; intracellular ice crystal formation and mechanical damage of ice crystal; cell electrolyte toxicity concentration and PH value change; cell membrane lipid protein component denaturation; blood flow accumulation and microthrombosis; immune effect, etc. Freeze ablation operation not only has small surgical trauma, but also has the advantages of anesthetic pain, fewer postoperative complications, prevention of tumor spread, etc., and is highly praised by doctors and patients.

[0003] At present, lesion identification in freeze treatment operation or puncture operation is mainly through preoperative magnetic resonance (MR) and CT image to judge the lesion position and volume, which has the problems of single image, incomplete lesion display, imaging time and operation time, and cannot completely display the current lesion. Doctors rely on preoperative images to judge lesions, then plan puncture paths according to clinical experience, determine freeze parameters, and perform freeze ablation operation. The whole process cannot be monitored, and the operation process completely relies on the clinical experience of doctors, which has many risks.

[0004] It should be noted that the information disclosed in the background section of the present application is only intended to deepen the understanding of the general background of the present application, and should not be regarded as acknowledging or implying in any form that the information constitutes prior art known to those skilled in the art. SUMMARY

[0005] The purpose of the present application is to provide a surgical system, which can solve the problem that in the prior art, doctors rely on preoperative images to judge lesions, then plan operation paths according to clinical experience, determine operation parameters, and then perform operation. The whole operation process cannot be monitored in real time, and the operation process completely relies on the clinical experience of doctors, which has many risks.

[0006] To solve the above technical problems, the present application provides a surgical system, which comprises a control device and a surgical device, wherein the surgical device is in communication connection with the control device.

[0007] The control device comprises a lesion identification module, an automatic planning module and a control module in communication connection.

[0008] The lesion identification module is used to obtain real-time lesion information according to real-time medical images and preoperative medical images.

[0009] The automatic planning module is used to plan the surgical path based on the real-time lesion information in order to obtain the target surgical path.

[0010] The control module is used to control the surgical equipment to perform surgery based on the acquired surgical operation parameters and the target surgical path.

[0011] Optionally, the surgical system further includes a first image acquisition device, which is communicatively connected to the control device and is used to acquire real-time medical images during the operation.

[0012] Optionally, the lesion recognition module includes an image acquisition unit, an image registration unit, and a lesion recognition unit that are connected in communication.

[0013] The image acquisition unit is used to acquire preoperative medical images and intraoperative real-time medical images;

[0014] The image registration unit is used to register the preoperative medical image and the intraoperative real-time medical image to obtain a real-time registered image;

[0015] The lesion identification unit is used to obtain real-time lesion information based on the real-time registered image.

[0016] Optionally, if the preoperative medical image is a CT image or an MRI image, and the intraoperative real-time medical image is a CT image or an MRI image, then the image registration unit acquires the real-time registration image, including:

[0017] Three-dimensional modeling is performed on the preoperative medical images to obtain preoperative three-dimensional medical images;

[0018] Three-dimensional modeling is performed on the intraoperative real-time medical images to obtain intraoperative real-time three-dimensional medical images;

[0019] The preoperative three-dimensional medical image is registered to the intraoperative real-time three-dimensional medical image to obtain a real-time registered image.

[0020] Optionally, if the preoperative medical image is a CT image or an MRI image, and the intraoperative real-time medical image is an ultrasound image, then the image registration unit acquires the real-time registration image, including:

[0021] Three-dimensional modeling is performed on the preoperative medical images to obtain preoperative three-dimensional medical images;

[0022] The preoperative three-dimensional medical image is registered to the intraoperative real-time medical image to obtain a first real-time registered image;

[0023] The intraoperative real-time medical image is registered to the preoperative three-dimensional medical image to obtain a second real-time registered image.

[0024] Optionally, the surgical system further includes a second image acquisition device that is communicatively connected to the control device, the second image acquisition device being used to acquire real-time images of the patient's skin during the operation;

[0025] The image acquisition unit is also used to acquire real-time images of the patient's skin during the operation;

[0026] The image registration unit acquires real-time registered images, including:

[0027] Three-dimensional modeling is performed on the preoperative medical images to obtain preoperative three-dimensional medical images;

[0028] Three-dimensional modeling is performed on the intraoperative real-time medical images to obtain intraoperative real-time three-dimensional medical images;

[0029] Three-dimensional modeling is performed on the real-time intraoperative patient skin images to obtain real-time intraoperative human body model images;

[0030] The preoperative three-dimensional medical image and the intraoperative real-time three-dimensional medical image are registered and fused to obtain a first real-time fused image;

[0031] The first real-time fused image is registered to the intraoperative real-time human model image to obtain a real-time registered image.

[0032] Optionally, the lesion identification unit acquires real-time lesion information, including:

[0033] A pre-trained deep neural network model is used to identify lesions in the registered image in order to obtain real-time lesion information.

[0034] Optionally, the lesion identification unit acquires real-time lesion information, including:

[0035] A pre-trained deep neural network model is used to segment the preoperative three-dimensional medical image to obtain segmented images;

[0036] The segmented image is fused with the first real-time registered image to obtain a second real-time fused image;

[0037] The segmented image is fused with the second real-time registered image to obtain a third real-time fused image;

[0038] The second real-time fused image is fused with the third real-time fused image to obtain a fourth real-time fused image;

[0039] Real-time lesion information is obtained based on the fourth real-time fused image.

[0040] Optionally, the real-time lesion information includes real-time lesion location information. Further optionally, the real-time lesion information also includes one or more of the following: real-time lesion volume information, real-time lesion shape information, and real-time key organ / tissue information.

[0041] Optionally, the automatic planning module obtains the target surgical path, including:

[0042] Based on the real-time lesion information, obtain the location information of at least one lesion target point and multiple puncture points;

[0043] Each puncture point is evaluated according to pre-set conditions to obtain the target puncture point;

[0044] Connect the corresponding lesion target point and the target puncture point to obtain the target puncture path.

[0045] Optionally, obtaining the location information of at least one lesion target and multiple puncture points based on the real-time lesion information includes:

[0046] Obtain the spatial mapping relationship between the image coordinate system and the surgical equipment coordinate system;

[0047] Based on the spatial mapping relationship and the real-time lesion information, the real-time location information of the lesion in the coordinate system of the surgical device is obtained;

[0048] Based on the real-time location information of the lesion in the coordinate system of the surgical device, the location information of at least one lesion target point and multiple puncture points in the coordinate system of the surgical device is obtained.

[0049] Optionally, the step of evaluating each of the puncture points according to preset conditions to obtain the target puncture point includes:

[0050] Step A: Score each puncture point according to the pre-set scoring criteria, and take the puncture point with the highest score as the target puncture point;

[0051] Step B: Determine whether the target puncture point can cover all lesions;

[0052] If not, proceed to step C;

[0053] Step C: Score each non-target puncture point according to the pre-set scoring criteria, and take the non-target puncture point with the highest score as the target puncture point.

[0054] Step D: Determine whether all the target puncture points can collectively cover all lesions;

[0055] If not, repeat steps C and D until all the target puncture points can collectively cover the lesion.

[0056] Optionally, the step of scoring each puncture point according to a pre-set scoring criterion and selecting the puncture point with the highest score as the target puncture point includes:

[0057] Each puncture point is scored according to a pre-set set of multiple scoring criteria to obtain scores for each puncture point.

[0058] Based on the weights corresponding to the pre-set scoring criteria, the comprehensive score of each puncture point is calculated.

[0059] The puncture point with the highest overall score will be used as the target puncture point.

[0060] The step of scoring each non-target puncture point according to a pre-set scoring criterion, and selecting the non-target puncture point with the highest score as the target puncture point, includes:

[0061] Each non-target puncture point is scored according to a pre-set set of multiple scoring criteria to obtain the scores for each non-target puncture point.

[0062] Calculate the comprehensive score for each non-target puncture point based on the weights corresponding to the pre-set scoring criteria.

[0063] The non-target puncture point with the highest overall score will be used as the target puncture point.

[0064] Optionally, the control device further includes a functional safety module that is communicatively connected to the lesion recognition module. The functional safety module is used to monitor the real-time motion trajectory of the surgical device based on the registration image output by the image registration unit.

[0065] Optionally, the functional safety module is further configured to obtain safe operation boundary information based on the real-time lesion information, and determine whether the real-time movement trajectory of the surgical device exceeds the safe operation boundary area based on the safe operation boundary information.

[0066] Optionally, the automatic planning module is also used to plan surgical operation parameters based on the real-time lesion information.

[0067] Optionally, the surgical device includes a drive unit and surgical instruments, the surgical instruments being mounted on the drive unit, and the control module being used to control the drive unit to drive the surgical instruments to perform surgery according to the acquired surgical operation parameters and the target surgical path.

[0068] Optionally, the drive unit is a robotic arm, and the end of the robotic arm is equipped with a fixator for fixing the surgical instrument.

[0069] Optionally, the surgical instrument is an instrument used to perform a puncture procedure;

[0070] The automatic planning module is used to plan the puncture path based on the real-time lesion information in order to obtain the target puncture path.

[0071] The control module is used to control the surgical device to perform puncture surgery based on the acquired surgical operation parameters and the target puncture path.

[0072] Optionally, the surgical instrument is a cryoablation needle, and the surgical equipment further includes a refrigeration device. The control module is used to control the refrigeration device to provide a cold source to the cryoablation needle according to the surgical operation parameters.

[0073] Optionally, the surgical operation parameters include freezing time, number of freezing cycles, and freezing dose.

[0074] Optionally, the surgical system further includes a human-computer interaction module that is communicatively connected to the control device, the human-computer interaction module being used for data display and interaction.

[0075] Optionally, the control device further includes a data storage module for storing and managing data.

[0076] Optionally, the first image acquisition device is an ultrasound machine, and the surgical system further includes a support, the support including a base and a first fixing device and a second fixing device mounted on the base, the first fixing device being used to fix the ultrasound probe of the ultrasound machine, the second fixing device being used to fix the head cover of the ultrasound machine, and the first fixing device being able to move closer to and further away from the second fixing device.

[0077] Compared with the prior art, the surgical system provided by the present invention has the following advantages:

[0078] (1) The surgical system provided by the present invention plans the surgical path based on the real-time lesion information obtained, thereby ensuring the real-time nature of the planned target surgical path, which is more conducive to the subsequent surgical execution stage to more accurately eliminate the lesion. Compared with the surgical system in the prior art, the present invention can not only ensure the accuracy of the operation and avoid the many risks brought about by relying entirely on the doctor's clinical experience in the surgical process, but also effectively reduce the doctor's workload and improve the doctor's work efficiency, so that the doctor can devote more energy to the analysis of the condition and the optimization of the treatment plan.

[0079] (2) The lesion identification module in this invention acquires preoperative medical images and intraoperative real-time medical images, and registers the preoperative medical images and the intraoperative real-time medical images to obtain real-time registered images, thereby obtaining high-definition intraoperative real-time images. Based on the real-time registered images, real-time lesion information is obtained. This not only effectively improves the accuracy of lesion identification and reduces the workload of doctors, laying a good foundation for the subsequent surgical path planning stage and surgical execution stage, but also ensures that the acquired lesion information is synchronized with the surgical time. This solves the problem in the prior art that the current lesion cannot be fully displayed because the imaging time precedes the surgical time, which is more conducive to the elimination of lesions.

[0080] (3) The control device in this invention also includes a functional safety module. The functional safety module can monitor the real-time movement trajectory of the surgical equipment during the surgical procedure based on the real-time registered image. Since the real-time registered image is high-definition and accurate, it can achieve rapid and accurate monitoring of the surgical equipment. Compared with the prior art, the monitoring of the surgical equipment in this invention is real-time and can avoid deviation from the actual surgical path. At the same time, the functional safety module can obtain safe operation boundary information based on the real-time lesion information, ensuring the accuracy of the safety boundary, thereby greatly improving the safety performance of the surgical system during the surgical procedure.

[0081] (4) Since the surgical system provided by the present invention also includes a human-computer interaction module, the entire surgical process can be displayed in real time through the human-computer interaction module, so that the entire surgical process can be carried out under the full monitoring of the doctor. At the same time, the doctor can also observe real-time images during the operation, thereby enabling the doctor to obtain more surgical information and further reducing the surgical risk. Attached Figure Description

[0082] Figure 1 This is a schematic diagram illustrating an application scenario of the surgical system according to one embodiment of the present invention;

[0083] Figure 2 This is a block diagram of the surgical system according to one embodiment of the present invention;

[0084] Figure 3 This is a schematic diagram of the structure of the bracket according to one embodiment of the present invention;

[0085] Figure 4 This is a schematic diagram of the image registration process in the first embodiment of the present invention;

[0086] Figure 5 This is a schematic diagram of the image registration process in the second embodiment of the present invention;

[0087] Figure 6This is a schematic diagram of the image registration process in the third embodiment of the present invention;

[0088] Figure 7 This is a schematic diagram of the process of using a deep neural network model for lesion identification in one embodiment of the present invention;

[0089] Figure 8 This is a schematic diagram of the process of using a deep neural network model for lesion identification in another embodiment of the present invention;

[0090] Figure 9 This is a schematic diagram of the training process of a deep neural network model according to one embodiment of the present invention.

[0091] Figure 10 This is a schematic diagram of the process for planning the puncture path in one embodiment of the present invention.

[0092] Figure 11 This is a schematic diagram of the process for obtaining the target puncture point in one embodiment of the present invention;

[0093] Figure 12 A schematic diagram of the block structure of a surgical system provided in another embodiment of the present invention;

[0094] Figure 13 A schematic diagram illustrating the workflow of a functional safety module provided in one embodiment of the present invention;

[0095] Figure 14 This is a partial structural schematic diagram of a surgical device provided according to an embodiment of the present invention;

[0096] Figure 15 This is a partial structural schematic diagram of a driving unit provided in one embodiment of the present invention;

[0097] Figure 16 A schematic diagram of the structure of the bracket provided in another embodiment of the present invention;

[0098] Figure 17 A schematic diagram of the structure of a refrigeration device provided in one embodiment of the present invention;

[0099] Figure 18 A partial structural schematic diagram of a cryoablation needle provided in one embodiment of the present invention;

[0100] Figure 19 This is a schematic flowchart of a cryoablation process provided in one embodiment of the present invention.

[0101] The accompanying figure is labeled as follows:

[0102] First image acquisition device - 100; Control device - 200; Surgical equipment - 300; Lesion recognition module - 210; Image acquisition unit - 211; Image registration unit - 212; Lesion recognition unit - 213; Automatic planning module - 220; Control module - 230; Second image acquisition device - 400; Drive unit - 310; Surgical instrument - 320; Fixator - 311; Functional safety module - 240; Human-computer interaction module - 600; Data storage module - 250; Bracket - 500; Base - 510; First fastener device - 520; Second fixation device - 530; Ultrasonic probe - 110; Headgear - 120; Scale - 321; Heat insulation coating - 322; Display screen - 323; Refrigeration device - 330; Pre-cooling device - 331; Heat exchange device - 332; Refrigerated gas source - 333; Valve - 334. Detailed Implementation

[0103] The following is in conjunction with the appendix Figures 1 to 19 The surgical system proposed in this invention will be further described in detail below with specific embodiments. The advantages and features of this invention will become clearer from the following description. It should be noted that the accompanying drawings in this invention are in a very simplified form and use non-precise proportions, only for the purpose of conveniently and clearly illustrating the embodiments of this invention. Please refer to the accompanying drawings to make the objectives, features, and advantages of this invention more apparent and understandable. It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed in the specification, and are not intended to limit the implementation conditions of this invention. Any modifications to the structure, changes in proportions, or adjustments to the size, if they are the same as or similar to the effects and objectives achieved by this invention, should still fall within the scope of the technical content disclosed in this invention. Specific design features of the invention disclosed herein, including, for example, specific dimensions, orientations, positions, and shapes, will be determined in part by the specific application and usage environment. Furthermore, in the embodiments described below, sometimes the same reference numerals are used together in different drawings to indicate the same parts or parts with the same function, and repeated descriptions are omitted. In this specification, similar reference numerals and letters are used to denote similar items; therefore, once an item is defined in one figure, it need not be discussed further in subsequent figures. Furthermore, if the methods described herein involve a series of steps, and the order of these steps presented herein is not necessarily the only possible order in which they can be performed, some of the described steps may be omitted and / or other steps not described herein may be added to the method.

[0104] Furthermore, it should be noted that in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0105] The main objective of this invention is to provide a surgical system for performing automated surgery, which has the advantages of accuracy, reliability, safety, and efficiency. It can solve the problem in the prior art where doctors rely on preoperative images to determine lesions and then plan the surgical path and determine the surgical operation parameters based on clinical experience. The surgical process is entirely dependent on the doctor's clinical experience, which poses numerous risks.

[0106] To achieve the above objectives, the present invention provides a surgical system, please refer to... Figure 1 and Figure 2 ,in, Figure 1 A schematic diagram illustrating an application scenario of the surgical system provided by one embodiment of the present invention is provided. Figure 2 A schematic block diagram of a surgical system according to an embodiment of the present invention is shown. Figure 1 and Figure 2 As shown, the surgical system includes a control device 200 and a surgical device 300, the surgical device 300 being communicatively connected to the control device 200. Optionally, the surgical system further includes a first image acquisition device 100. The first image acquisition device 100 is communicatively connected to the control device 200 and is used to acquire real-time medical images during surgery. Specifically, the first image acquisition device 100 can be an ultrasound machine, a CT scanner, or an MR device. The ultrasound machine can acquire real-time ultrasound images during surgery, the CT scanner can acquire real-time CT images during surgery, and the MR device can acquire real-time MRI images during surgery. Of course, as those skilled in the art will understand, the first image acquisition device can also be other imaging devices capable of acquiring medical images besides ultrasound machines, CT scanners, and MR devices; this invention does not limit this.

[0107] Please refer to Figure 3 The diagram illustrates the structure of the support provided in one embodiment of the present invention. Figure 3As shown, when the first image acquisition device 100 is an ultrasound machine, the surgical system further includes a support 500. The support 500 includes a base 510 and a first fixing device 520 and a second fixing device 520 mounted on the base 510. The first fixing device 520 is used to fix the ultrasound probe 110 of the ultrasound machine, and the second fixing device 520 is used to fix the head cover 120 of the ultrasound machine. The first fixing device 520 can be close to or away from the second fixing device 530. Therefore, by setting the support 500, the placement of the ultrasound probe 110 and the head cover 120 can be made more convenient. The head cover 120 is used to cover the ultrasound probe 110 when it is in use, thereby effectively preventing cross-infection of bacteria and further improving the safety of the surgical procedure. When it is necessary to use the ultrasound probe 110 to acquire images, the first fixation device 520 can be moved toward the location of the second fixation device 530 to insert the ultrasound probe 110 into the head cover 120. Finally, the ultrasound probe 110 covered with the head cover 120 is inserted into the patient's body to acquire real-time medical images during the operation.

[0108] Specifically, a slide rail can be provided on the base 510. The first fixing device 520 is connected to a driving device, such as a motor. Under the drive of the driving device, the first fixing device 520 can slide along the slide rail to move closer to and away from the second fixing device 530. Of course, as those skilled in the art will understand, in some other embodiments, the first fixing device 520 can also move closer to and away from the second fixing device 530 by other means in the prior art.

[0109] Please continue to refer to this. Figure 1 and Figure 2 ,like Figure 1 and Figure 2As shown, the control device 200 includes a lesion identification module 210, an automatic planning module 220, and a control module 230, all connected in communication. The lesion identification module 210 is used to acquire real-time lesion information based on intraoperative real-time medical images and preoperative medical images. The automatic planning module 220 is used to plan the surgical path based on the real-time lesion information to obtain a target surgical path. The control module 230 is used to control the surgical device 300 to perform surgery based on the acquired surgical operation parameters and the target surgical path. Therefore, the surgical system provided by this invention can automatically acquire real-time lesion information based on intraoperative real-time medical images and preoperative medical images through the lesion identification module 210; automatically plan the surgical path based on the real-time lesion information through the automatic planning module 220, such as planning a puncture surgical path or a cutting surgical path; and automatically control the surgical device 300 to perform surgery based on the planned path (i.e., the target path) through the control module 230, thereby achieving the purpose of eliminating the lesion. As can be seen, the surgical system provided by this invention plans the surgical path based on the acquired real-time lesion information, thereby ensuring the real-time nature of the planned target surgical path. This makes it more conducive to the subsequent surgical execution stage to more accurately eliminate the lesion. Compared with the surgical systems in the prior art, this invention can not only ensure the accuracy of the surgery and avoid the many risks brought about by relying entirely on the doctor's clinical experience in the surgical process, but also effectively reduce the doctor's workload and improve the doctor's work efficiency, so that the doctor can devote more energy to the analysis of the condition and the optimization of the treatment plan.

[0110] Preferably, the real-time lesion information may include real-time lesion location information, real-time lesion volume information, real-time lesion shape information, and real-time critical organ tissue information. It should be noted that, as those skilled in the art will understand, in some embodiments, the real-time lesion information may only include real-time lesion location information; in some embodiments, the real-time lesion information may only include real-time lesion location information and real-time lesion volume information; in some embodiments, the real-time lesion information may include real-time lesion location information, real-time lesion volume information, and real-time lesion shape information; in some embodiments, the real-time lesion information may only include real-time lesion location information and real-time critical organ tissue information. Furthermore, it should be noted that the real-time critical organ tissue information includes information about the organ where the real-time lesion is located and surrounding organs at risk.

[0111] Furthermore, such as Figure 1 and Figure 2As shown, the lesion identification module 210 includes an image acquisition unit 211, an image registration unit 212, and a lesion identification unit 213; the image acquisition unit 211 is used to acquire preoperative medical images and intraoperative real-time medical images; the image registration unit 212 is used to register the preoperative medical images and the intraoperative real-time medical images to obtain real-time registered images; the lesion identification unit 213 is used to obtain real-time lesion information based on the real-time registered images. Therefore, the lesion identification module 210 in this invention acquires preoperative medical images and intraoperative real-time medical images, and registers the preoperative medical images and the intraoperative real-time medical images to obtain real-time registered images, thereby obtaining high-definition intraoperative real-time images. Based on the real-time registered images, real-time lesion information is obtained. This not only effectively improves the accuracy of lesion identification and reduces the workload of doctors, laying a good foundation for the subsequent surgical path planning and surgical execution stages, but also ensures that the acquired real-time lesion information is synchronized with the surgical time. This solves the problem in the prior art where the current lesion cannot be fully displayed because the imaging time precedes the surgical time, which is more conducive to the elimination of lesions.

[0112] Please continue to refer to this. Figure 4 The diagram illustrates the image registration process provided in the first embodiment of the present invention. Figure 4 As shown, when the preoperative medical image is a CT image or an MRI image, and the intraoperative real-time medical image is a CT image or an MRI image, the image registration unit 212 specifically registers the preoperative medical image and the intraoperative real-time medical image through the following process to obtain a real-time registered image:

[0113] Three-dimensional modeling is performed on the preoperative medical images to obtain preoperative three-dimensional medical images;

[0114] Three-dimensional modeling is performed on the intraoperative real-time medical images to obtain intraoperative real-time three-dimensional medical images;

[0115] The preoperative three-dimensional medical image is registered to the intraoperative real-time three-dimensional medical image to obtain a real-time registered image.

[0116] Image registration refers to finding one or more spatial transformations for one image to achieve a spatial match with corresponding points in another image. For medical images, this matching means that the same anatomical point on the human body has the same spatial location in two matched medical images. There are many methods for medical image registration, which can be classified in various ways. For example, based on the different image features selected, they can be divided into registration methods based on internal image features and those based on external image features. Methods based on internal image features include boundary-based methods and voxel similarity-based methods; methods based on external image features include calibration frame methods and skin-surface marking methods. If classified according to the linearity or nonlinearity of the registration geometric transformation, they can be divided into linear registration transformations and nonlinear registration transformations. Linear registration transformations include rigid transformations, affine transformations, and projection transformations; nonlinear registration transformations are what we usually call elastic registration transformations. Specifically, in this embodiment, an elastic registration method can be used. This involves constructing a suitable elastic transformation model and, based on this model, registering the preoperative three-dimensional medical image to the intraoperative real-time three-dimensional medical image to obtain the registered intraoperative real-time three-dimensional medical image, i.e., the registered image. Because elastic registration has stronger local adaptability, more accurate registration can be achieved by using this method. It should be noted that, as those skilled in the art will understand, in other embodiments, other image registration methods besides elastic registration can be used to register the preoperative three-dimensional medical image to the intraoperative real-time three-dimensional medical image, such as rigid transformation registration, affine transformation registration, and projection transformation registration. This invention does not limit the scope of these methods.

[0117] Since the preoperative CT or MRI images acquired in this embodiment are high-resolution images, registering the preoperative three-dimensional medical images to the intraoperative real-time three-dimensional medical images can effectively improve the clarity of the registered intraoperative real-time three-dimensional medical images. This lays a good foundation for subsequent lesion identification and improves the accuracy of lesion identification. Furthermore, this embodiment first performs three-dimensional modeling on the preoperative and intraoperative real-time medical images, and then registers the preoperative three-dimensional medical images to the intraoperative real-time three-dimensional medical images, thereby obtaining a three-dimensional registered image. This three-dimensional registered image can then more comprehensively display real-time lesion information.

[0118] Furthermore, the real-time registered image is the image obtained by fusing the registered preoperative 3D medical image with the intraoperative real-time 3D medical image. Image fusion uses a specific algorithm to combine two or more images into a new image. The fusion result utilizes the spatiotemporal correlation and information complementarity of the two (or more) images, resulting in a more comprehensive and clearer description of the scene. Therefore, this invention obtains a real-time registered image by fusing the registered preoperative 3D medical image with the intraoperative real-time 3D medical image. This real-time registered image incorporates information from both the preoperative and intraoperative 3D medical images, enabling a more comprehensive display of lesions and further improving the accuracy of subsequent lesion identification.

[0119] Please continue to refer to this. Figure 5 The diagram illustrates the image registration process provided by the second embodiment of the present invention. Figure 5 As shown, in this embodiment, the preoperative medical image is a CT image or an MRI image, and the intraoperative real-time medical image is an ultrasound image. The image registration unit 212 specifically registers the preoperative medical image and the intraoperative real-time medical image through the following process to obtain a real-time registered image:

[0120] Three-dimensional modeling is performed on the preoperative medical images to obtain preoperative three-dimensional medical images;

[0121] The preoperative three-dimensional medical image is registered to the intraoperative real-time medical image to obtain a first real-time registered image;

[0122] The intraoperative real-time medical image is registered to the preoperative three-dimensional medical image to obtain a second real-time registered image.

[0123] Therefore, by registering the preoperative three-dimensional medical image to the intraoperative real-time medical image, a local two-dimensional real-time image, namely the first real-time registered image, can be obtained; by registering the intraoperative real-time medical image to the preoperative three-dimensional medical image, a global three-dimensional real-time image, namely the second real-time registered image, can be obtained. Thus, more information about the lesion can be obtained through the first real-time registered image and the second real-time registered image.

[0124] Furthermore, to improve registration efficiency, the step of registering the intraoperative real-time medical image to the preoperative three-dimensional medical image to obtain a second real-time registered image includes:

[0125] Three-dimensional modeling is performed on the intraoperative real-time medical images to obtain intraoperative real-time three-dimensional medical images;

[0126] The intraoperative real-time three-dimensional medical image is registered to the preoperative three-dimensional medical image to obtain a second real-time registered image.

[0127] Therefore, by first performing 3D modeling on the intraoperative real-time medical images to obtain intraoperative real-time 3D medical images, and then registering the intraoperative real-time 3D medical images to the preoperative 3D medical images to obtain a second real-time registered image, the registration efficiency can be greatly improved. Specifically, continuous scanning can be performed using an ultrasound probe to obtain ultrasound images at different scanning layers (i.e., intraoperative real-time medical images), and then 3D modeling can be performed on the ultrasound images at different scanning layers to obtain the intraoperative real-time 3D medical images. It should be noted that, as those skilled in the art will understand, in some other embodiments, the obtained ultrasound images at different scanning layers can also be registered layer by layer to the preoperative 3D medical images to obtain the second real-time registered image.

[0128] Please continue to refer to this. Figure 2 ,like Figure 2 As shown, the surgical system also includes a second image acquisition device 400 that is communicatively connected to the control device. The second image acquisition device 400 is used to acquire real-time images of the patient's skin during the operation.

[0129] Please continue to refer to this. Figure 6 The diagram illustrates the image registration process provided by the third embodiment of the present invention, as shown below. Figure 6 As shown, the image registration unit 212 specifically registers the preoperative medical image and the intraoperative real-time medical image through the following process to obtain a real-time registered image:

[0130] Three-dimensional modeling is performed on the preoperative medical images to obtain preoperative three-dimensional medical images;

[0131] Three-dimensional modeling is performed on the intraoperative real-time medical images to obtain intraoperative real-time three-dimensional medical images;

[0132] Three-dimensional modeling is performed on the real-time intraoperative patient skin images to obtain real-time intraoperative human body model images;

[0133] The preoperative three-dimensional medical image and the intraoperative real-time three-dimensional medical image are registered and fused to obtain a first real-time fused image;

[0134] The first real-time fused image is registered to the intraoperative real-time human model image to obtain a real-time registered image.

[0135] Therefore, in this embodiment, the second image acquisition device 400 acquires real-time intraoperative patient skin images, and the image registration unit 212 performs three-dimensional modeling on the real-time intraoperative patient skin images to obtain a real-time intraoperative human body model image. Then, the first real-time fused image obtained by registering and fusing the preoperative three-dimensional medical image and the real-time intraoperative three-dimensional medical image is registered to the real-time intraoperative human body model image, thereby obtaining a three-dimensional real-time registered image containing the human body contour. During the process of lesion identification in the real-time registered image using the deep neural network model described below, the actual location of the lesion in the human body can be directly output, which is more conducive to the planning of subsequent surgical paths and the execution of related surgeries.

[0136] Specifically, in this embodiment, the preoperative medical image can be a CT image or an MRI image, and the intraoperative real-time medical image can be a CT image, an MRI image, or an ultrasound image. The second image acquisition device 400 can specifically be an optical monitor, in which case the second image acquisition device 400 can acquire real-time intraoperative patient skin images based on optical tracking methods. Of course, as those skilled in the art will understand, the second image acquisition device 400 can also be a binocular camera, in which case the second image acquisition device 400 can acquire real-time intraoperative patient skin images based on the principle of binocular vision measurement.

[0137] Furthermore, the real-time registered image is obtained by fusing the registered intraoperative real-time human model image with the first real-time fused image. Therefore, the real-time registered image obtained at this time simultaneously incorporates information from the preoperative 3D medical image, the intraoperative real-time 3D medical image, and the intraoperative 3D human model image. This allows for a more comprehensive display of lesions in the real-time registered image, further improving the accuracy of subsequent lesion identification. This lays a solid foundation for subsequent automatic surgical path planning and automatic surgical execution, further enhancing the automated surgical effect of the surgical system provided by this invention.

[0138] In one exemplary embodiment, the lesion identification unit 213 identifies lesions in the real-time registered image by employing a pre-trained deep neural network model to obtain real-time lesion information.

[0139] The real-time registration image can be obtained using the registration method provided in the first or third embodiment. Therefore, by employing a pre-trained deep neural network model to identify lesions in the real-time registration image, this invention not only automatically identifies lesions, laying a solid foundation for subsequent automatic surgical path planning and surgical execution, further improving the automated surgical effect of the surgical system provided by this invention, but also effectively reduces the workload of doctors, improves their work efficiency, and allows them to devote more energy to patient condition analysis and treatment plan optimization. Furthermore, by using a pre-trained deep neural network model to identify the real-time registration image, the accuracy and efficiency of lesion identification can be effectively improved. Specifically, different colors and shades can be used in the real-time registration image to display the identified lesions, the organs containing the lesions, and surrounding organs at risk. This setup makes it easier for doctors to observe the location of the lesions.

[0140] Please continue to refer to this. Figure 7 The diagram illustrates a flowchart of a deep neural network model for lesion identification according to an embodiment of the present invention. Figure 7 As shown, in this embodiment, lesion identification can be performed using a neural network model through the following process:

[0141] Step 1: Input the real-time registration image;

[0142] Step 2: Based on the features of the real-time registered image, perform modular division to obtain multiple image modules;

[0143] Step 3: Identify the features of each image module, and reconstruct the image based on the identified features to obtain the corresponding reconstructed image;

[0144] Step 4: Based on the reconstructed image, obtain real-time lesion information and accuracy information through deep learning;

[0145] Step 5: Determine whether the accuracy rate is greater than a preset threshold;

[0146] If so, output the real-time lesion information;

[0147] If not, return to step two and re-identify the lesions until the accuracy rate is greater than the preset threshold.

[0148] In this embodiment, the real-time registered image can be a real-time registered image obtained by using the registration method provided in the first or third embodiment.

[0149] Please continue to refer to this. Figure 8The diagram illustrates a process for identifying lesions using a deep neural network model, as provided in another embodiment of the present invention. Figure 8 As shown, in this embodiment, the lesion identification unit 213 specifically obtains real-time lesion information through the following process:

[0150] A pre-trained deep neural network model is used to segment the preoperative three-dimensional medical image to obtain segmented images;

[0151] The segmented image is fused with the first real-time registered image to obtain a second real-time fused image;

[0152] The segmented image is fused with the second real-time registered image to obtain a third real-time fused image;

[0153] The second real-time fused image is fused with the third real-time fused image to obtain a fourth real-time fused image;

[0154] Real-time lesion information is obtained based on the fourth real-time fused image.

[0155] Therefore, by using a pre-trained deep neural network model to segment the acquired preoperative 3D medical images, segmented images containing the lesion, the organ containing the lesion, and surrounding organs at risk can be obtained. Specifically, different colors and shades can be used to display the lesion, the organ containing the lesion, and surrounding organs at risk. The segmented images are then fused with the first real-time registered image and the second real-time registered image to obtain a second real-time fused image and a third real-time fused image. Finally, the second real-time fused image is fused with the third real-time fused image to output a fourth real-time fused image containing the real-time coordinates of the lesion, the organ containing the lesion, and surrounding organs at risk.

[0156] Please continue to refer to this. Figure 9 The diagram illustrates the training process of a deep neural network model provided in one embodiment of the present invention. Figure 9 As shown, the deep neural network model can be obtained through the following process:

[0157] Obtain training samples;

[0158] The training samples are preprocessed, including region segmentation and annotation of lesions and key organs.

[0159] Based on the idea of ​​clustering, a loss function for a three-dimensional segmentation deep neural network is constructed, and a three-dimensional deep neural network structure is designed.

[0160] The preprocessed training samples are input into the three-dimensional deep neural network structure for training to obtain a deep neural network model.

[0161] The training samples are derived from existing case images. Regarding how to construct a 3D segmentation deep neural network loss function based on clustering, design the 3D deep neural network structure, and how to train the 3D deep neural network structure using preprocessed training samples to obtain the deep neural network model, existing technologies can be consulted, and therefore this invention will not elaborate further.

[0162] Please refer to 10, which schematically illustrates a flowchart of a planned puncture path provided by an embodiment of the present invention. Figure 10 As shown, the automatic planning module 220 specifically plans the puncture surgical path through the following process:

[0163] Based on the real-time lesion information, obtain the location information of at least one lesion target point and multiple puncture points;

[0164] Each puncture point is evaluated according to pre-set conditions to obtain the target puncture point;

[0165] Connect the corresponding lesion target point and the target puncture point to obtain the target puncture path.

[0166] Therefore, based on the target puncture path, the insertion angle and depth of the puncture instrument can be obtained. The control module 230 can automatically control the surgical device 300 to reach the lesion location according to the target puncture path, so as to automatically perform the puncture surgery and achieve the purpose of automatically eliminating the lesion.

[0167] Further, the step of obtaining the location information of at least one lesion target and multiple puncture points based on the real-time lesion information includes:

[0168] Obtain the spatial mapping relationship between the image coordinate system and the surgical equipment coordinate system;

[0169] Based on the spatial mapping relationship and the real-time lesion information, the real-time location information of the lesion in the coordinate system of the surgical device is obtained;

[0170] Based on the location information of the lesion in the coordinate system of the surgical device, obtain the location information of at least one lesion target point and multiple puncture points in the coordinate system of the surgical device.

[0171] Specifically, the image coordinate system is the coordinate system of the real-time registered image or the fourth real-time fused image containing the real-time lesion information. The acquisition of the spatial mapping relationship between coordinate systems can refer to existing methods, and will not be elaborated upon here. Therefore, this invention establishes a spatial mapping relationship between the image coordinate system and the surgical device 300 coordinate system, and based on this spatial mapping relationship, converts the lesion's position information in the image coordinate system into its position information in the surgical device 300 coordinate system. Then, based on the lesion's position information in the surgical device 300 coordinate system, it obtains the position information of at least one lesion target point and multiple puncture points in the surgical device 300 coordinate system. This ensures that the final target puncture path is directly associated with the surgical device 300, enabling the control module 230 to automatically control the surgical device 300 to perform the puncture surgery according to the target puncture path.

[0172] The lesion target point refers to the end point of the planned puncture path, and the location of the obtained lesion target point is the end point of the planned puncture path. The puncture point refers to the starting point of the planned puncture path, and the location of the obtained puncture point is the starting point of the planned puncture path. In actual operation, the number and location of the lesion target points can be set according to the specific situation of the location and number of lesions. For example, when there are multiple lesions, there are also multiple lesion target points, that is, one lesion corresponds to at least one lesion target point. Multiple target points can also be set for the same lesion according to the actual situation such as lesion volume and type. When obtaining the puncture point based on the real-time lesion information, multiple puncture points that uniformly cover the lesion can be set according to the location, volume and other information of the lesion, or they can be set by the doctor based on experience. As one embodiment of the present invention, multiple puncture points can also be selected based on the pre-obtained patient's body surface data. Then, based on the pre-set conditions, each of the puncture points is evaluated to obtain the target puncture point. Finally, the corresponding lesion target point and the target puncture point are connected to obtain the target puncture path.

[0173] Please continue to refer to this. Figure 11 The diagram illustrates a process for obtaining the target puncture point according to an embodiment of the present invention. Figure 11 As shown, the process of evaluating each puncture point according to pre-set conditions to obtain the target puncture point specifically includes the following steps:

[0174] Step A: Score each puncture point according to the pre-set scoring criteria, and take the puncture point with the highest score as the target puncture point;

[0175] Step B: Determine whether the target puncture point can cover all lesions;

[0176] If not, proceed to step C;

[0177] Step C: Score each non-target puncture point according to the pre-set scoring criteria, and take the non-target puncture point with the highest score as the target puncture point.

[0178] Step D: Determine whether all the target puncture points can collectively cover all lesions;

[0179] If not, repeat steps C and D until all the target puncture points can collectively cover the lesion.

[0180] Specifically, if the result of step B is that the target puncture point can cover all lesions, then the selection of the target puncture point ends, and the corresponding lesion target point and the target puncture point are directly connected to obtain the target puncture path. It should be noted that determining whether the target puncture point can cover all lesions means whether the surgical instruments used for puncture can reach all lesions through the target puncture point without touching surrounding organs at risk. Similarly, determining whether all the target puncture points can collectively cover all lesions means whether the surgical instruments used for puncture can reach all lesions through all the target puncture points without touching surrounding organs at risk.

[0181] Further, the step of scoring each puncture point according to a pre-set scoring criterion and selecting the puncture point with the highest score as the target puncture point includes:

[0182] Each puncture point is scored according to a pre-set set of multiple scoring criteria to obtain scores for each puncture point.

[0183] Based on the weights corresponding to the pre-set scoring criteria, the comprehensive score of each puncture point is calculated.

[0184] The puncture point with the highest overall score will be used as the target puncture point.

[0185] The step of scoring each non-target puncture point according to a pre-set scoring criterion, and selecting the non-target puncture point with the highest score as the target puncture point, includes:

[0186] Each non-target puncture point is scored according to a pre-set set of multiple scoring criteria to obtain the scores for each non-target puncture point.

[0187] Calculate the comprehensive score for each non-target puncture point based on the weights corresponding to the pre-set scoring criteria.

[0188] The non-target puncture point with the highest score is selected as the target puncture point.

[0189] Specifically, the scoring criteria include: puncture distance (i.e., the distance between the puncture point and the target lesion), whether the puncture path touches surrounding organs at risk, and the number of target lesions that the puncture point can reach. In practice, the multiple puncture points are first connected to the target lesions one by one to obtain multiple puncture paths. For example, when there are N puncture points and M target lesions, N×M puncture paths can be obtained, meaning each puncture point corresponds to M puncture paths. Next, based on the scoring criteria for the puncture distance of each puncture path, the corresponding puncture point is scored in the first category. The more puncture paths with the shortest puncture distance among the M puncture paths corresponding to the puncture point, the higher the first category score for that puncture point. Then, based on the scoring criteria for whether each puncture path touches surrounding organs at risk, the corresponding puncture point is scored in the second category. The more puncture paths among the M puncture paths corresponding to the puncture point that do not touch surrounding organs at risk, the higher the second category score for that puncture point. Next, based on the scoring criteria for the number of lesion targets that the puncture point can reach, each puncture point is scored in the third category. The more lesion targets that the puncture point can reach, the higher the third category score for that puncture point. Finally, based on the weights corresponding to each scoring criterion, the comprehensive score for each puncture point is calculated, and the puncture point with the highest comprehensive score is selected as the target puncture point. When there are multiple lesions, the target puncture point obtained initially may not cover all lesions. Therefore, it is necessary to reselect target puncture points for the uncovered lesions that were not covered by the initially obtained target puncture point. In practice, firstly, all puncture points other than the initially obtained target puncture point (i.e., non-target puncture points) are connected to the target lesion corresponding to the uncovered lesion to obtain multiple puncture paths. Then, each of the non-target puncture points is scored according to various scoring criteria, and a comprehensive score is obtained for each non-target puncture point. The non-target puncture point with the highest score is selected as the target puncture point. Then, it is determined whether the initially obtained target puncture point and the target puncture point obtained this time can jointly cover all lesions. If not, then all puncture points other than the initially obtained target puncture point are connected to the target lesion corresponding to the uncovered lesion to obtain multiple puncture paths. The remaining non-target puncture points are connected one by one to the target lesions corresponding to the remaining uncovered lesions to obtain multiple puncture paths. Then, each non-target puncture point is scored according to various scoring criteria, and a comprehensive score of each non-target puncture point is obtained. The non-target puncture point with the highest score is taken as the target puncture point. Then, it is determined whether all the target puncture points (i.e., all the target puncture points obtained in the first, second and current times) can cover all the lesions. If not, the above steps are repeated to select new target puncture points until all lesions can be covered.

[0190] To further improve the surgical effect and efficiency of the surgical system provided by this invention, in an exemplary embodiment, the automatic planning module 220 is also used to plan surgical operation parameters based on the real-time lesion information. For example, for cryoablation surgery, information such as the volume and shape of the lesion can be obtained based on the real-time lesion information. Based on the volume and shape of the lesion, parameters such as the cryoablation volume, freezing time, number of freezing cycles, and freezing dose can be automatically set. Thus, by automatically planning surgical operation parameters based on the obtained real-time lesion information, the workload of doctors can be further reduced and surgical efficiency can be improved. In addition, compared with the prior art method of determining surgical operation parameters based on doctors' experience, this invention can further reduce surgical risks. It should be noted that, as those skilled in the art will understand, in some other embodiments, the surgical operation parameters can also be parameters determined manually by doctors based on the obtained real-time lesion information.

[0191] Please continue to refer to this. Figure 12 The diagram illustrates a block structure of a surgical system according to another embodiment of the present invention. Figure 12 As shown, in this embodiment, the surgical system further includes a human-computer interaction module 600 communicatively connected to the control device 200. The human-computer interaction module 600 is used for data display and interaction. The human-computer interaction module 600 may include a display device and interactive software. Thus, the doctor can view the planned surgical path through the display device and adjust the surgical path in real time according to the actual situation. Furthermore, the surgical process can be displayed in real-time in 3D through the display device, and the interactive software can receive control information from the doctor in real time, such as pausing or controlling the surgical process. In addition, the human-computer interaction module 600 can also record and display surgical-related information. Therefore, by setting up the human-computer interaction module 600, the entire surgical process can be conducted entirely under the doctor's monitoring, and the doctor has complete control, able to confirm, interrupt, and modify the surgical process at any time. Simultaneously, the human-computer interaction module 600 also allows the doctor to observe real-time intraoperative images, thereby obtaining more surgical information and further reducing surgical risks.

[0192] Furthermore, such as Figure 2 and Figure 12 As shown, the control device 200 also includes a data storage module 250, which is used for data storage and management. Therefore, the data storage module 250 can store image data, patient data, surgery-related data, and provide data management functions.

[0193] Furthermore, such as Figure 2 and Figure 12As shown, the control device 200 also includes a functional safety module 240 communicatively connected to the lesion recognition module 210. The functional safety module 240 monitors the real-time motion trajectory of the surgical device 300 based on the real-time registered image output by the image registration unit 212. When the real-time motion trajectory of the surgical device 300 deviates significantly from the planned path, an alarm message is output. Specifically, a deviation threshold can be set; when the detected deviation between the real-time motion trajectory and the planned path exceeds the threshold, an alarm message is output. Thus, during the surgical procedure, the functional safety module 240 monitors the real-time motion trajectory of the surgical device 300 to prevent deviations from the actual surgical path. This ensures that the surgical device 300 can be stopped promptly in case of an accident, preventing harm to the patient and guaranteeing the safety performance of the surgical device 300 during the surgical procedure.

[0194] Furthermore, the functional safety module 240 also acquires the real-time lesion information through the lesion identification unit 213 to generate safe operation boundary information. For example, based on information about critical organ tissues, it sets a range of areas that will not damage other tissues (i.e., safe operation boundary areas), and determines whether the real-time movement trajectory of the surgical device 300 exceeds the safe operation boundary area based on the acquired safe operation boundary information. When the real-time movement trajectory of the surgical device 300 touches or exceeds the safe operation boundary area, an alarm message is output.

[0195] Therefore, the control module 230 can control the surgical device 300 to perform surgery based on the acquired surgical operation parameters, the target surgical path, and the safety operation boundary information. The functional safety module 240 can monitor the real-time movement trajectory of the surgical device 300. When the functional safety module 240 detects that the surgical device 300 has encountered the operation boundary, the system will automatically stop operation and issue an alarm to further improve the safety performance during the surgery. For example, in puncture surgery, when the puncture needle encounters the operation boundary, the system will automatically stop operation and automatically withdraw the needle to provide safety assurance.

[0196] Please continue to refer to this. Figure 13 The diagram illustrates the workflow of a functional safety module provided in one embodiment of the present invention. Figure 13As shown, the first image acquisition device 100 can acquire real-time medical images during surgery. The image acquisition unit 211 can acquire the real-time medical images acquired by the first image acquisition device 100 during surgery. The image registration unit 212 can register the preoperative medical images with the real-time medical images during surgery in real time to obtain real-time registered images. The human-computer interaction module 600 can display the real-time registered images, thereby displaying the movement trajectory (e.g., puncture trajectory) of the surgical device 300 in real time. The functional safety module 240 can determine in real time whether the movement trajectory (e.g., puncture trajectory) of the surgical device 300 deviates from the target surgical path and whether the movement trajectory (e.g., puncture trajectory) of the surgical device 300 exceeds the safe operating boundary. If the determination result is that the movement trajectory (e.g., puncture trajectory) of the surgical device 300 deviates too much from the target surgical path, or the movement trajectory (e.g., puncture trajectory) of the surgical device 300 exceeds the safe operating boundary, the system will automatically stop and output alarm information. The alarm information can be output through sound, light, or display of alarm information on the human-computer interaction interface.

[0197] Please continue to refer to this. Figure 14 The diagram illustrates a partial structural schematic of the surgical device 300 provided in one embodiment of the present invention. Figure 14 As shown, the surgical device 300 includes a drive unit 310 and a surgical instrument 320. The surgical instrument 320 is mounted on the drive unit 310. The control module 230 is used to control the drive unit 310 to drive the surgical instrument 320 to perform surgery according to the acquired surgical operation parameters and the target surgical path.

[0198] For details, please refer to Figure 15 The diagram illustrates a partial structural schematic of a driving unit provided in one embodiment of the present invention. Figure 14 and Figure 15 As shown, the drive unit 310 can be a robotic arm, and the end of the robotic arm is equipped with a retainer 311 for fixing the surgical instrument 320. For example, when the surgical instrument 320 is a puncture instrument, the retainer 311 is a puncture device, which can hold the puncture instrument. Therefore, by driving the surgical instrument 320 with a robotic arm to perform surgery, the surgical risk can be further reduced and the safety of the surgical procedure improved. It should be noted that, as those skilled in the art will understand, in some other embodiments, the drive unit 310 can also be other automated devices besides a robotic arm, and the present invention does not limit this.

[0199] Please continue to refer to this. Figure 16The diagram illustrates the structure of a support provided in another embodiment of the present invention. Figure 16 As shown, in this embodiment, in addition to a first fixing device 520 for fixing the ultrasound probe 110 of the ultrasound machine and a second fixing device 530 for fixing the head cover 120 of the ultrasound machine, the bracket 500 also has a fixator 311 for fixing the surgical instrument 320, such as a puncture device for fixing a puncture instrument. In actual use, the ultrasound probe 110 and the surgical instrument 320 can be fixed together on the bracket 500, and then the bracket 500 can be mounted on the drive unit 310, such as a robotic arm. Thus, the drive unit 310 can simultaneously control the ultrasound probe 110 and the surgical instrument 320, thereby making operation more convenient.

[0200] Furthermore, the surgical instrument 320 can be an instrument for performing puncture surgery, such as a biopsy needle for performing biopsy puncture surgery or a cryoablation needle for performing ablation surgery. Of course, as those skilled in the art will understand, the surgical instrument 320 can also be other instruments besides those for performing puncture surgery, and the present invention does not limit this.

[0201] When the surgical instrument 320 is used to perform a puncture procedure, i.e., when the surgical device 300 is used to perform a puncture procedure, the automatic planning module 220 is used to plan the puncture path based on the real-time lesion information to obtain a target puncture path; the control module 230 is used to control the surgical device 300 to perform the puncture procedure based on the obtained surgical operation parameters and the target puncture path. Regarding how the automatic planning module 220 plans the puncture path based on the real-time lesion information, please refer to the relevant content in the surgical path planning section above, and therefore will not be elaborated further.

[0202] When the surgical instrument 320 is a cryoablation needle, such as Figure 2 and Figure 12 As shown, the surgical device 300 also includes a cooling device 330. The control module 230 is used to control the cooling device 330 to provide a cold source to the cryoablation needle according to the surgical operation parameters, thereby enabling the cryoablation needle to perform cryoablation surgery according to the required surgical operation parameters, such as freezing time, number of freezing cycles, and freezing dosage. Thus, by controlling the cooling device 330 to provide a cold source to the cryoablation needle according to the surgical operation parameters, the purpose of autonomously eliminating lesions can be achieved.

[0203] Please refer to Figure 17 The diagram illustrates the structure of a refrigeration device according to an embodiment of the present invention.Figure 17 As shown, the refrigeration device 330 includes a pre-cooling device 331, a heat exchange device 332, and a refrigerant gas source 333. Both the pre-cooling device 331 and the heat exchange device 332 are connected to the refrigerant gas source 333. This allows for better control of the temperature and flow rate of the gas flowing into the cryoablation needle. A valve 334 is installed on the pipeline connecting the heat exchange device 332 and the refrigerant gas source 333, allowing for control of the gas flow rate through the heat exchange device 332. The working principles of the pre-cooling device 331 and the heat exchange device 332 can be found in existing technology and will not be elaborated upon here.

[0204] Please continue to refer to this. Figure 18 The diagram illustrates a partial structural schematic of a cryoablation needle provided in one embodiment of the present invention. Figure 18 As shown, the cryoablation needle is equipped with a scale 321, a heat-insulating coating 322, a temperature sensor (not shown), and a display screen 323. Specifically, the scale 321 is located on the outer surface of the cryoablation needle, thereby allowing for precise adjustment of the insertion depth. The heat-insulating coating 322 is located on the inner surface of the cryoablation needle, except for the tip (the end closest to the patient). This effectively prevents frostbite to normal tissues other than the lesion during treatment, improving the safety of the procedure. The temperature sensor is located at the tip of the cryoablation needle, allowing for real-time monitoring of the tip's temperature. The display screen 323 is located at the needle-holding end of the cryoablation needle (the end closest to the operator (i.e., the doctor)). The display screen 323 can display the results measured by the temperature sensor in real time, so that the doctor can monitor the current temperature of the tip of the cryoablation needle in real time, thereby further improving the cryoablation effect.

[0205] Please continue to refer to this. Figure 19 The diagram illustrates a flow chart of the cryoablation process provided in one embodiment of the present invention. Figure 19 As shown, the cryoablation process includes the following steps:

[0206] The cryoablation needle is inserted into the lesion along the target puncture path;

[0207] When the freezing mode is activated, the ice ball formed at the tip of the needle gradually increases in size.

[0208] Continuous cryoablation caused the ice ball to enlarge and cover the lesion;

[0209] The reheating mode is activated, and the ice puck melts.

[0210] Determine if the required number of freezing cycles has been reached;

[0211] If so, the cryoablation needle is withdrawn, and the procedure is completed;

[0212] If not, restart the freeze mode.

[0213] In summary, compared with the prior art, the surgical system provided by the present invention has the following advantages:

[0214] (1) The surgical system provided by the present invention can automatically acquire real-time lesion information based on intraoperative and preoperative medical images through the lesion identification module; automatically plan the surgical path based on the real-time lesion information through the automatic planning module; and automatically control the surgical equipment to perform surgery based on the planned real-time surgical path (i.e., the target path) through the control module, so as to achieve the purpose of eliminating lesions. It can be seen that the surgical system provided by the present invention can ensure the real-time nature of the planned target surgical path by planning the surgical path based on the acquired real-time lesion information, which is more conducive to the accurate elimination of lesions in the subsequent surgical execution stage. Compared with the surgical system in the prior art, the present invention can not only ensure the accuracy of the surgery and avoid the many risks brought about by relying entirely on the doctor's clinical experience in the surgical process, but also effectively reduce the workload of doctors and improve their work efficiency, so that doctors can devote more energy to the analysis of the condition and the optimization of the treatment plan.

[0215] (2) The lesion identification module in this invention acquires preoperative medical images and intraoperative real-time medical images, and registers the preoperative medical images and the intraoperative real-time medical images to obtain real-time registered images, thereby obtaining high-definition intraoperative real-time images. Based on the real-time registered images, real-time lesion information is obtained. This not only effectively improves the accuracy of lesion identification and reduces the workload of doctors, laying a good foundation for the subsequent surgical path planning stage and surgical execution stage, but also ensures that the acquired lesion information is synchronized with the surgical time. This solves the problem in the prior art that the current lesion cannot be fully displayed because the imaging time precedes the surgical time, which is more conducive to the elimination of lesions.

[0216] (3) Since the control device in this invention also includes a functional safety module, the functional safety module can monitor the real-time movement trajectory of the surgical equipment during the surgical procedure based on the real-time registered image. Since the real-time registered image is high-definition and accurate, it can achieve rapid and accurate monitoring of the surgical equipment. Compared with the prior art, the monitoring of the surgical equipment in this invention is real-time and can avoid deviation from the actual surgical path. At the same time, the functional safety module can obtain safe operation boundary information based on the real-time lesion information, ensuring the accuracy of the safety boundary, thereby greatly improving the safety performance of the surgical system during the surgical procedure.

[0217] (4) Since the surgical system provided by the present invention also includes a human-computer interaction module, the entire surgical process can be displayed in real time through the human-computer interaction module, so that the entire surgical process can be carried out under the full monitoring of the doctor. At the same time, the doctor can also observe real-time images during the operation, thereby enabling the doctor to obtain more surgical information and further reducing the surgical risk.

[0218] It should be noted that the apparatus and methods disclosed in the embodiments herein can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments herein. In this regard, each block in a flowchart or block diagram may represent a module, program, or part of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system to perform the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0219] In addition, the functional modules in the various embodiments of this article can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0220] Furthermore, in the description of this specification, the reference to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., means that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0221] In summary, the above embodiments have provided a detailed description of the lesion identification method, surgical path planning method, storage medium, and surgical system proposed in this invention. Of course, the above description is merely a description of preferred embodiments of this invention and is not intended to limit the scope of this invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure are within the protection scope of this invention. Obviously, those skilled in the art can make various modifications and variations to this invention without departing from the spirit and scope of this invention. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A surgical system, characterized in that, It includes a control device, a first image acquisition device, a second image acquisition device, and surgical equipment, wherein the surgical equipment, the first image acquisition device, and the second image acquisition device are all communicatively connected to the control device; The first image acquisition device is used to acquire real-time medical images during surgery; The second image acquisition device is used to acquire real-time images of the patient's skin during surgery; The control device includes a lesion identification module, an automatic planning module, and a control module that are connected in communication. The lesion identification module includes an image acquisition unit, an image registration unit, and a lesion identification unit that are connected in communication. The image acquisition unit is used to acquire preoperative medical images, intraoperative real-time medical images, and intraoperative real-time patient skin images; The image registration unit is used to register the preoperative medical image and the intraoperative real-time medical image to obtain a real-time registered image; The lesion identification unit is used to obtain real-time lesion information based on the real-time registered image. The real-time lesion information includes one or more of the following: real-time lesion location information, real-time lesion volume information, real-time lesion shape information, and real-time key organ and tissue information. The automatic planning module is used to plan the surgical path based on the real-time lesion information to obtain the target surgical path, and to plan the surgical operation parameters based on the real-time lesion information. The control module is used to control the surgical equipment to perform surgery based on the acquired surgical operation parameters and the target surgical path; The image registration unit acquires real-time registered images, including: Three-dimensional modeling is performed on the preoperative medical images to obtain preoperative three-dimensional medical images; Three-dimensional modeling is performed on the intraoperative real-time medical images to obtain intraoperative real-time three-dimensional medical images; Three-dimensional modeling is performed on the real-time intraoperative patient skin images to obtain real-time intraoperative human body model images; The preoperative three-dimensional medical image and the intraoperative real-time three-dimensional medical image are registered and fused to obtain a first real-time fused image; The first real-time fused image is registered to the intraoperative real-time human model image to obtain a real-time registered image.

2. The surgical system according to claim 1, characterized in that, The lesion identification unit acquires real-time lesion information, including: A pre-trained deep neural network model is used to identify lesions in the real-time registered images in order to obtain real-time lesion information.

3. The surgical system according to claim 1, characterized in that, The automatic planning module obtains the target surgical path, including: Based on the real-time lesion information, obtain the location information of at least one lesion target point and multiple puncture points; Each puncture point is evaluated according to pre-set conditions to obtain the target puncture point; Connect the corresponding lesion target point and the target puncture point to obtain the target puncture path.

4. The surgical system according to claim 3, characterized in that, The step of obtaining the location information of at least one lesion target point and multiple puncture points based on the real-time lesion information includes: Obtain the spatial mapping relationship between the image coordinate system and the surgical equipment coordinate system; Based on the spatial mapping relationship and the real-time lesion information, the real-time location information of the lesion in the coordinate system of the surgical device is obtained; Based on the real-time location information of the lesion in the coordinate system of the surgical device, the location information of at least one lesion target point and multiple puncture points in the coordinate system of the surgical device is obtained.

5. The surgical system according to claim 3, characterized in that, The step of evaluating each puncture point according to pre-set conditions to obtain the target puncture point includes: Step A: Score each puncture point according to the pre-set scoring criteria, and take the puncture point with the highest score as the target puncture point; Step B: Determine whether the target puncture point can cover all lesions; If not, proceed to step C; Step C: Score each non-target puncture point according to the pre-set scoring criteria, and take the non-target puncture point with the highest score as the target puncture point. Step D: Determine whether all the target puncture points can collectively cover all lesions; If not, repeat steps C and D until all the target puncture points can collectively cover the lesion.

6. The surgical system according to claim 5, characterized in that, The step of scoring each puncture point according to a pre-set scoring criterion and selecting the puncture point with the highest score as the target puncture point includes: Each puncture point is scored according to a pre-set set of multiple scoring criteria to obtain scores for each puncture point. Based on the weights corresponding to the pre-set scoring criteria, the comprehensive score of each puncture point is calculated. The puncture point with the highest overall score will be used as the target puncture point. The step of scoring each non-target puncture point according to a pre-set scoring criterion, and selecting the non-target puncture point with the highest score as the target puncture point, includes: Each non-target puncture point is scored according to a pre-set set of multiple scoring criteria to obtain the scores for each non-target puncture point. Calculate the comprehensive score for each non-target puncture point based on the weights corresponding to the pre-set scoring criteria. The non-target puncture point with the highest overall score will be used as the target puncture point.

7. The surgical system according to claim 1, characterized in that, The control device also includes a functional safety module that is communicatively connected to the lesion recognition module. The functional safety module is used to monitor the real-time movement trajectory of the surgical device based on the real-time registered image output by the image registration unit.

8. The surgical system according to claim 7, characterized in that, The functional safety module is also used to obtain safe operation boundary information based on the real-time lesion information, and to determine whether the real-time movement trajectory of the surgical device exceeds the safe operation boundary area based on the safe operation boundary information.

9. The surgical system according to claim 1, characterized in that, The surgical device includes a drive unit and surgical instruments. The surgical instruments are mounted on the drive unit. The control module is used to control the drive unit to drive the surgical instruments to perform surgery according to the acquired surgical operation parameters and the target surgical path.

10. The surgical system according to claim 9, characterized in that, The drive unit is a robotic arm, and a fixator for fixing the surgical instrument is installed at the end of the robotic arm.

11. The surgical system according to claim 10, characterized in that, The surgical instruments are instruments used to perform puncture surgery; The automatic planning module is used to plan the puncture path based on the real-time lesion information in order to obtain the target puncture path. The control module is used to control the surgical device to perform puncture surgery based on the acquired surgical operation parameters and the target puncture path.

12. The surgical system according to claim 11, characterized in that, The surgical instrument is a cryoablation needle, and the surgical equipment also includes a refrigeration device. The control module is used to control the refrigeration device to provide a cold source to the cryoablation needle according to the surgical operation parameters.

13. The surgical system according to claim 12, characterized in that, The surgical parameters include freezing time, number of freezing cycles, and freezing dose.

14. The surgical system according to claim 1, characterized in that, The surgical system also includes a human-computer interaction module that is communicatively connected to the control device. The human-computer interaction module is used for data display and interaction.

15. The surgical system according to claim 1, characterized in that, The control device also includes a data storage module, which is used for data storage and management.

16. The surgical system according to claim 1, characterized in that, The first image acquisition device is an ultrasound machine, and the surgical system also includes a support. The support includes a base and a first fixing device and a second fixing device mounted on the base. The first fixing device is used to fix the ultrasound probe of the ultrasound machine, and the second fixing device is used to fix the head cover of the ultrasound machine. The first fixing device can move closer to and further away from the second fixing device.

Images