Tumor ablation puncture operation planning method and system

Abstract

The invention discloses a tumor ablation puncture operation planning method and system, and relates to the technical field of medical image processing, and the method comprises the steps: obtaining multi-modal image data of a tumor region, carrying out the preprocessing, carrying out the fusion processing of the preprocessed multi-modal image data through an image fusion algorithm, generating a fusion image, and carrying out the planning of a tumor ablation puncture operation based on the fusion image. A tumor area is automatically marked by using an image segmentation technology, a doctor manually corrects a segmentation result, an optimal puncture path is selected through design and simulation according to a fusion image with a tumor mark, a model and a tumor entity position are accurately aligned by using a space positioning technology based on the fusion image and augmented reality equipment, and an AR view is generated. And under the guidance of the AR view, the puncture needle carries out ablation operation according to the optimal puncture path. Through high-quality images and three-dimensional guidance, high precision and visualization of the whole process from image acquisition to operation implementation are achieved, and the safety and success rate of the operation are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method and system for planning tumor ablation puncture surgery. Background Art

[0002] In recent years, with the rapid development of medical imaging technology and computer science technology, multimodal image fusion technology has played an increasingly important role in tumor diagnosis and treatment. Traditional tumor ablation puncture surgery mainly relies on single-modality imaging data, such as CT or MRI, but due to their respective limitations, such as CT's lack of soft tissue contrast and MRI's lack of hard tissue details, single-modality imaging is difficult to provide comprehensive information support.

[0003] The current puncture path planning mainly relies on the doctor's experience and lacks accurate simulation tools, making it impossible to effectively evaluate and select multiple paths before surgery. In traditional surgeries, doctors mainly rely on two-dimensional images for puncture operations and lack intuitive three-dimensional view guidance, which limits puncture accuracy and increases surgical risks. The present invention improves the accuracy and safety of surgery through technical means such as multimodal image fusion, automatic segmentation and correction, optimal puncture path simulation, and AR technology to guide the puncture needle to perform ablation operations. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a tumor ablation puncture surgery planning method and system to solve the problem of lack of precise guidance in puncture path planning.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a method for planning a tumor ablation puncture surgery, which comprises:

[0008] Acquire multimodal imaging data of the tumor area and perform preprocessing;

[0009] The pre-processed multi-modal image data is fused using an image fusion algorithm to generate a fused image;

[0010] Based on the fused image, the image segmentation technology is used to automatically mark the tumor area, and the doctor manually corrects the segmentation result;

[0011] According to the fused image with tumor annotation, the best puncture path is selected through design and simulation;

[0012] Based on the fused image and augmented reality device, the model is accurately aligned with the tumor entity position using spatial positioning technology to generate an AR view;

[0013] Guided by the AR view, the puncture needle performs the ablation operation along the optimal puncture path.

[0014] As a preferred solution of the tumor ablation puncture surgery planning method of the present invention, wherein: the multimodal image data of the tumor area is obtained and preprocessed, and the specific steps are as follows:

[0015] Scanning the patient's tumor area using a CT scanner and an MRI scanner to obtain multimodal imaging data of the tumor area;

[0016] Use a CT scanner and an MRI scanner to simultaneously scan the patient's tumor area to obtain multimodal imaging data of the tumor area;

[0017] Standardize multimodal imaging data and dynamically adjust standardization parameters based on local characteristics of the image;

[0018] The multimodal imaging data are registered and the CT and MRI images are registered to the same coordinate system using a rigid transformation matrix.

[0019] As a preferred solution of the tumor ablation puncture surgery planning method of the present invention, wherein: the pre-processed multi-modal image data is fused using an image fusion algorithm to generate a fused image, and the specific steps are as follows:

[0020] The Laplacian pyramid fusion method in the fusion algorithm based on multi-resolution analysis is selected to decompose the CT image and MRI image into sub-bands with different resolutions;

[0021] Through the downsampling operation, the original image l0 is reduced by half to obtain l1, and the downsampling operation is repeated on the reduced image to form a series of images {l0,l1....l i};

[0022] For {l0,l1....l i}, the subband L in the Laplace pyramid is calculated by the difference between two adjacent layers of images i , i is the index variable, generating a series of subbands {L0,L1....L i-1}, each sub-band represents high-frequency information at different resolutions;

[0023] Fusion is performed on each subband. For each pixel position (x, y), the following fusion rule is selected:

[0024]

[0025] Among them, F i(x, y) is the pixel value of the fused i-th layer image at position (x, y), is the pixel value of the CT image of the i-th layer at position (x, y), is the pixel value of the MRI image of the i-th layer at position (x, y); and They are and The weight of

[0026] The fused sub-bands are reconstructed back to the fused image of the original resolution through the inverse Laplacian pyramid, starting from the sub-band with the lowest resolution and gradually reconstructed upwards. At each step, the current sub-band is merged with the reconstructed image of the previous layer to obtain the final fused image.

[0027] As a preferred solution of the tumor ablation puncture surgery planning method of the present invention, wherein: based on the fusion image, the tumor area is automatically marked using image segmentation technology, and the doctor manually corrects the segmentation result. The specific steps are as follows:

[0028] The standard deviation of the Gaussian filter is adjusted according to the characteristics and noise level of the image, and multiple Gaussian filters with different standard deviations are used to denoise the fused image;

[0029] The image is divided into several small areas, each small area is subjected to histogram equalization processing, and the processed small area images are reassembled into a complete image;

[0030] Select a U-Net model and extract a training data set from the fused image to train the U-Net model's ability to identify tumor boundaries;

[0031] Add self-attention mechanism to the encoder and decoder parts of the U-Net model to calculate the importance weights of feature maps;

[0032] The preprocessed fused image is input into the trained U-Net model, and the preliminary segmentation result is output, which is a binary image representing the tumor area and non-tumor area;

[0033] Doctors viewed the preliminary segmentation results and original images through visualization software and manually corrected the segmentation results using interactive tools;

[0034] Using active learning methods, the most informative samples are selected from the doctors' correction results through diversity sampling, the selected samples are added to the training set, the U-Net model is retrained, and the performance of the model is gradually improved;

[0035] The revised segmentation results are integrated into the patient's medical records as a basis for subsequent treatment.

[0036] As a preferred solution of the tumor ablation puncture surgery planning method of the present invention, wherein: the optimal puncture path is selected through design and simulation based on the fused image with tumor annotation, and the specific steps are as follows:

[0037] Convert the two-dimensional images of tumors segmented by the U-Net model and corrected by doctors into VTK format, a data format suitable for three-dimensional modeling;

[0038] The segmented two-dimensional images are stacked into three-dimensional voxel data, each voxel represents a pixel point in the image, and its pixel value indicates the tissue type at its location;

[0039] A three-dimensional density field is constructed from the voxel data, where the position and value of each voxel is used to represent the density distribution of the tissue;

[0040] The three-dimensional density field is divided into several cubes with 8 vertices, and the density values ​​of the 8 vertices are calculated. According to the density values ​​of these vertices, the intersection points of the isosurface and the cube edge are determined;

[0041] According to the intersection points of the isosurface and the cube edge, triangular patches are generated, which constitute the surface of the three-dimensional model, and the generated triangular patches are smoothed;

[0042] The Marching Cubes algorithm constructs an intuitive and interactive three-dimensional environment by finding isosurfaces in a three-dimensional density field. The isosurface is a surface composed of all points whose density value is equal to a certain threshold.

[0043] In the three-dimensional model, the doctor determines the starting and ending points of the puncture based on the tumor location and the actual location of the tumor;

[0044] Using the path planning algorithm, the optimal path from the puncture start point to the end point is found in the three-dimensional model under the conditions of avoiding important tissues, avoiding hard tissue obstacles and maintaining the smoothness of the path;

[0045] By simulating the puncture process in a 3D model, the path is checked for any unforeseen obstacles. If so, the puncture path is replanned.

[0046] As a preferred solution of the tumor ablation puncture surgery planning method of the present invention, wherein: based on the fusion image and augmented reality device, the three-dimensional model is accurately aligned with the tumor entity position using spatial positioning technology to generate an AR view, and the specific steps are as follows:

[0047] Connecting the augmented reality device to the computer and calibrating the augmented reality device;

[0048] The infrared tracking system installed in the operating room enables precise alignment of the 3D model and the tumor entity with the patient’s actual tumor location;

[0049] The infrared tracking system contains multiple cameras, which capture information from reflective markers attached to medical devices and patients, collect marker motion data through sensors on augmented reality devices, and convert it into a coordinate system in the fused image;

[0050] The three-dimensional model in the fused image is aligned with the real scene, and the tumor model, puncture path and real scene are superimposed and displayed on the display to generate an AR view.

[0051] As a preferred solution of the tumor ablation puncture surgery planning method of the present invention, wherein: guided by the AR view, the puncture needle performs the ablation operation along the optimal puncture path, and the specific steps are as follows:

[0052] The doctor uses the AR view to reconfirm the optimal puncture path, fine-tune the path through gesture control, and lock the puncture path;

[0053] Disinfect the patient's puncture site and mark the puncture starting point. Perform local anesthesia on the patient's puncture site based on the patient's weight, age and health condition.

[0054] The size of the puncture needle is selected according to the size and depth of the tumor. Under the guidance of the AR view, the doctor performs the puncture along the optimal puncture path.

[0055] When the ablation device connected to the end of the puncture needle reaches the center of the tumor, the ablation device is activated, and the doctor observes the position of the puncture needle and the changes in the surrounding tissues in real time through the AR view;

[0056] After the ablation is completed, the puncture needle is removed, the puncture site is checked for abnormalities, and the wound at the puncture starting point is treated;

[0057] After surgery, the patient's heart rate, blood pressure, and blood oxygen saturation were monitored and recorded, and the recovery of the patient's puncture wound was observed;

[0058] An imaging review will be conducted 24 hours after surgery, and the results will be combined with the patient's postoperative clinical manifestations to evaluate the tumor ablation effect;

[0059] Based on the evaluation results, the next treatment plan is formulated.

[0060] In a second aspect, the present invention provides a tumor ablation puncture surgery planning system, including an image acquisition and processing module, an image fusion module, a segmentation and correction module, a puncture path planning module, an AR view generation module and an ablation operation execution module;

[0061] The image acquisition and processing module is used to acquire multimodal image data of the tumor area and perform preprocessing;

[0062] The image fusion module is used to fuse the pre-processed multi-modal image data using an image fusion algorithm to generate a fused image;

[0063] The segmentation and correction module is used to automatically mark the tumor area based on the fused image using image segmentation technology, and the doctor manually corrects the segmentation result;

[0064] The puncture path planning module is used to select the best puncture path through design and simulation according to the fused image with tumor annotation;

[0065] The AR view generation module is used to generate an AR view by accurately aligning the model with the tumor entity position based on the fused image and the augmented reality device using the spatial positioning technology;

[0066] The ablation operation execution module is used to perform the ablation operation along the optimal puncture path with the puncture needle guided by the AR view.

[0067] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the tumor ablation puncture surgery planning method as described in the first aspect of the present invention is implemented.

[0068] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the tumor ablation puncture surgery planning method as described in the first aspect of the present invention is implemented.

[0069] Through dynamic standardization and registration processing, the consistency and reliability of multimodal data are improved; through the Laplace pyramid fusion method, the multi-scale information of the image is retained, and the clarity and contrast of the image are enhanced; through multi-scale denoising and adaptive histogram equalization, the local contrast of the image is improved, making the tumor area more obvious; through interactive correction and active learning, the model performance is optimized and the accuracy of the segmentation results is improved; through the Marching Cubes algorithm, a three-dimensional model is constructed, which provides an intuitive and interactive three-dimensional environment to help doctors better understand the location of the tumor; through the path planning algorithm, the safety and smoothness of the puncture path are ensured, and the accuracy of the operation is improved; through the AR view to guide the puncture operation, the safety and effectiveness of the surgical process are ensured.

[0070] The beneficial effects of the present invention are as follows: the present invention improves the consistency and reliability of multimodal data through dynamic standardization and registration processing; retains the multi-scale information of the image and enhances the clarity and contrast of the image through the Laplace pyramid fusion method; improves the local contrast of the image and makes the tumor area more obvious through multi-scale denoising and adaptive histogram equalization; optimizes the model performance and improves the accuracy of the segmentation result through interactive correction and active learning; constructs a three-dimensional model through the Marching Cubes algorithm, provides an intuitive and interactive three-dimensional environment, and helps doctors better understand the location of the tumor; ensures the safety and smoothness of the puncture path through the path planning algorithm and improves the accuracy of the operation; guides the puncture operation through the AR view to ensure the safety and effectiveness of the surgical process. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0072] Figure 1 This is a flow chart of the tumor ablation puncture surgery planning method in Example 1.

[0073] Figure 2 This is a module diagram of the tumor ablation puncture surgery planning system in Example 1. DETAILED DESCRIPTION

[0074] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0075] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a tumor ablation puncture surgery planning method, comprising the following steps:

[0076] S1. Acquire multimodal imaging data of the tumor area and perform preprocessing.

[0077] Furthermore, the patient's tumor area is scanned synchronously using a CT scanner and an MRI scanner to ensure the temporal consistency of the data and obtain multimodal imaging data of the tumor area. These two devices can provide detailed anatomical structure and soft tissue contrast respectively;

[0078] The multimodal imaging data is standardized, and the standardization parameters are dynamically adjusted according to the local characteristics of the image to retain more detailed information. At the same time, the intensity of all images falls within the same range. The expression is as follows:

[0079]

[0080] Where I is the original image intensity, I std is the normalized image intensity, I max and I min are the maximum and minimum intensity values ​​in the original image, γ is the adjustment coefficient, and β is the offset coefficient;

[0081] The adjustment coefficient γ is used to adjust the normalized image intensity range and is dynamically adjusted according to the local contrast to enhance the areas with higher contrast. The expression is as follows:

[0082]

[0083] Among them, γ0 is the benchmark adjustment coefficient, k c is the contrast gain coefficient, which is used to control the degree of contrast influence, C(x,y) is the local contrast, C max is the global maximum local contrast, N is the number of pixels in the neighborhood, N(x,y) refers to the neighborhood centered on the pixel point (x,y), A(i,j) is the pixel value of the image at position (i,j), and μ(x,y) is the average pixel value in N(x,y);

[0084] β is used to adjust the normalized image intensity offset and is dynamically adjusted according to the local brightness to maintain the details of the brighter areas. The expression is as follows:

[0085]

[0086] Where β0 is the reference offset coefficient, k B is the brightness gain coefficient, B(x,y) is the local brightness, B max is the global maximum local brightness;

[0087] The multimodal imaging data are registered and the CT and MRI images are registered to the same coordinate system using a rigid transformation matrix.

[0088] S2. Using an image fusion algorithm to fuse the pre-processed multimodal image data to generate a fused image.

[0089] Furthermore, the Laplacian pyramid fusion method in the fusion algorithm based on multi-resolution analysis is selected to decompose the CT and MRI images into sub-bands with different resolutions;

[0090] Furthermore, the original image l0 is reduced by half through downsampling operation to obtain l1, and the downsampling operation is repeated on the reduced image to form a series of images {l0,l1....l i};

[0091] The downsampling operation is specifically a filtering and decimation method. Before decimation, a Gaussian filter is applied to the image to remove high-frequency components, and then pixel decimation is performed to obtain a new image with a reduced size and a certain degree of smoothness.

[0092] For each subband l i , by calculating l i With the previous layer l i-1 The difference between the two subbands is obtained by i , i is the index variable, generating a series of subbands {L0,L1....L i-1}, each sub-band represents high-frequency information at different resolutions;

[0093] For each pixel position (x, y), the adaptive weights of the CT image and the MRI image are calculated and fused on each sub-band. The expression is as follows:

[0094]

[0095] Among them, F i (x, y) is the pixel value of the fused i-th layer image at position (x, y), is the pixel value of the CT image of the i-th layer at position (x, y), is the pixel value of the MRI image of the i-th layer at position (x, y); and They are and The weight of ; ∝ is the adjustment parameter used to control the sensitivity of the weight;

[0096] Furthermore, the fused sub-bands are reconstructed back to the fused image of the original resolution through the inverse Laplacian pyramid, starting from the sub-band with the lowest resolution and gradually reconstructing upwards. At each step, the current sub-band is merged with the reconstructed image of the previous layer to obtain the final fused image;

[0097] It should be noted that the final fused image combines the hard tissue details provided by CT and the soft tissue contrast provided by MRI to obtain a more comprehensive view of the tumor area.

[0098] S3. Based on the fused image, the tumor area is automatically marked using image segmentation technology, and the doctor manually corrects the segmentation result.

[0099] Furthermore, the standard deviation of the Gaussian filter is adjusted according to the characteristics and noise level of the image, and multiple Gaussian filters with different standard deviations are used to denoise the fused image;

[0100] The image is divided into several small areas, and each small area is subjected to histogram equalization processing to enhance the contrast of the fused image, making the boundary between the tumor area and the surrounding tissue more obvious, and the processed small area images are recombined into a complete image;

[0101] Preferably, a U-Net model, a convolutional neural network model specially designed for biomedical image segmentation, is selected, and a training data set is extracted from the fused image to train the U-Net model's ability to identify tumor boundaries;

[0102] Add a self-attention mechanism to the encoder and decoder parts of the U-Net model. By calculating the importance weights of feature maps, key features are enhanced and irrelevant features are suppressed, so that the model can better focus on the tumor area and improve segmentation accuracy.

[0103] The preprocessed fused image is input into the trained U-Net model, and the preliminary segmentation result is output, which is a binary image representing the tumor area and non-tumor area;

[0104] Furthermore, doctors view and display the preliminary segmentation results and original images through visualization software and manually modify the segmentation results using interactive tools;

[0105] Using active learning methods, the most informative samples are selected from the doctors' correction results through diversity sampling, the selected samples are added to the training set, the U-Net model is retrained, and the performance of the model is gradually improved;

[0106] The revised segmentation results are integrated into the patient's medical records as a basis for subsequent treatment.

[0107] S4. According to the fused image with tumor annotation, the optimal puncture path is selected through design and simulation.

[0108] Convert the two-dimensional images of tumors segmented by the U-Net model and corrected by doctors into VTK format, a data format suitable for three-dimensional modeling;

[0109] The segmented two-dimensional images are stacked into three-dimensional voxel data, each voxel represents a pixel point in the image, and its pixel value indicates the tissue type at its location;

[0110] A three-dimensional density field is constructed from the voxel data, where the position and value of each voxel is used to represent the density distribution of the tissue;

[0111] The three-dimensional density field is divided into several cubes with 8 vertices, and the density values ​​of the 8 vertices are calculated. According to the density values ​​of these vertices, the intersection points of the isosurface and the cube edge are determined;

[0112] According to the intersection points of the isosurface and the cube edge, triangular patches are generated, and these triangular patches constitute the surface of the three-dimensional model;

[0113] Smooth the generated triangular patches to eliminate jagged edges and improve the smoothness and aesthetics of the model;

[0114] The Marching Cubes algorithm constructs an intuitive and interactive three-dimensional environment by finding isosurfaces in the three-dimensional density field to more accurately simulate the puncture path. The isosurface is a surface composed of all points with density values ​​equal to a certain threshold.

[0115] In the three-dimensional model, the doctor determines the starting point (skin puncture point) and end point (tumor center) of the puncture based on the tumor location and the actual location of the tumor. In order to reduce harm to the patient, the starting point should be selected at the place closest to the tumor and will not pass through important structures, while the end point is located in the core of the tumor;

[0116] Furthermore, the path planning algorithm is used to find the best path from the puncture start point to the end point in the three-dimensional model under the conditions of avoiding important tissues, avoiding hard tissue obstacles and maintaining the smoothness of the path;

[0117] By simulating the puncture process in a three-dimensional model, we can check whether there are any unforeseen important tissues or obstacles along the path. If so, we can replan the puncture path.

[0118] S5. Based on the fused image and augmented reality device, the three-dimensional model is accurately aligned with the tumor entity position using spatial positioning technology to generate an AR view.

[0119] Furthermore, the augmented reality device is connected to a computer to facilitate access to the fused image data;

[0120] Calibrate the augmented reality device and adjust the relative position between the camera of the augmented reality device and the patient to ensure that the real scene captured by the camera is consistent with the coordinate system in the fused image;

[0121] The infrared tracking system installed in the operating room enables precise alignment of the 3D model and the tumor entity with the patient’s actual tumor location;

[0122] The infrared tracking system consists of multiple cameras that capture information from reflective markers attached to medical devices and patients, and collects marker motion data through sensors on augmented reality devices;

[0123] By calculating the transformation matrix between the augmented reality device and the fused image, and converting it into the coordinate system in the fused image, the spatial correspondence between the two is ensured;

[0124] The three-dimensional model in the fused image is aligned with the real scene, and the tumor model, puncture path and real scene are superimposed and displayed on the display to generate an AR view.

[0125] S6. Guided by the AR view, the puncture needle performs the ablation operation along the optimal puncture path.

[0126] Furthermore, the doctor can reconfirm the optimal puncture path through the AR view, fine-tune the path through gesture control, and lock the puncture path;

[0127] Disinfect the patient's puncture site and mark the puncture starting point. Perform local anesthesia on the patient's puncture site based on the patient's weight, age and health condition.

[0128] The size of the puncture needle is selected according to the size of the tumor and the depth of its location. Under the guidance of the AR view, the doctor performs the puncture along the optimal puncture path, keeping the needle tip consistent with the path displayed on the AR view.

[0129] Furthermore, when the ablation device connected to the end of the puncture needle reaches the center of the tumor, the ablation device is activated, and the doctor observes the position of the puncture needle and the changes in the surrounding tissues in real time through the AR view to ensure that the ablation range covers the entire tumor area without damaging the surrounding healthy tissues;

[0130] After the ablation is completed, the puncture needle is removed, the puncture site is checked for abnormalities, and the wound at the puncture starting point is treated;

[0131] Furthermore, after the operation, the patient's heart rate, blood pressure, and blood oxygen saturation are monitored and recorded, and the patient's puncture wound recovery status is observed;

[0132] An imaging review will be conducted 24 hours after surgery, and the results will be combined with the patient's postoperative clinical manifestations to evaluate the tumor ablation effect;

[0133] Based on the evaluation results, the next treatment plan is formulated.

[0134] This embodiment also provides a tumor ablation puncture surgery planning system, including: an image acquisition and processing module, an image fusion module, a segmentation and correction module, a puncture path planning module, an AR view generation module and an ablation operation execution module;

[0135] The image acquisition and processing module is used to acquire multimodal image data of the tumor area and perform preprocessing;

[0136] The image fusion module is used to fuse the pre-processed multi-modal image data using an image fusion algorithm to generate a fused image;

[0137] The segmentation and correction module is used to automatically mark the tumor area based on the fused image using image segmentation technology, and the doctor manually corrects the segmentation result;

[0138] The puncture path planning module is used to select the best puncture path through design and simulation based on the fused image with tumor annotations;

[0139] The AR view generation module is used to generate an AR view by accurately aligning the model with the tumor entity position based on the fused image and the augmented reality device using the spatial positioning technology;

[0140] The ablation operation execution module is used to perform the ablation operation along the optimal puncture path with the puncture needle guided by the AR view.

[0141] This embodiment also provides a computer device, which is suitable for the case of a tumor ablation puncture surgery planning method, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the tumor ablation puncture surgery planning method proposed in the above embodiment.

[0142] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0143] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for planning tumor ablation puncture surgery proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0144] In summary, the present invention improves the consistency and reliability of multimodal data through: dynamic standardization and registration processing; retains the multi-scale information of the image and enhances the clarity and contrast of the image through the Laplace pyramid fusion method; improves the local contrast of the image and makes the tumor area more obvious through multi-scale denoising and adaptive histogram equalization; optimizes the model performance and improves the accuracy of the segmentation result through interactive correction and active learning; constructs a three-dimensional model through the Marching Cubes algorithm, provides an intuitive and interactive three-dimensional environment, and helps doctors better understand the location of the tumor; ensures the safety and smoothness of the puncture path through the path planning algorithm, and improves the accuracy of the operation; guides the puncture operation through the AR view, ensuring the safety and effectiveness of the surgical process.

[0145] Example 2, referring to Table 1, is the second example of the present invention. To further verify the technical solution of the present invention, experimental simulation data of a tumor ablation puncture surgery planning method is provided.

[0146] Ten patients with lung tumors were selected, and all patients agreed to participate in the experiment and signed an informed consent form. The experiment was divided into two groups, one group using the method of the present invention (experimental group), and the other group using the traditional method (control group).

[0147] Each patient was scanned using a CT scanner and an MRI scanner to obtain multimodal imaging data, which were then standardized to ensure that the intensity of all images fell within the same range, and the CT and MRI images were registered to the same coordinate system using a rigid body transformation matrix.

[0148] The Laplacian pyramid fusion method is selected to decompose the CT and MRI images into sub-bands of different resolutions. Through downsampling operations, a series of images are formed, and the sub-bands in the Laplacian pyramid are calculated. Fusion is performed on each sub-band, and finally a fused image with the original resolution is reconstructed.

[0149] Gaussian filtering and histogram equalization are used to enhance the contrast of the fused image. The U-Net model is selected and the training data set is extracted from the fused image. The U-Net model is trained to recognize tumor boundaries. The trained U-Net model is loaded, the fused image is processed and segmented, and the preliminary segmentation results are output. The doctor views the preliminary segmentation results through visualization software and uses interactive tools to manually correct the segmentation results.

[0150] Based on the corrected segmentation results, the Marching Cubes algorithm is used to construct a three-dimensional model of the tumor and surrounding important tissues. In the three-dimensional model, the doctor determines the starting and ending points of the puncture according to the location of the tumor, and uses the path planning algorithm to find the best path from the starting point to the end point of the puncture in the three-dimensional model, and simulates the puncture process to check whether the path is appropriate.

[0151] The augmented reality device is connected to a computer and calibrated. An infrared tracking system is used to accurately align the three-dimensional model with the patient's actual tumor location to generate an AR view.

[0152] The doctor confirms the optimal puncture path through the AR view and performs the ablation operation under guidance. The doctor disinfects the patient's puncture site, marks the puncture starting point, and performs local anesthesia. Under the guidance of the AR view, the doctor performs the puncture operation according to the optimal puncture path. After the ablation is completed, the doctor checks whether there is any abnormality at the puncture site and treats the wound. The patient's vital signs are monitored after the operation, and an imaging review is performed 24 hours after the operation.

[0153] The details are shown in Table 1 below:

[0154] Table 1 Comparison of fusion image and ablation operation effects

[0155]

[0156] In the experiment, the experimental group used the method of the present invention, and the control group used the traditional method. From the table data, it can be seen that all indicators of the experimental group are better than those of the control group.

[0157] The average score of the experimental group was 9.5 points, while the average score of the control group was 7.5 points, which showed that the fused images of the experimental group were of higher quality, with clearer details and better contrast.

[0158] The average score of the experimental group was 9.8 points, while the average score of the control group was 8.2 points. This shows that the segmentation results of the experimental group are more accurate, reducing the workload of doctors' manual correction.

[0159] The average time for the experimental group was 15 minutes, while the average time for the control group was 25 minutes.

[0160] The average time for the experimental group was 22 minutes, while the average time for the control group was 30 minutes. The experimental group also showed greater efficiency in the ablation procedure, reducing the procedure time.

[0161] Through the above data comparison, the present invention has significant advantages in improving the accuracy, efficiency and safety of surgical planning and implementation, and provides strong support for clinical practice.

Claims

1. A method for planning tumor ablation puncture surgery, characterized in that: include, Acquire multimodal imaging data of the tumor area and perform preprocessing; The pre-processed multi-modal image data is fused using an image fusion algorithm to generate a fused image; Based on the fused image, the image segmentation technology is used to automatically mark the tumor area, and the doctor manually corrects the segmentation result; According to the fused image with tumor annotation, the best puncture path is selected through design and simulation; Based on the fused image and augmented reality device, the model is accurately aligned with the tumor entity position using spatial positioning technology to generate an AR view; Guided by the AR view, the puncture needle performs the ablation operation along the optimal puncture path.

2. The tumor ablation puncture surgery planning method according to claim 1, characterized in that: The multimodal image data of the tumor area is obtained and preprocessed. The specific steps are as follows: Use a CT scanner and an MRI scanner to simultaneously scan the patient's tumor area to obtain multimodal imaging data of the tumor area; Standardize multimodal imaging data and dynamically adjust standardization parameters based on local characteristics of the image; The multimodal imaging data are registered and the CT and MRI images are registered to the same coordinate system using a rigid transformation matrix.

3. The tumor ablation puncture surgery planning method according to claim 1, characterized in that: The image fusion algorithm is used to fuse the pre-processed multi-modal image data to generate a fused image. The specific steps are as follows: The Laplacian pyramid fusion method in the fusion algorithm based on multi-resolution analysis is selected to decompose the CT image and MRI image into sub-bands with different resolutions; Through the downsampling operation, the original image l0 is reduced by half to obtain l1, and the downsampling operation is repeated on the reduced image to form a series of images {l0,l1....l i }; For {l0,l1....l i }, the subband L in the Laplace pyramid is calculated by the difference between two adjacent layers of images i , i is the index variable, generating a series of subbands {L0,L1....L i-1 }, each sub-band represents high-frequency information at different resolutions; Fusion is performed on each subband. For each pixel position (x, y), the following fusion rule is selected: Among them, F i (x, y) is the pixel value of the fused i-th layer image at position (x, y), is the pixel value of the CT image of the i-th layer at position (x, y), is the pixel value of the MRI image of the i-th layer at position (x, y); and They are and The weight of The fused sub-bands are reconstructed back to the fused image of the original resolution through the inverse Laplacian pyramid, starting from the sub-band with the lowest resolution and gradually reconstructed upwards. At each step, the current sub-band is merged with the reconstructed image of the previous layer to obtain the final fused image.

4. The tumor ablation puncture surgery planning method according to claim 2, characterized in that: Based on the fused image, the image segmentation technology is used to automatically mark the tumor area, and the doctor manually corrects the segmentation result. The specific steps are as follows: The standard deviation of the Gaussian filter is adjusted according to the characteristics and noise level of the image, and multiple Gaussian filters with different standard deviations are used to denoise the fused image; The image is divided into several small areas, each small area is subjected to histogram equalization processing, and the processed small area images are reassembled into a complete image; Select a U-Net model and extract a training data set from the fused image to train the U-Net model's ability to identify tumor boundaries; Add self-attention mechanism to the encoder and decoder parts of the U-Net model to calculate the importance weights of feature maps; The preprocessed fused image is input into the trained U-Net model, and the preliminary segmentation result is output, which is a binary image representing the tumor area and non-tumor area; Doctors viewed the preliminary segmentation results and original images through visualization software and manually corrected the segmentation results using interactive tools; Using active learning methods, the most informative samples are selected from the doctors' correction results through diversity sampling, the selected samples are added to the training set, and the U-Net model is retrained; The revised segmentation results are integrated into the patient's medical records as a basis for subsequent treatment.

5. The tumor ablation puncture surgery planning method according to claim 3, characterized in that: The optimal puncture path is selected through design and simulation based on the fused image with tumor annotation. The specific steps are as follows: Convert the two-dimensional images of tumors segmented by the U-Net model and corrected by doctors into VTK format, a data format suitable for three-dimensional modeling; The segmented two-dimensional images are stacked into three-dimensional voxel data, each voxel represents a pixel point in the image, and its pixel value indicates the tissue type at its location; A three-dimensional density field is constructed from the voxel data, where the position and value of each voxel is used to represent the density distribution of the tissue; The three-dimensional density field is divided into several cubes with 8 vertices, and the density values ​​of the 8 vertices are calculated. According to the density values ​​of these vertices, the intersection points of the isosurface and the cube edge are determined; According to the intersection points of the isosurface and the cube edge, triangular patches are generated, which constitute the surface of the three-dimensional model, and the generated triangular patches are smoothed; The Marching Cubes algorithm constructs an intuitive and interactive three-dimensional environment by finding isosurfaces in a three-dimensional density field. The isosurface is a surface composed of all points whose density value is equal to a certain threshold. In the three-dimensional model, the doctor determines the starting and ending points of the puncture based on the tumor location and the actual location of the tumor; Using the path planning algorithm, the optimal path from the puncture start point to the end point is found in the three-dimensional model under the conditions of avoiding important tissues, avoiding hard tissue obstacles and maintaining the smoothness of the path; By simulating the puncture process in a three-dimensional model, the path is checked for any unforeseen obstacles. If so, the puncture path is replanned.

6. The tumor ablation puncture surgery planning method according to claim 4, characterized in that: Based on the fusion image and augmented reality device, the three-dimensional model is accurately aligned with the tumor entity position using spatial positioning technology to generate an AR view. The specific steps are as follows: Connecting the augmented reality device to the computer and calibrating the augmented reality device; The infrared tracking system installed in the operating room enables precise alignment of the three-dimensional model and the tumor entity with the patient's actual tumor location; The infrared tracking system contains multiple cameras, which capture information from reflective markers attached to medical devices and patients, collect marker motion data through sensors on augmented reality devices, and convert it into a coordinate system in the fused image; The three-dimensional model in the fused image is aligned with the real scene, and the tumor model, puncture path and real scene are superimposed and displayed on the display to generate an AR view.

7. The tumor ablation puncture surgery planning method according to claim 5, characterized in that: The AR view is used as a guide, and the puncture needle performs the ablation operation along the optimal puncture path. The specific steps are as follows: The doctor uses the AR view to reconfirm the optimal puncture path, fine-tune the path through gesture control, and lock the puncture path; Disinfect the patient's puncture site and mark the puncture starting point. Perform local anesthesia on the patient's puncture site based on the patient's weight, age and health condition. The size of the puncture needle is selected according to the size and depth of the tumor. Under the guidance of the AR view, the doctor performs the puncture along the optimal puncture path. When the ablation device connected to the end of the puncture needle reaches the center of the tumor, the ablation device is activated, and the doctor observes the position of the puncture needle and the changes in the surrounding tissues in real time through the AR view; After the ablation is completed, the puncture needle is removed, the puncture site is checked for abnormalities, and the wound at the puncture starting point is treated; After surgery, the patient's heart rate, blood pressure, and blood oxygen saturation were monitored and recorded, and the recovery of the patient's puncture wound was observed; An imaging review will be performed 24 hours after surgery, and the review results will be combined with the patient's postoperative clinical manifestations to evaluate the tumor ablation effect; Based on the evaluation results, the next treatment plan is formulated.

8. A tumor ablation puncture surgery planning system, based on the tumor ablation puncture surgery planning method according to any one of claims 1 to 7, characterized in that: It includes an image acquisition and processing module, an image fusion module, a segmentation and correction module, a puncture path planning module, an AR view generation module and an ablation operation execution module; The image acquisition and processing module is used to acquire multimodal image data of the tumor area and perform preprocessing; The image fusion module is used to fuse the pre-processed multi-modal image data using an image fusion algorithm to generate a fused image; The segmentation and correction module is used to automatically mark the tumor area based on the fused image using image segmentation technology, and the doctor manually corrects the segmentation result; The puncture path planning module is used to select the best puncture path through design and simulation according to the fused image with tumor annotation; The AR view generation module is used to generate an AR view by accurately aligning the model with the tumor entity position based on the fused image and the augmented reality device using the spatial positioning technology; The ablation operation execution module is used to guide the AR view so that the puncture needle performs the ablation operation along the optimal puncture path.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the tumor ablation puncture surgery planning method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the tumor ablation puncture surgery planning method according to any one of claims 1 to 7 are implemented.

Images