Tumor ablation puncture surgery planning method and system

Abstract

The present invention discloses a method and system for planning tumor ablation puncture surgery, which relates to the field of medical image processing technology, including obtaining multimodal image data of the tumor area and preprocessing it, fusing the preprocessed multimodal image data using an image fusion algorithm to generate a fused image, automatically annotating the tumor area based on the fused image using image segmentation technology, and manually correcting the segmentation result by a doctor, selecting the optimal puncture path through design and simulation based on the fused image with tumor annotation, accurately aligning the model with the actual position of the tumor using spatial positioning technology based on the fused image and an augmented reality device, generating an AR view, and using the AR view as a guide, performing the ablation operation with the puncture needle along the optimal puncture path. The present invention achieves high precision and visualization of the entire process from image acquisition to surgical implementation through high-quality imaging and three-dimensional guidance, significantly improving the safety and success rate of the surgery.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method and system for planning tumor ablation and puncture surgery. Background Art

[0002] In recent years, with the rapid development of medical imaging and computer science, multimodal image fusion technology has played an increasingly important role in tumor diagnosis and treatment. Traditional tumor ablation and puncture procedures rely primarily on single-modality imaging data, such as CT or MRI. However, due to their respective limitations, such as CT's lack of soft tissue contrast and MRI's lack of hard tissue detail, single-modality imaging cannot provide comprehensive information support.

[0003] Current puncture path planning mainly relies on the doctor's experience and lacks accurate simulation tools, making it impossible to effectively evaluate and select multiple paths before surgery. During traditional surgery, doctors mainly rely on two-dimensional images for puncture operations and lack intuitive three-dimensional view guidance, resulting in limited puncture accuracy and increased surgical risks. The present invention improves the accuracy and safety of surgery through technical means such as multimodal image fusion, automatic segmentation and correction, optimal puncture path simulation, and AR technology to guide the puncture needle to perform ablation operations. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a tumor ablation puncture surgery planning method and system to solve the problem of lack of precise guidance in puncture path planning.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a method for planning tumor ablation puncture surgery, which includes:

[0008] Acquire multimodal imaging data of the tumor area and perform preprocessing;

[0009] The pre-processed multimodal image data is fused using an image fusion algorithm to generate a fused image;

[0010] Based on the fused image, the tumor area is automatically marked using image segmentation technology, and the doctor manually corrects the segmentation results;

[0011] Based on the fused image with tumor annotations, the optimal puncture path is selected through design and simulation;

[0012] Based on fused images and augmented reality devices, spatial positioning technology is used to accurately align the model with the actual location of the tumor to generate an AR view;

[0013] Guided by the AR view, the puncture needle performs the ablation operation along the optimal puncture path.

[0014] As a preferred solution of the tumor ablation puncture surgery planning method of the present invention, wherein: the multimodal imaging data of the tumor area is obtained and preprocessed, the specific steps are as follows:

[0015] Scanning the patient's tumor area using a CT scanner and an MRI scanner to obtain multimodal imaging data of the tumor area;

[0016] Use a CT scanner and an MRI scanner to simultaneously scan the patient's tumor area to obtain multimodal imaging data of the tumor area;

[0017] Standardize multimodal imaging data and dynamically adjust normalization parameters based on local image characteristics;

[0018] The multimodal imaging data are registered and the CT and MRI images are registered to the same coordinate system using a rigid body transformation matrix.

[0019] As a preferred solution of the tumor ablation puncture surgery planning method of the present invention, wherein: the pre-processed multimodal image data is fused using an image fusion algorithm to generate a fused image, and the specific steps are as follows:

[0020] The Laplacian pyramid fusion method is selected from the fusion algorithm based on multi-resolution analysis to decompose the CT image and MRI image into sub-bands with different resolutions;

[0021] Through the downsampling operation, the original image l0 is reduced by half to obtain l1, and the downsampling operation is repeated on the reduced image to form a series of images {l0,l1....l i};

[0022] For {l0,l1....l i}, calculate the subband L in the Laplace pyramid by the difference between two adjacent layers of images i , i is the index variable, generating a series of sub-bands {L0,L1....L i-1}, each sub-band represents high-frequency information at different resolutions;

[0023] Fusion is performed on each subband. For each pixel position (x, y), the following fusion rule is selected:

[0024]

[0025] Among them, F i(x,y) is the pixel value of the fused i-th layer image at position (x,y), is the pixel value of the CT image of the i-th layer at position (x, y), is the pixel value of the MRI image of the i-th layer at position (x, y); and They are and The weight of

[0026] The fused sub-bands are reconstructed back to the fused image of the original resolution through the inverse Laplacian pyramid, starting from the sub-band with the lowest resolution and gradually reconstructing upwards. At each step, the current sub-band is merged with the reconstructed image of the previous layer to obtain the final fused image.

[0027] As a preferred solution of the tumor ablation puncture surgery planning method of the present invention, wherein: based on the fusion image, the tumor area is automatically marked using image segmentation technology, and the doctor manually corrects the segmentation result. The specific steps are as follows:

[0028] The standard deviation of the Gaussian filter is adjusted according to the image characteristics and noise level, and multiple Gaussian filters with different standard deviations are used to denoise the fused image;

[0029] Divide the image into several small areas, perform histogram equalization on each small area, and reassemble the processed small area images into a complete image;

[0030] Select a U-Net model and extract a training dataset from the fused image to train the U-Net model's ability to identify tumor boundaries;

[0031] Add a self-attention mechanism to the encoder and decoder parts of the U-Net model to calculate the importance weights of the feature maps;

[0032] The preprocessed fusion image is input into the trained U-Net model, and the output is a preliminary segmentation result, which is a binary image representing the tumor area and non-tumor area;

[0033] Doctors review the preliminary segmentation results and original images through visualization software and manually modify the segmentation results using interactive tools;

[0034] Using active learning methods, we select the most informative samples from the doctors' correction results through diversity sampling, add the selected samples to the training set, and retrain the U-Net model to gradually improve the model's performance.

[0035] The revised segmentation results are integrated into the patient's medical records as a basis for subsequent treatment.

[0036] As a preferred solution of the tumor ablation puncture surgery planning method of the present invention, wherein: the optimal puncture path is selected through design and simulation based on the fused image with tumor annotation, the specific steps are as follows:

[0037] Convert the two-dimensional images of tumors segmented by the U-Net model and corrected by doctors into VTK format, a data format suitable for three-dimensional modeling;

[0038] The segmented two-dimensional images are stacked into three-dimensional voxel data, where each voxel represents a pixel in the image, and its pixel value indicates the tissue type at its location;

[0039] A three-dimensional density field is constructed based on the voxel data, where the position and value of each voxel are used to represent the density distribution of the tissue;

[0040] The three-dimensional density field is divided into several cubes containing 8 vertices, and the density values ​​of the 8 vertices are calculated. According to the density values ​​of these vertices, the intersection points of the isosurface and the cube edges are determined;

[0041] According to the intersection of the isosurface and the cube edge, triangular patches are generated. These triangular patches form the surface of the three-dimensional model, and the generated triangular patches are smoothed.

[0042] The Marching Cubes algorithm constructs an intuitive and interactive three-dimensional environment by finding isosurfaces in a three-dimensional density field. The isosurface is a surface consisting of all points whose density value is equal to a certain threshold.

[0043] In the three-dimensional model, the doctor determines the starting and ending points of the puncture based on the tumor location and the actual location of the tumor;

[0044] Using the path planning algorithm, the optimal path from the puncture start point to the end point is found in the 3D model while taking into account the avoidance of important tissues, avoiding hard tissue obstacles and maintaining the smoothness of the path.

[0045] By simulating the puncture process in a 3D model, the path is checked for unforeseen obstacles. If so, the puncture path is replanned.

[0046] As a preferred solution of the tumor ablation puncture surgery planning method of the present invention, wherein: based on the fusion image and augmented reality device, the three-dimensional model is accurately aligned with the tumor entity position using spatial positioning technology to generate an AR view. The specific steps are as follows:

[0047] Connecting the augmented reality device to the computer and calibrating the augmented reality device;

[0048] An infrared tracking system installed in the operating room enables precise alignment of the 3D model and the tumor entity with the patient's actual tumor location;

[0049] The infrared tracking system consists of multiple cameras that capture information from reflective markers attached to medical devices and patients. The sensors on the augmented reality device collect the marker motion data and convert it into a coordinate system in the fused image.

[0050] The three-dimensional model in the fused image is aligned with the real scene, and the tumor model, puncture path and real scene are superimposed and displayed on the display to generate an AR view.

[0051] As a preferred embodiment of the tumor ablation puncture surgery planning method of the present invention, wherein: the AR image is used as a guide, and the puncture needle performs the ablation operation along the optimal puncture path. The specific steps are as follows:

[0052] The doctor uses the AR view to reconfirm the optimal puncture path, fine-tune the path through gesture control, and lock the puncture path;

[0053] Disinfect the patient's puncture site and mark the puncture starting point. Perform local anesthesia on the patient's puncture site based on the patient's weight, age, and health status.

[0054] The puncture needle size is selected according to the tumor size and location depth. Under the guidance of the AR view, the doctor performs the puncture operation along the optimal puncture path;

[0055] When the ablation device connected to the end of the puncture needle reaches the center of the tumor, the ablation device is activated, and the doctor observes the position of the puncture needle and changes in surrounding tissues in real time through the AR view;

[0056] After the ablation is completed, the puncture needle is removed, the puncture site is checked for abnormalities, and the wound at the puncture starting point is treated;

[0057] After surgery, the patient's heart rate, blood pressure, and blood oxygen saturation were monitored and recorded, and the patient's puncture wound recovery was observed;

[0058] An imaging review is performed 24 hours after surgery, and the results are combined with the patient's postoperative clinical manifestations to evaluate the tumor ablation effect;

[0059] Based on the evaluation results, the next treatment plan is formulated.

[0060] In a second aspect, the present invention provides a tumor ablation puncture surgery planning system, comprising an image acquisition and processing module, an image fusion module, a segmentation and correction module, a puncture path planning module, an AR view generation module, and an ablation operation execution module;

[0061] The image acquisition and processing module is used to acquire multimodal image data of the tumor area and perform preprocessing;

[0062] The image fusion module is used to fuse the pre-processed multimodal image data using an image fusion algorithm to generate a fused image;

[0063] The segmentation and correction module is used to automatically mark the tumor area based on the fused image using image segmentation technology, and the doctor manually corrects the segmentation result;

[0064] The puncture path planning module is used to select the optimal puncture path through design and simulation based on the fused image with tumor annotation;

[0065] The AR view generation module is used to accurately align the model with the tumor entity position based on the fused image and augmented reality device using spatial positioning technology to generate an AR view;

[0066] The ablation operation execution module is used to perform the ablation operation along the optimal puncture path with the puncture needle guided by the AR view.

[0067] In a third aspect, an embodiment of the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the tumor ablation puncture surgery planning method as described in the first aspect of the present invention is implemented.

[0068] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the tumor ablation puncture surgery planning method as described in the first aspect of the present invention is implemented.

[0069] Dynamic standardization and registration processing improve the consistency and reliability of multimodal data; the Laplace pyramid fusion method retains the multi-scale information of the image, enhancing the image clarity and contrast; multi-scale denoising and adaptive histogram equalization improve the local contrast of the image, making the tumor area more distinct; interactive correction and active learning optimize model performance and improve the accuracy of segmentation results; the Marching Cubes algorithm constructs a three-dimensional model, providing an intuitive and interactive three-dimensional environment to help doctors better understand the location of the tumor; the path planning algorithm ensures the safety and smoothness of the puncture path, improving the precision of the surgery; and the AR view guides the puncture operation, ensuring the safety and effectiveness of the surgical process.

[0070] The beneficial effects of the present invention are as follows: the present invention improves the consistency and reliability of multimodal data through dynamic standardization and registration processing; retains the multi-scale information of the image and enhances the clarity and contrast of the image through the Laplace pyramid fusion method; improves the local contrast of the image and makes the tumor area more obvious through multi-scale denoising and adaptive histogram equalization; optimizes the model performance and improves the accuracy of the segmentation results through interactive correction and active learning; constructs a three-dimensional model through the Marching Cubes algorithm, provides an intuitive and interactive three-dimensional environment, and helps doctors better understand the location of the tumor; ensures the safety and smoothness of the puncture path through the path planning algorithm and improves the accuracy of the operation; and guides the puncture operation through the AR view to ensure the safety and effectiveness of the surgical process. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0072] Figure 1 This is a flow chart of the tumor ablation puncture surgery planning method in Example 1.

[0073] Figure 2 This is a module diagram of the tumor ablation puncture surgery planning system in Example 1. DETAILED DESCRIPTION

[0074] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0075] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a method for planning tumor ablation puncture surgery, comprising the following steps:

[0076] S1. Acquire multimodal imaging data of the tumor area and perform preprocessing.

[0077] Furthermore, the patient's tumor area is scanned synchronously with a CT scanner and an MRI scanner to ensure temporal consistency of the data and obtain multimodal imaging data of the tumor area. These two devices can provide detailed anatomical structure and soft tissue contrast respectively;

[0078] Normalize the multimodal image data and dynamically adjust the normalization parameters according to the local characteristics of the image to retain more detailed information. At the same time, the intensity of all images falls within the same range. The expression is as follows:

[0079]

[0080] Where, I is the original image intensity, I std is the normalized image intensity, I max and I min are the maximum and minimum intensity values ​​in the original image, γ is the adjustment coefficient, and β is the offset coefficient;

[0081] The adjustment coefficient γ is used to adjust the normalized image intensity range and is dynamically adjusted according to the local contrast to enhance the areas with higher contrast. The expression is as follows:

[0082]

[0083] Among them, γ0 is the benchmark adjustment coefficient, k c is the contrast gain coefficient, which is used to control the degree of contrast influence, C(x,y) is the local contrast, C max is the global maximum local contrast, N is the number of pixels in the neighborhood, N(x,y) refers to the neighborhood centered on the pixel point (x,y), A(i,j) is the pixel value at position (i,j) in the image, and μ(x,y) is the average pixel value within N(x,y);

[0084] β is used to adjust the normalized image intensity offset and is dynamically adjusted according to the local brightness to maintain the details of the brighter areas. The expression is as follows:

[0085]

[0086] Where β0 is the reference offset coefficient, k B is the brightness gain coefficient, B(x,y) is the local brightness, B max is the global maximum local brightness;

[0087] The multimodal imaging data are registered and the CT and MRI images are registered to the same coordinate system using a rigid body transformation matrix.

[0088] S2. Using an image fusion algorithm to fuse the pre-processed multimodal image data to generate a fused image.

[0089] Furthermore, the Laplacian pyramid fusion method in the fusion algorithm based on multi-resolution analysis is selected to decompose the CT and MRI images into sub-bands with different resolutions;

[0090] Furthermore, the original image l0 is reduced to half by downsampling operation to obtain l1, and the downsampling operation is repeated on the reduced image to form a series of images {l0,l1....l i};

[0091] The downsampling operation is specifically done through filtering and decimation. Before decimation, a Gaussian filter is applied to the image to remove high-frequency components, and then pixel decimation is performed to obtain a new image with reduced size while maintaining a certain degree of smoothness.

[0092] For each subband l i , by calculating l i With the previous layer l i-1 The difference between the two subbands is obtained, and the subband L in the Laplace pyramid is obtained. i , i is the index variable, generating a series of sub-bands {L0,L1....L i-1}, each sub-band represents high-frequency information at different resolutions;

[0093] For each pixel position (x, y), the adaptive weights of the CT image and the MRI image are calculated and fused on each sub-band. The expression is as follows:

[0094]

[0095] Among them, F i (x,y) is the pixel value of the fused i-th layer image at position (x,y), is the pixel value of the CT image of the i-th layer at position (x, y), is the pixel value of the MRI image of the i-th layer at position (x, y); and They are and The weight of ; ∝ is the adjustment parameter used to control the sensitivity of the weight;

[0096] Furthermore, the fused subbands are reconstructed back to the fused image of the original resolution through the inverse Laplacian pyramid, starting from the lowest resolution subband and gradually reconstructing upwards. At each step, the current subband is merged with the reconstructed image of the previous layer to obtain the final fused image;

[0097] It should be noted that the final fused image combines the hard tissue details provided by CT and the soft tissue contrast provided by MRI, thereby obtaining a more comprehensive view of the tumor area.

[0098] S3. Based on the fused image, the tumor area is automatically marked using image segmentation technology, and the doctor manually corrects the segmentation results.

[0099] Furthermore, the standard deviation of the Gaussian filter is adjusted according to the image characteristics and noise level, and multiple Gaussian filters with different standard deviations are used to denoise the fused image;

[0100] The image is divided into several small areas, and histogram equalization is performed on each small area to enhance the contrast of the fused image, making the boundary between the tumor area and the surrounding tissue more distinct. The processed small area images are then recombined into a complete image.

[0101] Preferably, a U-Net model, a convolutional neural network model specially designed for biomedical image segmentation, is selected, and a training dataset is extracted from the fused image to train the U-Net model's ability to identify tumor boundaries;

[0102] A self-attention mechanism is added to the encoder and decoder parts of the U-Net model. By calculating the importance weights of feature maps, key features are enhanced and irrelevant features are suppressed, enabling the model to better focus on the tumor area and improve segmentation accuracy.

[0103] The preprocessed fusion image is input into the trained U-Net model, and the output is a preliminary segmentation result, which is a binary image representing the tumor area and non-tumor area;

[0104] Furthermore, doctors can view and display the preliminary segmentation results and original images through visualization software and manually modify the segmentation results using interactive tools;

[0105] Using active learning methods, we select the most informative samples from the doctors' correction results through diversity sampling, add the selected samples to the training set, and retrain the U-Net model to gradually improve the model's performance.

[0106] The revised segmentation results are integrated into the patient's medical records as a basis for subsequent treatment.

[0107] S4. Based on the fused image with tumor annotation, the optimal puncture path is selected through design and simulation.

[0108] Convert the two-dimensional images of tumors segmented by the U-Net model and corrected by doctors into VTK format, a data format suitable for three-dimensional modeling;

[0109] The segmented two-dimensional images are stacked into three-dimensional voxel data, where each voxel represents a pixel in the image, and its pixel value indicates the tissue type at its location;

[0110] A three-dimensional density field is constructed based on the voxel data, where the position and value of each voxel are used to represent the density distribution of the tissue;

[0111] The three-dimensional density field is divided into several cubes containing 8 vertices, and the density values ​​of the 8 vertices are calculated. According to the density values ​​of these vertices, the intersection points of the isosurface and the cube edges are determined;

[0112] According to the intersection of the isosurface and the cube edge, triangular patches are generated, and these triangular patches constitute the surface of the three-dimensional model;

[0113] Smooth the generated triangular patches to eliminate jagged edges and improve the smoothness and aesthetics of the model;

[0114] The Marching Cubes algorithm constructs an intuitive and interactive three-dimensional environment by finding isosurfaces in the three-dimensional density field to more accurately simulate the puncture path. The isosurface is a surface consisting of all points with density values ​​equal to a certain threshold.

[0115] In the 3D model, the doctor determines the starting point (skin puncture point) and end point (tumor center) of the puncture based on the tumor location and actual tumor location. To minimize harm to the patient, the starting point should be selected closest to the tumor and not pass through important structures, while the end point should be located in the core of the tumor.

[0116] Furthermore, a path planning algorithm is used to find the optimal path from the puncture start point to the end point in the three-dimensional model while taking into account avoiding important tissues, avoiding hard tissue obstacles and maintaining the smoothness of the path.

[0117] By simulating the puncture process in a three-dimensional model, the path is checked to see if there are any unforeseen important tissues or obstacles. If so, the puncture path is replanned.

[0118] S5. Based on the fused image and augmented reality device, spatial positioning technology is used to accurately align the three-dimensional model with the actual location of the tumor to generate an AR view.

[0119] Furthermore, the augmented reality device is connected to a computer to facilitate access to the fused image data;

[0120] Calibrate the augmented reality device and adjust the relative position of the camera of the augmented reality device and the patient to ensure that the real scene captured by the camera is consistent with the coordinate system of the fused image;

[0121] An infrared tracking system installed in the operating room enables precise alignment of the 3D model and the tumor entity with the patient's actual tumor location;

[0122] The infrared tracking system consists of multiple cameras that capture information from reflective markers attached to medical devices and patients, and uses sensors on augmented reality devices to collect marker motion data.

[0123] By calculating the transformation matrix between the augmented reality device and the fused image, and converting it into the coordinate system of the fused image, the spatial correspondence between the two is ensured;

[0124] The three-dimensional model in the fused image is aligned with the real scene, and the tumor model, puncture path and real scene are superimposed and displayed on the display to generate an AR view.

[0125] S6. Guided by the AR view, the puncture needle performs the ablation operation along the optimal puncture path.

[0126] Furthermore, the doctor can use the AR view to reconfirm the optimal puncture path, fine-tune the path through gesture control, and lock the puncture path;

[0127] Disinfect the patient's puncture site and mark the puncture starting point. Perform local anesthesia on the patient's puncture site based on the patient's weight, age, and health status.

[0128] The puncture needle size is selected according to the tumor size and depth. Under the guidance of the AR view, the doctor performs the puncture along the optimal puncture path, keeping the needle tip consistent with the path displayed on the AR view.

[0129] Furthermore, when the ablation device connected to the end of the puncture needle reaches the center of the tumor, the ablation device is activated, and the doctor observes the position of the puncture needle and the changes in the surrounding tissue in real time through the AR view, ensuring that the ablation range covers the entire tumor area without damaging the surrounding healthy tissue;

[0130] After the ablation is completed, the puncture needle is removed, the puncture site is checked for abnormalities, and the wound at the puncture starting point is treated;

[0131] Furthermore, after the operation, the patient's heart rate, blood pressure, and blood oxygen saturation are monitored and recorded, and the patient's puncture wound recovery status is observed;

[0132] An imaging review is performed 24 hours after surgery, and the results are combined with the patient's postoperative clinical manifestations to evaluate the tumor ablation effect;

[0133] Based on the evaluation results, the next treatment plan is formulated.

[0134] This embodiment also provides a tumor ablation puncture surgery planning system, including: an image acquisition and processing module, an image fusion module, a segmentation and correction module, a puncture path planning module, an AR view generation module, and an ablation operation execution module;

[0135] The image acquisition and processing module is used to acquire multimodal image data of the tumor area and perform preprocessing;

[0136] The image fusion module is used to fuse the pre-processed multimodal image data using an image fusion algorithm to generate a fused image;

[0137] The segmentation and correction module is used to automatically mark the tumor area based on the fused image using image segmentation technology, and the doctor manually corrects the segmentation results;

[0138] The puncture path planning module is used to select the optimal puncture path through design and simulation based on the fused image with tumor annotations;

[0139] The AR view generation module is used to generate an AR view by accurately aligning the model with the tumor entity position based on the fused image and augmented reality device using spatial positioning technology;

[0140] The ablation operation module is used to perform the ablation operation along the optimal puncture path with the puncture needle guided by the AR view.

[0141] This embodiment also provides a computer device suitable for the case of a tumor ablation puncture surgery planning method, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the tumor ablation puncture surgery planning method proposed in the above embodiment.

[0142] The computer device may be a terminal, comprising a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse.

[0143] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for planning tumor ablation puncture surgery proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0144] In summary, the present invention improves the consistency and reliability of multimodal data through: dynamic normalization and registration processing; retains the multi-scale information of the image through the Laplace pyramid fusion method, enhancing the image clarity and contrast; improves the local contrast of the image and makes the tumor area more obvious through multi-scale denoising and adaptive histogram equalization; optimizes model performance and improves the accuracy of segmentation results through interactive correction and active learning; constructs a three-dimensional model through the Marching Cubes algorithm, providing an intuitive and interactive three-dimensional environment to help doctors better understand the location of the tumor; ensures the safety and smoothness of the puncture path through the path planning algorithm, improving the accuracy of the surgery; and guides the puncture operation through the AR view, ensuring the safety and effectiveness of the surgical process.

[0145] Example 2, referring to Table 1, is the second example of the present invention. To further verify the technical solution of the present invention, experimental simulation data of a tumor ablation puncture surgery planning method is provided.

[0146] Ten patients with lung tumors were selected. All patients agreed to participate in the experiment and signed informed consent forms. The experiment was divided into two groups, one group using the method of the present invention (experimental group) and the other group using the traditional method (control group).

[0147] Each patient was scanned using a CT scanner and an MRI scanner to acquire multimodal imaging data. The multimodal imaging data was standardized to ensure that the intensity of all images fell within the same range, and the CT and MRI images were aligned to the same coordinate system using a rigid body transformation matrix.

[0148] The Laplacian pyramid fusion method is selected to decompose the CT and MRI images into sub-bands of different resolutions. Through downsampling operations, a series of images are formed. The sub-bands in the Laplacian pyramid are calculated, and fusion is performed on each sub-band to finally reconstruct the fused image back to the original resolution.

[0149] Gaussian filtering and histogram equalization are used to enhance the contrast of the fused image. A U-Net model is selected, and a training dataset is extracted from the fused image. The U-Net model is trained to recognize tumor boundaries. The trained U-Net model is loaded, the fused image is processed and segmented, and the preliminary segmentation results are output. Doctors view the preliminary segmentation results through visualization software and manually correct the segmentation results using interactive tools.

[0150] Based on the corrected segmentation results, the Marching Cubes algorithm is used to construct a three-dimensional model of the tumor and surrounding important tissues. In the three-dimensional model, the doctor determines the starting and ending points of the puncture according to the tumor location. The path planning algorithm is used to find the optimal path from the starting point to the end point in the three-dimensional model, and the puncture process is simulated to check whether the path is appropriate.

[0151] Connect the augmented reality device to the computer and calibrate it. Use an infrared tracking system to accurately align the three-dimensional model with the patient's actual tumor location to generate an AR view.

[0152] The doctor confirms the optimal puncture path through the AR view and performs the ablation operation under guidance. The doctor disinfects the patient's puncture site, marks the starting point of the puncture, and performs local anesthesia. Under the guidance of the AR view, the doctor performs the puncture operation according to the optimal puncture path. After the ablation is completed, the doctor checks whether there is any abnormality at the puncture site and treats the wound. The patient's vital signs are monitored after the operation, and an imaging review is performed 24 hours after the operation.

[0153] The details are shown in Table 1 below:

[0154] Table 1 Comparison of fusion image and ablation operation effects

[0155]

[0156] In the experiment, the experimental group used the method of the present invention, and the control group used the traditional method. As can be seen from the table data, all indicators of the experimental group were better than those of the control group.

[0157] The average score of the experimental group was 9.5, while the average score of the control group was 7.5, indicating that the fused images in the experimental group were of higher quality, with clearer details and better contrast.

[0158] The average score of the experimental group was 9.8, while the average score of the control group was 8.2. This shows that the segmentation results of the experimental group were more accurate, reducing the workload of doctors' manual correction.

[0159] The average time for the experimental group was 15 minutes, while the average time for the control group was 25 minutes.

[0160] The experimental group had an average of 22 minutes, while the control group had an average of 30 minutes. The experimental group also demonstrated greater efficiency in the ablation procedure, reducing procedure time.

[0161] Through the above data comparison, the present invention has significant advantages in improving the accuracy, efficiency and safety of surgical planning and implementation, providing strong support for clinical practice.

Claims

1. A tumor ablation puncture surgery planning system, characterized by: It includes image acquisition and processing module, image fusion module, segmentation and correction module, puncture path planning module, AR view generation module and ablation operation execution module; The image acquisition and processing module is used to acquire multimodal image data of the tumor area and perform preprocessing; The image fusion module is used to fuse the pre-processed multimodal image data using an image fusion algorithm to generate a fused image; The segmentation and correction module is used to automatically mark the tumor area based on the fused image using image segmentation technology, and the doctor manually corrects the segmentation result; The puncture path planning module is used to select the optimal puncture path through design and simulation based on the fused image with tumor annotation; The AR view generation module is used to accurately align the model with the tumor entity position based on the fused image and augmented reality device using spatial positioning technology to generate an AR view; The ablation operation execution module is used to perform the ablation operation along the optimal puncture path with the puncture needle guided by the AR image; The tumor ablation puncture surgery planning system is used to perform a tumor ablation puncture surgery planning method, which includes the following steps: Acquire multimodal imaging data of the tumor area and perform preprocessing; The pre-processed multimodal image data is fused using an image fusion algorithm to generate a fused image; Based on the fused image, the tumor area is automatically marked using image segmentation technology, and the doctor manually corrects the segmentation results; Based on the fused image with tumor annotations, the optimal puncture path is selected through design and simulation; Based on fused images and augmented reality devices, spatial positioning technology is used to accurately align the model with the actual location of the tumor to generate an AR view; Guided by the AR view, the puncture needle performs ablation along the optimal puncture path; The multimodal image data of the tumor area is obtained and preprocessed, and the specific steps are as follows: Use a CT scanner and an MRI scanner to simultaneously scan the patient's tumor area to obtain multimodal imaging data of the tumor area; Perform normalization on multimodal image data and dynamically adjust normalization parameters according to the local characteristics of the image. The normalization expression is as follows: ； in, is the original image intensity, is the normalized image intensity, and are the maximum and minimum intensity values ​​in the original image, is the adjustment coefficient, is the offset coefficient; The multimodal imaging data are registered and the CT and MRI images are registered to the same coordinate system using a rigid body transformation matrix.

2. The tumor ablation puncture surgery planning system according to claim 1, characterized in that: The image fusion algorithm is used to fuse the pre-processed multimodal image data to generate a fused image. The specific steps are as follows: The Laplacian pyramid fusion method is selected from the fusion algorithm based on multi-resolution analysis to decompose the CT image and MRI image into sub-bands with different resolutions; Through downsampling operation, the original image Reduce it by half, and you get , repeat the downsampling operation on the reduced image to form a series of images ; for , the subbands in the Laplacian pyramid are calculated by the difference between two adjacent layers of images , Generate a series of subbands for the index variable , each sub-band represents high-frequency information at different resolutions; Fusion is performed on each subband, and for each pixel position , select the following fusion rules: ； in, After fusion Layer image at position The pixel value at For the CT images of layers at locations The pixel value at For the MRI images of layers at locations The pixel value at ; and They are and The weight of The fused sub-bands are reconstructed back to the fused image of the original resolution through the inverse Laplacian pyramid, starting from the sub-band with the lowest resolution and gradually reconstructing upwards. At each step, the current sub-band is merged with the reconstructed image of the previous layer to obtain the final fused image.

3. The tumor ablation puncture surgery planning system according to claim 2, characterized in that: Based on the fused image, the image segmentation technology is used to automatically mark the tumor area, and the doctor manually corrects the segmentation result. The specific steps are as follows: The standard deviation of the Gaussian filter is adjusted according to the image characteristics and noise level, and multiple Gaussian filters with different standard deviations are used to denoise the fused image; Divide the image into several small areas, perform histogram equalization on each small area, and reassemble the processed small area images into a complete image; Select a U-Net model and extract a training dataset from the fused image to train the U-Net model's ability to identify tumor boundaries; Add a self-attention mechanism to the encoder and decoder parts of the U-Net model to calculate the importance weights of the feature maps; The preprocessed fusion image is input into the trained U-Net model, and the output is a preliminary segmentation result, which is a binary image representing the tumor area and non-tumor area; Doctors review the preliminary segmentation results and original images through visualization software and manually modify the segmentation results using interactive tools; Using active learning methods, we select the most informative samples from the doctors' correction results through diversity sampling, add the selected samples to the training set, and retrain the U-Net model; The revised segmentation results are integrated into the patient's medical records as a basis for subsequent treatment.

4. The tumor ablation puncture surgery planning system according to claim 3, characterized in that: The optimal puncture path is selected through design and simulation based on the fused image with tumor annotation. The specific steps are as follows: Convert the two-dimensional images of tumors segmented by the U-Net model and corrected by doctors into VTK format, a data format suitable for three-dimensional modeling; The segmented two-dimensional images are stacked into three-dimensional voxel data, where each voxel represents a pixel in the image, and its pixel value indicates the tissue type at its location; A three-dimensional density field is constructed based on the voxel data, where the position and value of each voxel are used to represent the density distribution of the tissue; The three-dimensional density field is divided into several cubes containing 8 vertices, and the density values ​​of the 8 vertices are calculated. According to the density values ​​of these vertices, the intersection points of the isosurface and the cube edges are determined; According to the intersection of the isosurface and the cube edge, triangular patches are generated. These triangular patches form the surface of the three-dimensional model, and the generated triangular patches are smoothed. The Marching Cubes algorithm constructs an intuitive and interactive three-dimensional environment by finding isosurfaces in a three-dimensional density field. The isosurface is a surface consisting of all points whose density value is equal to a certain threshold. In the three-dimensional model, the doctor determines the starting and ending points of the puncture based on the tumor location and the actual location of the tumor; Using the path planning algorithm, the optimal path from the puncture start point to the end point is found in the 3D model while taking into account the avoidance of important tissues, avoiding hard tissue obstacles and maintaining the smoothness of the path. By simulating the puncture process in a 3D model, the path is checked for unforeseen obstacles. If so, the puncture path is replanned.

5. The tumor ablation puncture surgery planning system according to claim 4, characterized in that: Based on the fusion image and augmented reality device, the three-dimensional model is accurately aligned with the tumor entity position using spatial positioning technology to generate an AR view. The specific steps are as follows: Connecting the augmented reality device to the computer and calibrating the augmented reality device; An infrared tracking system installed in the operating room enables precise alignment of the 3D model and the tumor entity with the patient's actual tumor location; The infrared tracking system consists of multiple cameras that capture information from reflective markers attached to medical devices and patients. The sensors on the augmented reality device collect the marker motion data and convert it into a coordinate system in the fused image. The three-dimensional model in the fused image is aligned with the real scene, and the tumor model, puncture path and real scene are superimposed and displayed on the display to generate an AR view.

6. The tumor ablation puncture surgery planning system according to claim 5, characterized in that: The AR image is used as a guide, and the puncture needle performs the ablation operation along the optimal puncture path. The specific steps are as follows: The doctor uses the AR view to reconfirm the optimal puncture path, fine-tune the path through gesture control, and lock the puncture path; Disinfect the patient's puncture site and mark the puncture starting point. Perform local anesthesia on the patient's puncture site based on the patient's weight, age, and health condition. The puncture needle size is selected according to the tumor size and location depth. Under the guidance of the AR view, the doctor performs the puncture operation along the optimal puncture path; When the ablation device connected to the end of the puncture needle reaches the center of the tumor, the ablation device is activated, and the doctor observes the position of the puncture needle and changes in surrounding tissues in real time through the AR view; After the ablation is completed, the puncture needle is removed, the puncture site is checked for abnormalities, and the wound at the puncture starting point is treated; After surgery, the patient's heart rate, blood pressure, and blood oxygen saturation were monitored and recorded, and the recovery of the patient's puncture wound was observed; An imaging review is performed 24 hours after surgery, and the tumor ablation effect is evaluated by combining the review results with the patient's postoperative clinical manifestations; Based on the evaluation results, the next treatment plan is formulated.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, a tumor ablation puncture surgery planning method included in the tumor ablation puncture surgery planning system according to any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a tumor ablation puncture surgery planning method included in the tumor ablation puncture surgery planning system according to any one of claims 1 to 6 is implemented.

Images