Lung three-dimensional reconstruction method and system for intraoperative positioning

Abstract

The invention discloses a lung three-dimensional reconstruction method and system for intraoperative positioning, and the method comprises the steps: processing a lung CT image in a DICOM format through an improved lung parenchyma segmentation algorithm, precisely extracting a lung structure through a global threshold method, connected region analysis and mathematical morphology processing, and then processing a mask generated through segmentation through a real-time smooth interpolation method, and through a dynamic parameter adaptation technology, accurate matching of a segmentation result and a gray value range of modeling data is ensured. According to the method, the problems of artifacts, redundant surfaces and gray level jump of the lung parenchyma area caused by separation of segmentation and reconstruction in a traditional method are avoided, full-process automation from data reading and lung parenchyma segmentation to three-dimensional modeling and visualization is achieved, meanwhile, a user-friendly interaction interface is designed, and a convenient tool is provided for intraoperative navigation.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer image processing, relates to medical image processing, and is a lung three-dimensional reconstruction method and system for intraoperative positioning. Background Art

[0002] Lung parenchyma segmentation technology aims to accurately extract lung parenchyma regions from medical images, providing crucial support for the diagnosis and treatment of lung diseases. DICOM (Digital Imaging and Communications Medical Image) is a standard medical image format commonly used to store and transmit lung CT scan data. Lung parenchyma segmentation technology utilizes image processing and computer vision techniques to analyze and process DICOM images to distinguish lung tissue from other tissues, removing interference from factors such as the body and bed surface that can interfere with the doctor's diagnosis.

[0003] Surface rendering 3D reconstruction technology is a method for extracting 3D surface models from 2D image data. It is widely used in medical imaging, computer graphics, scientific visualization and other fields. Its core idea is to generate a 3D model describing the surface of an object by analyzing and processing a series of 2D slice image data, thereby providing users with intuitive and realistic visual effects. In the field of medical imaging, surface rendering 3D reconstruction technology has important application value. In computed tomography (CT) and magnetic resonance imaging (MRI), it is necessary to reconstruct a 3D model of an organ or tissue from a series of 2D slices to further assist diagnosis, provide imaging support for treatment planning and intraoperative navigation. Through 3D reconstruction, doctors can observe the morphology and location of the lesion more intuitively.

[0004] Currently, lung parenchyma segmentation and 3D reconstruction in the medical image processing field are typically performed as two relatively independent steps. For example, during the segmentation phase, the image data processing module extracts the target region, followed by adjustments to the segmented image using the image data setting module, before finally loading the image into the simulation module for 3D display. Adjustments between segmentation and reconstruction are necessary for the following reasons: 1. Mismatches between the segmentation results and the reconstruction algorithm parameters; 2. Errors may occur during the segmentation phase (such as blurred boundaries and noise interference), which are amplified during the reconstruction process. 3. In existing technologies, segmentation and reconstruction are typically two independent steps, requiring multiple read / write and format conversions during data transfer. These approaches lack a connection between segmentation and modeling, failing to optimize parameter matching between the two, which can lead to problems such as blurred model boundaries and artifacts. Similarly, mask data generated during the segmentation phase is used independently to reconstruct the 3D virtual model. The reconstruction process is not adapted to the grayscale information of the original CT data, resulting in reduced model accuracy, artifacts, redundant surfaces, and distorted illumination calculations.

[0005] In existing image segmentation techniques, there is a natural numerical gap between the binary mask (0 and 1) after segmentation and the continuous grayscale values of CT data (such as -1000 to 3000 HU). This causes the generated data to exhibit drastic grayscale value jumps at the boundaries between target and non-target areas. 3D reconstruction algorithms such as Marching Cubes are extremely sensitive to such jumps and may misinterpret such sudden value changes as true tissue boundaries, affecting the accuracy of the reconstructed model. Improper boundary processing or grayscale value range mismatch in the segmentation results may lead to:

[0006] Model artifacts and redundant surfaces: Due to the mismatch between mask data and modeling input, it is easy to introduce additional artifacts or redundant surface structures during the modeling process, affecting the visualization effect.

[0007] Complex parameter adjustment: Traditional methods require manual adjustment of the isosurface extraction threshold during the modeling process to adapt to the segmented data, which increases computational complexity and the need for manual intervention, affecting modeling efficiency.

[0008] Poor data integration: Segmentation data is not dynamically adapted to the modeling algorithm. Lighting calculation and shadow processing may produce errors due to the discontinuity of input data, reducing the realism of the 3D model. Summary of the Invention

[0009] The problem to be solved by the present invention is that lung parenchyma segmentation and three-dimensional reconstruction in the existing medical image processing field are usually treated as two relatively independent steps. The segmentation results are disconnected from the input data of the three-dimensional reconstruction, which affects the reconstruction effect and the assistance for intraoperative positioning.

[0010] The technical solution of the present invention is: a lung 3D reconstruction method for intraoperative positioning, which generates an image with a grayscale value range consistent with the original data by multiplying the dynamic adjustment mask of lung parenchyma segmentation with the original CT data in real time. At the same time, it combines a smooth interpolation method to optimize lung boundary transitions, enhance lighting and shadow calculations in the 3D modeling stage, and reduce artifacts. The method includes the following steps:

[0011] 1) Lung parenchyma segmentation: Perform image segmentation on lung CT images in DICOM format, eliminating interference from the trunk and bed board to obtain a lung parenchyma mask. Then, perform smooth interpolation on the edge of the lung parenchyma mask so that the boundary area of the lung parenchyma gradually transitions to the background grayscale value, obtaining a continuous grayscale lung parenchyma mask. This mask is multiplied with the original CT data in real time to obtain a segmented lung parenchyma image and save it in DICOM format.

[0012] 2) Lung 3D reconstruction: Rendering and displaying the original CT image and the segmented lung parenchyma reconstructed 3D model, obtaining the original lung 3D model with the body and bed board, as well as the segmented lung parenchyma 3D model;

[0013] 3) User interaction interface display: Design a user interaction application interface to implement interactive functions for the obtained 3D model, including transparency adjustment, 3D model angle switching, and lesion marking, to provide auxiliary support for intraoperative positioning.

[0014] Furthermore, lung parenchyma segmentation is specifically as follows:

[0015] Step 1, data preprocessing:

[0016] Read the lung CT image file in DICOM format, obtain the pixel data of the image, and store the pixel data of the image as a two-dimensional grayscale image matrix, where each element represents the grayscale value of a pixel;

[0017] Step 2, threshold iterative segmentation:

[0018] 1) Calculate the initial threshold: Calculate the average gray value T0 of the image as the initial threshold;

[0019] 2) Threshold iteration:

[0020] For each pixel, according to the current threshold t Original , the image is divided into two regions: background I b and Prospect I f ;

[0021] Calculate foreground area I f The number of pixels I fg and background area I b The number of pixels I bg ;

[0022] Calculate foreground area I f Average grayscale and background area I b Average grayscale

[0023] Update threshold:

[0024] If t Original =t New , then stop the iteration, otherwise t New Set as a new threshold and continue iterating;

[0025] 3) Obtain a binary image: According to the final threshold t Original , the lung CT image is binarized to obtain a binary image, in which the part with pixel value greater than the threshold is marked as foreground, and the part with pixel value less than the threshold is marked as background;

[0026] Step 3, mathematical morphology processing:

[0027] After inverting the binary image, a closed operation is performed using a circular structural unit and then inverted to eliminate noise and small breaks in the image, making the lung area more connected and smooth.

[0028] Step 4: Connected region analysis and extraction:

[0029] Perform connected region analysis on the binary image after morphological processing to find the connected regions in the image, and extract the largest connected region to obtain the torso mask;

[0030] Step 5, hole filling and area filtering:

[0031] The lung parenchyma is the two holes in the middle of the trunk. The mask obtained after filling the lung parenchyma holes in the trunk mask is subtracted from the mask before filling to obtain the mask of the lung parenchyma area. Then, based on the set lung parenchyma threshold, the area smaller than the lung parenchyma threshold is filtered out to remove the trachea and noise to obtain the lung parenchyma mask.

[0032] Step 6, image processing and saving:

[0033] The lung parenchyma mask obtained in step 5 is dynamically adjusted and multiplied with the original CT image in real time. The dynamic adjustment uses a smooth interpolation method to make the mask value of the lung parenchyma boundary area gradually transition and smoothly decrease from 1 to 0. The adjusted mask is multiplied with the original CT data in real time to obtain the lung parenchyma segmentation result. The segmentation result is saved in DICOM format and PNG format with the same metadata as the original CT data for subsequent 3D reconstruction and visualization in the user interaction interface.

[0034] Furthermore, the lung 3D reconstruction and user interaction interface display are specifically as follows:

[0035] 1) Read the original lung CT image and the DICOM format image of the lung parenchyma segmentation, preprocess the image using Gaussian smoothing, and control the smoothing degree by setting the standard deviation;

[0036] 2) Based on the smoothed image data, the Marching Cubes algorithm is applied to extract isosurfaces. For lung parenchymal segmentation images, the lung parenchyma is segmented based on a continuous grayscale lung parenchyma mask. The Marching Cubes algorithm's voxel threshold is dynamically set according to grayscale changes to determine which voxels will be included in the 3D surface. This ensures that the isosurface extraction process matches the grayscale values of the segmented data. By converting the 2D slice data into a 3D surface model, a preliminary 3D reconstruction of the lung parenchyma region is achieved.

[0037] 3) Calculating the normal vector of each vertex in the preliminary 3D reconstruction results, setting a characteristic angle, and smoothing the normal vectors within the set angle range to adjust the lighting effect of the 3D model during rendering. The characteristic angle is set dynamically and adjusted in conjunction with transparency during the rendering stage to make the model suitable for medical observation and enhance the visualization of the internal structure of the lungs;

[0038] 4) Create a mapper and set the mapper's input connection to the output after the normal vector calculation to configure the lighting and shadow effects; create a presentation object, associate the mapper with the presentation object, and set the presentation object's properties;

[0039] 5) Add the created demonstration object to the renderer, which is responsible for processing and displaying the added demonstration object, and finally presenting the 3D reconstructed lung model on the user interaction interface.

[0040] Furthermore, lung parenchyma segmentation and lung three-dimensional reconstruction are performed continuously and automatically. The storage address of the DICOM format image of the lung parenchyma segmentation is set, and the DICOM format image of the lung parenchyma segmentation is automatically read from the specified directory during three-dimensional reconstruction. The DICOM image reader loads these files, obtains image data and performs three-dimensional reconstruction, realizing one-click automatic processing from segmentation to three-dimensional reconstruction.

[0041] The present invention also provides a lung three-dimensional reconstruction system for intraoperative positioning, which is configured with a data processing module, a three-dimensional reconstruction module and a user interaction interface module according to the above-mentioned lung three-dimensional reconstruction method, including

[0042] The data processing module automatically reads the DICOM data of the original lung CT image by specifying the data input and output addresses, performs lung parenchyma segmentation, and saves the segmentation results to the specified path, providing data input for subsequent 3D modeling and user interface visualization;

[0043] The 3D reconstruction module reads DICOM images of lung parenchyma segmentation, performs Gaussian smoothing, applies the Marching Cubes algorithm to dynamically set voxel thresholds based on grayscale changes based on a continuous grayscale lung parenchyma mask, extracts 3D isosurfaces, converts 2D slice data into 3D surface models, and displays the models in the renderer by setting color and transparency properties in the user interface.

[0044] The user interaction interface module creates an embedded rendering window in the application window to display the three-dimensional lung model. The interactive functions include: rotating, translating and scaling the window scene, and realizing model switching, transparency adjustment and lesion location marking through interface controls.

[0045] The present invention proposes a full-process automated lung three-dimensional reconstruction method and system for intraoperative positioning. It adopts a collaboratively optimized lung parenchyma segmentation and three-dimensional modeling process, combined with user interaction design, to optimize the data adaptability and computational efficiency between segmentation and modeling, and has the following excellent results.

[0046] 1. Optimizing lung parenchyma segmentation

[0047] First, a global thresholding method, connected region analysis, and mathematical morphology processing techniques are used to perform high-precision segmentation of DICOM-formatted lung CT images, effectively eliminating interference from the torso and bed board. Specifically, the present invention smoothly interpolates and multiplies the segmentation results with the original CT data in real time, ensuring that the grayscale values of non-target areas smoothly decrease to zero, avoiding the grayscale jump problem encountered in prior art. This dynamic mapping mechanism enables the subsequent 3D reconstruction Marching Cubes algorithm to automatically adapt to the optimal voxel threshold range based on mask grayscale changes. The segmented data not only retains the original grayscale information but is also suitable for modeling calculations, improving the integrity and accuracy of the segmented data.

[0048] 2. Seamless integration of data segmentation and modeling

[0049] The segmentation masks of the prior art are generally used for annotation, and because segmentation and modeling are often designed separately, the problems encountered in three-dimensional reconstruction of lung images obtained by direct use of mask segmentation are not considered. The present invention uses the smooth interpolation processing of the mask and CT data and the segmented data generated by real-time multiplication as input to ensure seamless connection between the reconstruction input and the segmentation result. At the same time, the threshold of the Marching Cubes algorithm is optimized, and the voxel threshold of the MarchingCubes algorithm is dynamically set according to the grayscale change of the mask, so that the isosurface extraction process matches the grayscale value of the segmented data. This method effectively eliminates boundary artifacts, ensures clear model boundaries, and improves modeling accuracy.

[0050] 3. Optimized 3D reconstruction algorithm

[0051] Based on the segmented data, the present invention applies the Marching Cubes algorithm, dynamically setting voxel thresholds based on mask grayscale variations to perform 3D lung reconstruction. Integrating dynamic lighting calculation and transparency adjustment techniques during the modeling process, the original CT image and the segmented lung parenchyma data are rendered and displayed separately, resulting in an original 3D lung model with the torso and bed board, and an optimized 3D model containing only the lung parenchyma. By automatically extracting the CT value range of the target area based on the mask segmentation results and optimizing the matching between the segmented data and the Marching Cubes modeling parameters, artifacts and redundant surfaces are reduced, with the artifact rate reduced by 90%, making the resulting 3D model more realistic and reliable.

[0052] 4. User-friendly interactive design

[0053] Transparency adjustment can flexibly adjust the visualization effect of the 3D model and enhance the observation of the internal structure of the lungs;

[0054] Model switching function, quickly switching between the original lung model and the lung parenchyma model, making it easier for doctors to compare and observe;

[0055] Lesion marking function: users can input lesion coordinates, and the system will automatically mark the corresponding position in the 3D model to assist in precise positioning during surgery.

[0056] 5. Fully automated processing

[0057] 3D reconstruction is a very powerful concept used in almost all image processing techniques, and it is a very powerful concept used in almost all image processing techniques. The dynamic parameter setting of the Cubes algorithm eliminates the need for manual parameter adjustment, and the entire process takes less than 1 minute. Users only need to specify the input and output paths to complete the entire process from data reading and lung parenchyma segmentation to 3D modeling and visualization. This eliminates the need for manual parameter adjustment, greatly simplifying operations and improving efficiency.

[0058] The present invention solves the problems of mismatch between segmentation data and modeling input, artifact generation, and cumbersome manual adjustment in the existing technology by optimizing the seamless connection between lung parenchyma segmentation and three-dimensional modeling and optimizing parameter matching. It also optimizes the data transmission method for segmentation and reconstruction, and realizes a more efficient, more accurate, and more automated three-dimensional reconstruction method of the lungs, providing important technical support for real-time positioning during surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 Schematic diagram of the system of the present invention.

[0060] Figure 23 are comparison diagrams of the original CT data and the segmented lung parenchyma CT data of the present invention, (a) is the original CT data, and (b) is the segmented lung parenchyma CT data.

[0061] Figure 3 Comparison diagram of two lung models of the present invention, (a) is unsegmented, and (b) is segmented.

[0062] Figure 4 These are comparison diagrams of different transparency levels of the present invention, where (a) and (b) are the results presented by different transparency settings.

[0063] Figure 5 Schematic diagram of the simulated lesion marking of the present invention. DETAILED DESCRIPTION

[0064] The present invention designs and implements a lung three-dimensional reconstruction method and system for intraoperative positioning. A collaboratively optimized three-dimensional reconstruction scheme is proposed. A dynamically adjusted mask for lung parenchymal segmentation is multiplied with the original CT data in real time to generate a grayscale value range consistent with the original data. A smooth interpolation method is also used to optimize lung boundary transitions, making illumination and shadow calculations more accurate during the three-dimensional modeling phase. This ensures a seamless transition between segmentation and modeling, reduces artifacts, and improves modeling accuracy. First, an improved global thresholding method, connected region analysis, and mathematical morphology processing techniques are used to perform high-precision segmentation of DICOM-formatted lung CT images, effectively eliminating interference from the trunk and bedboard. A continuous grayscale mask is then used to smoothly interpolate the lung edges, maintaining high grayscale values in the lung parenchyma region and gradually transitioning the boundary region to the background grayscale value. The mask is then multiplied with the original CT data in real time, preserving the grayscale information of the original CT image while providing continuous data input for three-dimensional modeling and avoiding the abrupt boundary issues caused by traditional binary masks. The resulting image of the lung parenchyma segmentation improves the integrity and accuracy of the segmentation results. Then, an optimized 3D modeling algorithm is used to reconstruct the lungs in 3D. The original CT image and the segmented lung parenchyma data are reconstructed into a 3D model, which is rendered and displayed. This results in the original 3D lung model with the body and bed board, as well as the 3D model after segmentation of the lung parenchyma. This ensures a seamless connection between the segmentation data and the modeling process, reduces artifacts and redundant surfaces, and improves modeling accuracy. The entire process is fully automated, streamlining the operational process and improving efficiency. Finally, a user-friendly application interface is designed that supports transparency adjustment, switching between two 3D models, and lesion marking, providing precise auxiliary support for surgical positioning.

[0065] The specific embodiments of the present invention are described below with reference to the accompanying drawings.

[0066] Based on computer vision, this invention combines computer vision, three-dimensional reconstruction and user interface technology to design and implement a lung intraoperative positioning system. The system has one-click operation throughout the entire process, is simple and easy to understand, has low resource requirements, does not require high-end equipment support, and is easy to use.

[0067] like Figure 1 As shown, the present invention uses computer vision technology to process lung CT images in DICOM format, adopts a lung parenchyma segmentation algorithm to extract the lung structure, eliminates the interference of redundant trunk and bed board, and then performs three-dimensional reconstruction of the lungs. The original CT image and the segmented lung parenchyma data are reconstructed into a three-dimensional model for rendering and display. After three-dimensional reconstruction, the original lung three-dimensional model with the trunk and bed board and the three-dimensional model of the lung parenchyma segmentation are obtained, and a user interface is designed to provide user interaction functions.

[0068] The lung parenchyma segmentation algorithm is described in detail as follows.

[0069] Step 1, data preprocessing:

[0070] Read the DICOM format lung CT image file at the address set by the user and obtain the pixel data of the image. The pixel data of the image is stored as a two-dimensional grayscale image matrix, where each element represents the grayscale value of a pixel.

[0071] Step 2, threshold iterative segmentation:

[0072] 1) Calculate the initial threshold: First, calculate the average gray value T0 of the image as the initial threshold.

[0073] 2) Iterative process:

[0074] For each pixel I(x,y), (x,y) is the pixel coordinate, according to the current threshold t Original , the image is divided into two regions: background I b and Prospect I f ;

[0075] Calculate foreground area I f The number of pixels I fg and background area I b The number of pixels I bg ;

[0076] Calculate foreground area I f Average grayscale and background area I b Average grayscale

[0077] Update the threshold value according to the following formula:

[0078] If t Original=t New , then stop the iteration, otherwise t New Set it as the new threshold and continue iterating.

[0079] 3) Obtain a binary image: According to the final threshold t Original , the image is binarized to obtain a binary image, in which the part with pixel value greater than the threshold is marked as foreground, and the part with pixel value less than the threshold is marked as background.

[0080] Step 3, mathematical morphology processing:

[0081] To eliminate small cavities and noise within the lung parenchyma, the binary image is inverted to highlight the noise. Next, a closing operation is performed using circular building blocks with a radius of 5 to fill the small cavities within the lung parenchyma and smooth the boundaries without changing the area, making the lung region more connected and smooth. After completing the closing operation, the image is inverted.

[0082] Step 4: Connected region analysis and extraction

[0083] After the above steps, noise and holes have been eliminated. Next, we need to remove interference from the bed board. Connected region analysis is performed on the morphologically processed binary image to identify connected regions within the image. Clearly, there are two connected regions in the processed binary image: the larger lungs and trunk, and the smaller bed board. By extracting the largest connected region, we can obtain a mask for the lungs and trunk, removing interference from the bed board.

[0084] Step 5: Hole filling and area filtering

[0085] After threshold segmentation, it can be clearly seen that the lung trunk and lung parenchyma are marked as two parts. The lung parenchyma is the two holes in the middle of the trunk. The mask obtained after filling the lung parenchyma holes in the trunk mask is subtracted from the mask before filling. The trunk part can be removed to obtain the mask of the lung parenchyma area. Then, according to the set lung parenchyma threshold, the area with an area smaller than the lung parenchyma threshold is filtered out to remove the trachea and noise, and the final lung parenchyma mask is obtained.

[0086] Step 6: Image processing and saving

[0087] The lung parenchyma mask obtained in step 5 is dynamically adjusted and multiplied in real time with the original CT image to ensure a grayscale value range consistent with the original CT data, optimize the accuracy of subsequent 3D modeling, and obtain an image of the segmented lung parenchyma. Dynamic adjustment uses a smooth interpolation method to gradually transition the mask value of the lung parenchyma boundary area from 1 to 0, avoiding the grayscale mutation problem caused by traditional binary masks (0 or 1). The adjusted mask is multiplied with the original CT data in real time, not only removing non-lung areas but also preserving the original grayscale information of the lung parenchyma, enhancing the accuracy of lighting and shadow calculations during 3D modeling. The segmentation results are saved in DICOM and PNG formats with the same metadata as the original data for subsequent 3D reconstruction and visualization. The DICOM format is used for subsequent 3D modeling, and the PNG format is used for user visualization in the user interface.

[0088] This invention solves the disconnect between segmentation and reconstruction in existing techniques by combining real-time mask multiplication with CT data and dynamic threshold adaptation, achieving a seamless transition. During the multiplication process, the invention not only preserves the original CT value information of the target area but also ensures that the grayscale values of non-target areas decrease smoothly to zero through real-time smooth interpolation, avoiding the grayscale jump problem seen in existing techniques. This dynamic mapping mechanism enables the MarchingCubes algorithm to automatically adapt the optimal voxel threshold range based on the mask during subsequent 3D reconstruction, without manual intervention. This reduces the artifact rate from 15% in traditional methods to below 2%. For example, suppose the lung pixel values in the original CT data range from 800 to 1200. However, in existing techniques, the post-segmentation mask only performs binarization: the lung region mask = 1, and other regions mask = 0. This creates an unnatural grayscale transition from 0 (background) to 800 (the lowest value in the lungs) in the CT data. When processing this boundary, MarchingCubes can easily misjudge and generate additional surfaces. Through dynamic mapping and smooth interpolation, the present invention retains the original CT value of the lung area, and smoothly transitions other (trunk) areas to 0, ensuring that the grayscale value of the lung parenchyma area does not form a sudden change at the boundary.

[0089] Figure 2 As shown, it is a comparison diagram of the original data and the data after lung parenchyma segmentation as described in the present invention. It can be seen that the segmented lung parenchyma part successfully removes the interference of the redundant torso and bed board part, which can more clearly observe the lung condition.

[0090] Building on the aforementioned lung parenchyma segmentation implementation, the present invention reads segmented lung parenchyma DICOM data for 3D reconstruction. By dynamically analyzing the voxel values within the 3D dataset and determining the isosurfaces between voxels, the 3D surface of the object is constructed. Because only lung parenchyma data needs to be processed, the computational effort and resources required are minimal, enabling implementation on a standard computer.

[0091] The detailed steps of lung 3D reconstruction are described below.

[0092] 1) Set the DICOM data storage address of the original CT image and the segmented lung parenchyma image, read the DICOM format lung CT image files from the specified directory, load these files through the DICOM image reader, obtain the image data, and ensure the complete reading of the CT image data.

[0093] 2) To address the common noise interference problem in lung CT images, Gaussian smoothing is used to preprocess the images. The degree of smoothing is precisely controlled by setting the standard deviation to optimize image quality, reduce the impact of noise on 3D modeling, and ensure clearer details during reconstruction.

[0094] 3) Based on the smoothed image data, the CT value range of the target area is automatically extracted according to the mask segmentation results and used as the input threshold of the Marching Cubes algorithm. The Marching Cubes algorithm is used to extract the isosurface. The voxel threshold of the Marching Cubes algorithm is dynamically set according to the grayscale changes of the mask to determine which voxels will be included in the 3D surface. By converting the 2D slice data into a 3D surface model, a preliminary 3D reconstruction of the lung structure is achieved.

[0095] Traditional Marching Cubes algorithms typically require manual setting of fixed voxel thresholds when extracting isosurfaces, which can easily lead to segmentation error propagation, resulting in artifacts or redundant surfaces. This method multiplies the binary mask generated by lung parenchyma segmentation with the original CT data in real time to generate a grayscale value range consistent with the original data. The Marching Cubes algorithm's voxel threshold is then dynamically set based on the mask's grayscale, ensuring that the isosurface extraction process accurately matches the grayscale values of the segmented data. Furthermore, the Marching Cubes algorithm, combined with dynamic lighting calculation and transparency adjustment techniques, smoothes the normal vector of each vertex, ensuring a more realistic lighting effect when rendering the model.

[0096] 4) Calculate the normal vector of each vertex of the preliminary 3D reconstruction results, set the characteristic angle, and smooth the normal vector within the set angle range to adjust the lighting effect of the 3D model during rendering. Incorporate transparency adjustment function during the rendering stage to make the model more suitable for medical observation and enhance the visualization ability of the internal structure of the lungs.

[0097] 5) Create a mapper, set the mapper's input connection to the output of the normal vector calculation to ensure correct lighting and shadow effects, create a presentation object, associate the mapper with the presentation object, and set the presentation object's properties.

[0098] 6) Add the created demonstration object to the renderer, which is responsible for processing and displaying the added demonstration object, and finally presenting the 3D reconstructed lung model on the interface.

[0099] After the above 3D model reconstruction steps, a 3D model of the lung parenchyma is obtained. To facilitate user interaction and ease of use, the present invention has designed a user interface: First, an application window is created, and then a special rendering window is created within this window, which can be embedded within the initial window. Next, an initial renderer is set up and the objects to be displayed are added to it, resulting in the 3D lung model described above. User interaction is then initialized, allowing users to manipulate the scene through rotation, translation, and scaling. Other controls and functions, such as buttons or menus, are added to the application to interact with the rendering window, and the application's event loop is run to ensure proper functioning of the window and user interaction. In the present invention, the user can switch between displaying the original 3D lung model and the 3D lung parenchyma model using a button, and the transparency of the model can be adjusted using a function button to enhance the physician's ability to observe the lung interior more clearly. Furthermore, the present invention provides a simulated lesion marking function. By entering lesion coordinates in the interactive interface, the physician will mark the corresponding location in the model with a small ball, simulating the lesion's location and providing accurate positioning and guidance for lung surgery.

[0100] Based on the above-mentioned lung three-dimensional reconstruction method, the present invention provides a lung three-dimensional reconstruction system for intraoperative positioning, including a data processing module, a three-dimensional reconstruction module and a user interface module.

[0101] The data processing module automatically reads the DICOM data of the original CT image and performs precise segmentation of the lung parenchyma by specifying the data input and output addresses. The segmentation algorithm effectively eliminates interference from non-target areas such as the trunk and bed board, avoiding the grayscale jump problem that occurs in existing technologies. It generates lung parenchyma DICOM data and corresponding PNG data with the same metadata as the original data, and saves them to the specified output path, providing high-quality data input for subsequent three-dimensional modeling and visualization. After the lung parenchyma segmentation, the data is free of noise, the trunk and bed board parts are eliminated, and only the lung parenchyma part is retained. The grayscale value of the lung parenchyma area will not form a sudden change at the boundary, allowing doctors to see the lung structure more intuitively without being interfered with by other factors, and to complete the diagnosis more accurately.

[0102] 3D reconstruction module, after the lung parenchyma segmentation is completed, this module performs Gaussian smoothing on the segmented DICOM data based on the segmentation results to reduce the impact of noise. Subsequently, the Marching Cubes algorithm is applied to dynamically set the voxel threshold according to the grayscale change based on the continuous grayscale lung parenchyma mask, extract the three-dimensional isosurface, and accurately convert the two-dimensional slice data into a three-dimensional surface model. The model is optimized by setting properties such as color and transparency and displayed in the renderer. By switching the display of the original CTCT image and the three-dimensional model of the lung parenchyma segmentation, the user can dynamically compare and analyze the lung structure. The system also supports real-time response to user interaction to ensure that the model display changes with the operation.

[0103] The user interface module features an intuitive user interface. It first creates an embedded rendering window within the application window to display the 3D lung model. Users can interactively rotate, translate, and zoom the window scene, and use interface controls to switch models, adjust transparency, and mark lesion locations. This interface not only enhances user operability of the model but also enables precise assistance in intraoperative lung positioning and lesion marking, significantly improving the flexibility and operability of clinical applications.

[0104] like Figure 3 As shown in FIG, two models are generated by 3D reconstruction of original DICOM data and segmented lung parenchyma data using the lung 3D reconstruction algorithm constructed by the present invention. Figure 3 The model (a) completely retains the lungs and the human body, allowing users to observe the spatial relationship between the lungs and the surrounding human tissues in a three-dimensional image. Figure 3 The model (b) is a 3D model of only the lung parenchyma, helping users eliminate interference from extraneous parts, observe the lung interior more clearly, and make a more accurate diagnosis. Users can switch between the two models using the model switch button on the user interface.

[0105] Figure 4 The following figure shows a comparison of the model transparency using the transparency button on the user interface. Adjusting the model's transparency allows users to observe and analyze the internal conditions more deeply, more clearly viewing the tissue structures surrounding the lesion, while ensuring the integrity of the model, facilitating more accurate assessment of lung conditions.

[0106] Figure 5 The figure below shows the effect of marking a simulated lesion with a small ball at the corresponding coordinate position using the lesion coordinate input button on the user interface. When the user finds an area suspected of being a lesion, they can use this method to mark it with a small ball, assisting the doctor in surgical positioning and guidance, improving the accuracy and success rate of the operation. Furthermore, the user can manipulate the scene by rotating, translating, and zooming, providing a comprehensive view of the model.

Claims

1. A three-dimensional lung reconstruction method for intraoperative positioning, characterized by The dynamic adjustment mask for lung parenchyma segmentation is multiplied with the original CT image in real time to generate an image with a grayscale value range consistent with the original data. At the same time, a smooth interpolation method is combined to optimize the lung boundary transition, enhance the lighting and shadow calculation in the 3D modeling stage, and reduce artifacts. The following steps are included: 1) Lung parenchyma segmentation: Perform image segmentation on lung CT images in DICOM format, eliminating interference from the trunk and bed board to obtain a lung parenchyma mask. Then, perform smooth interpolation on the edges of the lung parenchyma mask so that the boundary area of the lung parenchyma gradually transitions to the background grayscale value. This continuous grayscale lung parenchyma mask is then multiplied with the original CT image in real time to obtain a segmented lung parenchyma image and save it in DICOM format. 2) Lung 3D reconstruction: Rendering and displaying the original CT image and the segmented lung parenchyma reconstructed 3D model, obtaining the original lung 3D model with the body and bed board, as well as the segmented lung parenchyma 3D model; 3) User interaction interface display: Design a user interaction application interface to implement interactive functions for the obtained 3D model, including transparency adjustment, 3D model angle switching, and lesion marking, to provide auxiliary support for intraoperative positioning.

2. A lung three-dimensional reconstruction method for intraoperative positioning according to claim 1, characterized in that Lung parenchyma segmentation is specifically as follows: Step 1, data preprocessing: Read the lung CT image file in DICOM format, obtain the pixel data of the image, and store the pixel data of the image as a two-dimensional grayscale image matrix, where each element represents the grayscale value of a pixel; Step 2, threshold iterative segmentation: 1) Calculate the initial threshold: Calculate the average gray value T0 of the image as the initial threshold; 2) Threshold iteration: For each pixel, according to the current threshold t Original , the image is divided into two regions: background I b and Prospect I f ; Calculate foreground area I f The number of pixels I fg and background area I b The number of pixels I bg ; Calculate foreground area I f Average grayscale and background area I b Average grayscale Update threshold: If t Original =t New , then stop the iteration, otherwise t New Set as a new threshold and continue iterating; 3) Obtain a binary image: According to the final threshold t Original , the lung CT image is binarized to obtain a binary image, in which the part with pixel value greater than the threshold is marked as foreground, and the part with pixel value less than the threshold is marked as background; Step 3, mathematical morphology processing: After inverting the binary image, a closed operation is performed using a circular structural unit and then inverted to eliminate noise and small breaks in the image, making the lung area more connected and smooth. Step 4: Connected region analysis and extraction: Perform connected region analysis on the binary image after morphological processing to find the connected regions in the image, and extract the largest connected region to obtain the torso mask; Step 5, hole filling and area filtering: The lung parenchyma is the two holes in the middle of the trunk. The mask obtained after filling the lung parenchyma holes in the trunk mask is subtracted from the mask before filling to obtain the mask of the lung parenchyma area. Then, based on the set lung parenchyma threshold, the area smaller than the lung parenchyma threshold is filtered out to remove the trachea and noise to obtain the lung parenchyma mask. Step 6, image processing and saving: The lung parenchyma mask obtained in step 5 is dynamically adjusted and multiplied with the original CT image in real time. The dynamic adjustment uses a smooth interpolation method to make the mask value of the lung parenchyma boundary area gradually transition and smoothly decrease from 1 to 0. The adjusted mask is multiplied with the original CT image in real time to obtain the lung parenchyma segmentation result. The segmentation result is saved in DICOM format and PNG format with the same metadata as the original CT data for subsequent 3D reconstruction and visualization in the user interaction interface.

3. The method for three-dimensional reconstruction of the lungs for intraoperative positioning according to claim 1, characterized in that The lung 3D reconstruction and user interface display are as follows: 1) Read the original lung CT image and the DICOM format image of the lung parenchyma segmentation, preprocess the image using Gaussian smoothing, and control the smoothing degree by setting the standard deviation; 2) Based on the smoothed image data, the Marching Cubes algorithm is applied to extract isosurfaces. For lung parenchymal segmentation images, the lung parenchyma is segmented based on a continuous grayscale lung parenchyma mask. The Marching Cubes algorithm's voxel threshold is dynamically set according to grayscale changes to determine which voxels will be included in the 3D surface. This ensures that the isosurface extraction process matches the grayscale values of the segmented data. By converting the 2D slice data into a 3D surface model, a preliminary 3D reconstruction of the lung parenchyma region is achieved. 3) Calculating the normal vector of each vertex in the preliminary 3D reconstruction results, setting a characteristic angle, and smoothing the normal vectors within the set angle range to adjust the lighting effect of the 3D model during rendering. The characteristic angle is set dynamically and adjusted in conjunction with transparency during the rendering stage to make the model suitable for medical observation and enhance the visualization of the internal structure of the lungs; 4) Create a mapper and set the mapper's input connection to the output of the normal vector calculation to configure the lighting and shadow effects; Create a presentation object, associate a mapper with the presentation object, and set the presentation object's properties; 5) Add the created demonstration object to the renderer, which is responsible for processing and displaying the added demonstration object, and finally presenting the 3D reconstructed lung model on the user interaction interface.

4. The method for three-dimensional reconstruction of lungs for intraoperative positioning according to claim 1, characterized in that Lung parenchyma segmentation and lung 3D reconstruction are performed continuously and automatically. The storage address of the DICOM format images of the lung parenchyma segmentation is set. During 3D reconstruction, the DICOM format images of the lung parenchyma segmentation are automatically read from the specified directory. The DICOM image reader loads these files, obtains image data and performs 3D reconstruction, realizing one-click automatic processing from segmentation to 3D reconstruction.

5. A lung three-dimensional reconstruction system for intraoperative positioning, characterized by The lung three-dimensional reconstruction method according to any one of claims 1 to 4 is configured with a data processing module, a three-dimensional reconstruction module and a user interaction interface module, comprising: The data processing module automatically reads the DICOM data of the original lung CT image by specifying the data input and output addresses, performs lung parenchyma segmentation, and saves the segmentation results to the specified path, providing data input for subsequent 3D modeling and user interface visualization; The 3D reconstruction module reads the DICOM format image of the lung parenchyma segmentation from the original lung CT image, performs Gaussian smoothing, applies the Marching Cubes algorithm to dynamically set the voxel threshold based on the grayscale changes based on the continuous grayscale lung parenchyma mask, extracts the 3D isosurface, converts the 2D slice data into a 3D surface model, and displays the model in the renderer by setting the color and transparency properties in the user interface. The user interaction interface module creates an embedded rendering window in the application window to display the three-dimensional lung model. The interactive functions include: rotating, translating and scaling the window scene, and realizing model switching, transparency adjustment and lesion location marking through interface controls.

Images