A continuous slice three-dimensional reconstruction method based on CT space reference constraint

Abstract

The application discloses a continuous slice three-dimensional reconstruction method based on CT space reference constraint and belongs to the technical field of image processing, which comprises the following steps: acquiring a CT scanning image sequence of paraffin-embedded tissue before slicing, and constructing a space reference position corresponding to each CT scanning image in the CT scanning image sequence; acquiring continuous slice images corresponding to the paraffin-embedded tissue; performing cross-modal image registration on the CT scanning images and the continuous slice images to obtain registered continuous slice images; performing space position calibration on the registered continuous slice images according to the space reference position corresponding to the CT scanning images; and performing image superposition on the calibrated continuous slice images or segmentation masks thereof and completing three-dimensional reconstruction. The application can reduce registration error accumulation and improve the spatial consistency and structural fidelity of three-dimensional reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method for three-dimensional reconstruction of continuous slices based on CT spatial reference constraints. Background Technology

[0002] 3D reconstruction technology is a multidisciplinary technology supported by digital image processing, computer graphics, and artificial intelligence algorithms. It achieves 3D reconstruction of biological tissue structures and their biological information by extracting features, spatially registering, and modeling discrete 2D image sequences or multi-source detection data. While retaining the advantages of high resolution and high signal-to-noise ratio in 2D imaging, this technology can reconstruct the 3D topological structure and in-situ architectural features of tissues on a spatial scale. It has been widely applied in fields such as the analysis of the microstructure of biological tissues, the spatial distribution of pathological features, and the study of heterogeneity, becoming an important technical tool in biomedical research.

[0003] In existing technologies, digital 3D reconstruction methods based on serial biological tissue sections have become an important technical approach for analyzing tissue microstructure due to their compatibility with traditional pathological procedures and good multimodal compatibility. These methods typically involve: sequential ultrathin sectioning, staining, and digital scanning of biological samples to obtain a two-dimensional image sequence; subsequent preprocessing, image registration, and image overlay using computer algorithms to construct 3D volumetric data; and finally, combining image segmentation and information annotation techniques to achieve a 3D representation of tissue structure and function. With the development of computational pathology, existing technologies have proposed image registration methods combining rigid and non-rigid transformations, as well as semantic segmentation and structural recognition methods incorporating deep learning, to improve the alignment accuracy between sections and the ability to identify tissue components, thereby achieving 3D reconstruction of larger-scale tissues at cellular resolution.

[0004] In the aforementioned technical process, image registration is a crucial step in achieving 3D reconstruction. It estimates the spatial transformation relationship between adjacent slices by optimizing the similarity measurement between images and combining it with deformation constraints, thereby restoring the tissue continuity disrupted by the slicing process. However, in the preparation of serial biological tissue slices, operations such as cutting, spreading, and mounting inevitably introduce mechanical deformation and sample preparation artifacts. Mechanical deformation mainly manifests as nonlinear deformations such as stretching and compression, while sample preparation artifacts include wrinkles, tears, and local defects. These factors make it difficult for registration methods based on the slice image's own information to accurately establish stable spatial correspondences. Especially when using a technique of registering adjacent slices one by one, errors are easily accumulated, thus affecting the overall reconstruction accuracy.

[0005] After image registration, 3D volumetric data is typically constructed by image overlay, which involves stacking registered 2D slices layer by layer along the spatial direction in slice order. Some improved schemes enhance registration accuracy by introducing feature point matching or energy function optimization strategies, and achieve 3D reconstruction based on layer-by-layer transformation accumulation. However, in the prior art closest to this application, the image overlay process still primarily relies on layer-by-layer registration results, and its spatial transformation is also cumulative, easily leading to overall morphological shifts or local structural distortions in the reconstructed volumetric data. Furthermore, due to unavoidable discontinuities in slices and tissue defects, relying solely on image similarity for alignment cannot guarantee the global consistency of the 3D structure, thus affecting the geometric accuracy and spatial continuity of the reconstruction results. In addition, the aforementioned prior art generally lacks a stable external spatial reference or global structural constraints, making the reconstruction process highly sensitive to slice quality and registration accuracy, and difficult to accurately restore the true 3D spatial topology of biological tissue. For whole tissue samples with curvature changes, such as the meniscus, pre-segmentation is usually required due to considerations of slice quality and operational difficulty. If only slice image registration is relied upon without inter-block spatial reference, the reconstructed tissue blocks are prone to problems such as unnatural connection between blocks, inaccurate superposition, or even local overlap because the direction of tissue curvature change is inconsistent with the direction of tissue slice normal.

[0006] Therefore, in order to address the problems of error accumulation and structural distortion caused by relying solely on the information of the sliced ​​images themselves for layer-by-layer registration and overlay in existing technologies, how to introduce a reliable external spatial reference to provide global constraints, thereby improving the accuracy and stability of image registration and image overlay, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] The purpose of this invention is to address the problems of registration error accumulation, inaccurate spatial positioning, and easy overall distortion of the reconstructed structure in existing three-dimensional reconstruction techniques based on continuous biological tissue slices. This invention proposes a continuous slice three-dimensional reconstruction method based on external spatial reference constraints to improve the registration accuracy of slice images and the spatial consistency of three-dimensional reconstruction.

[0008] To achieve the above objectives, this invention provides a method for three-dimensional reconstruction of continuous slices based on CT spatial reference constraints, comprising: acquiring a sequence of CT scan images of paraffin-embedded tissue before slicing, and constructing spatial reference positions corresponding to each CT scan image in the CT scan image sequence; acquiring continuous slice images corresponding to the paraffin-embedded tissue; performing cross-modal image registration on the CT scan images and the continuous slice images to obtain registered continuous slice images; calibrating the spatial position of the registered continuous slice images according to the spatial reference positions corresponding to the CT scan images; and overlaying the calibrated continuous slice images or their segmentation masks to complete three-dimensional reconstruction.

[0009] As a further improvement to the above technical solution, the step of acquiring the continuous slice image includes: performing tissue segmentation and paraffin embedding on the collected biological tissue sample to form at least one paraffin-embedded tissue; performing continuous slicing, staining and whole-slide imaging on the paraffin-embedded tissue to obtain continuous slice images with slice numbers; the slice numbers are used to characterize the preparation order of the continuous slice images.

[0010] As a further improvement to the above technical solution, before performing cross-modal image registration on the CT scan image and the continuous slice image, the method further includes converting the CT scan image and the continuous slice image into a unified structure image. The conversion steps include: determining the continuous slice image corresponding to the CT scan image in the tissue slice normal direction according to the spatial reference position corresponding to the CT scan image and the preparation order of the continuous slice image; extracting tissue regions from the CT scan image and the corresponding continuous slice image respectively and performing binarization processing to obtain their respective binary structure images.

[0011] As a further improvement to the above technical solution, the step of obtaining the registered continuous slice image includes: extracting matching features between the CT scan image and the continuous slice image based on the binary structure image of the CT scan image and the binary structure image of the corresponding continuous slice image; constructing global registration relationship information between the CT scan image and the continuous slice image based on the matching features; and performing joint optimization of rigid transformation and non-rigid transformation on the continuous slice image based on the global registration relationship information to obtain the registered continuous slice image; the rigid transformation is used to realize translation, rotation and scaling alignment, and the non-rigid transformation is used to correct tissue deformation and slice distortion.

[0012] As a further improvement to the above technical solution, under the condition that the cross-modal image registration is performed in the downsampled image space, obtaining the registered continuous slice image further includes: acquiring downsampled registration transformation information obtained by joint optimization of the rigid transformation and the non-rigid transformation; mapping the downsampled registration transformation information to the original high-resolution space to obtain high-resolution registration transformation information; and performing spatial transformation on the continuous slice image that has not undergone downsampling processing based on the high-resolution registration transformation information to obtain the high-resolution registered continuous slice image.

[0013] As a further improvement to the above technical solution, the spatial position calibration step includes: calibrating the registered continuous slice images corresponding to the CT scan images to the corresponding spatial reference positions to form reference anchor slices; based on the spatial reference position of the reference anchor slices, the preparation order of the continuous slice images and the slice thickness of the continuous slices, determining the spatial positions of the remaining registered continuous slice images along the tissue slice normal direction to obtain a spatially calibrated continuous slice image sequence.

[0014] As a further improvement to the above technical solution, the step of generating the segmentation mask includes: performing target structure segmentation on the continuous slice image to generate a binary segmentation mask corresponding to the target structure; under the condition that the continuous slice image is processed into blocks, performing target structure segmentation on each block image to obtain a binary segmentation mask corresponding to each block image, and stitching the binary segmentation masks corresponding to each block image back to the full-view scale according to the spatial position of the block images in the continuous slice image to obtain a full-view binary segmentation mask; under the condition that the continuous slice image is not processed into blocks, using the binary segmentation mask corresponding to the target structure as the full-view binary segmentation mask; and establishing the correspondence between the full-view binary segmentation mask and the calibrated continuous slice image based on the registration result and spatial position calibration result of the continuous slice image to obtain a segmentation mask corresponding to the calibrated continuous slice image.

[0015] As a further improvement to the above technical solution, the three-dimensional reconstruction steps include: according to the spatial position of the calibrated continuous slice image, performing interlayer overlay on the calibrated continuous slice image or its corresponding segmentation mask to generate three-dimensional volume data; under the condition that multiple paraffin-embedded tissues correspond to different tissue blocks, based on the spatial reference position of the CT scan image corresponding to each tissue block, performing inter-block stitching on the three-dimensional volume data corresponding to each tissue block to obtain stitched three-dimensional volume data; and generating a three-dimensional reconstruction model based on the three-dimensional volume data.

[0016] As a further improvement to the above technical solution, before obtaining the CT scan image sequence before the paraffin-embedded tissue section, the method further includes: obtaining the MRI image sequence of the biological tissue used to form the paraffin-embedded tissue; performing tissue region segmentation on the MRI image sequence, and performing three-dimensional reconstruction based on the tissue region segmentation results to obtain a biological tissue appearance morphology model; determining the appearance contour of the biological tissue according to the biological tissue appearance morphology model, and providing an appearance contour reference for the tissue block processing of the biological tissue.

[0017] Compared with existing technologies, the continuous slice three-dimensional reconstruction method proposed in this invention has the following advantages: (1) Significantly reduces registration error accumulation and improves overall spatial consistency. Existing technologies typically employ adjacent slices for pairwise registration and cascade stacking, which easily leads to the propagation and accumulation of errors between layers, causing overall structural drift or distortion. This invention constructs a cross-modal registration system for CT images and stained slice images, and establishes a global relationship network in conjunction with the VALIS algorithm. Registration is completed under a rigid and non-rigid joint optimization framework, effectively avoiding the error accumulation problem caused by traditional layer-by-layer cascading.

[0018] (2) Achieving a unity between macroscopic structural constraints and microscopic morphological restoration, thereby improving the fidelity of 3D reconstruction. Existing methods only perform registration based on local features of slices, lacking a stable macroscopic structural reference, which easily leads to overall morphological distortion. This invention utilizes the overall contour data provided by MRI and the continuous, complete volume data structure provided by CT images to guide the slice registration and overlay process.

[0019] (3) Effectively solves the problems of slice overlap and tomography, and improves the stability of superposition. In view of the problems of inter-block overlap and spatial misalignment caused by missing slices in the prior art, the present invention proposes a slice spatial positioning method based on CT images. By mapping the stained slices to the unified CT coordinate system and interpolating and filling in the actual slice thickness, the continuous spatial calibration of all slices is achieved. The reconstruction results of the present invention are continuous and without obvious tomography in both visual and quantitative evaluation, indicating that the method has higher robustness when dealing with incomplete data. At the same time, for the whole tissue block reconstruction scenario, the spatial reference position provided by the CT scan image sequence can constrain the relative spatial relationship and curvature change between each tissue block, so that the tissue blocks after block reconstruction can transition continuously when splicing, reduce the inaccurate superposition or local overlap between blocks caused by lack of spatial position calibration, and improve the problem of unnatural connection between tissue blocks.

[0020] (4) Improve cross-modal registration robustness and enhance the adaptability of complex tissues. In the registration process, this invention combines the multi-feature fusion strategy of the VALIS framework, namely, using Daisy and Disk, and unifies the structural expression of different modal images through binary segmentation, which effectively reduces the impact of factors such as staining differences and inconsistent gray-scale distribution on registration.

[0021] (5) Avoid high-resolution information loss and improve the ability to preserve structural details. The present invention adopts a strategy of low-resolution registration and high-resolution mapping restoration, applying the registration transformation parameters to the original high-resolution image to avoid information loss caused by multiple interpolations.

[0022] (6) Achieve consistent optimization of the entire process from data acquisition to 3D reconstruction. This invention organically combines MRI modeling, CT constraints, histological slicing, deep learning segmentation and 3D reconstruction to form a complete technical chain. Targeted optimization strategies are introduced in each key link to achieve a synergistic improvement in overall reconstruction accuracy and stability. Attached Figure Description

[0023] Figure 1 This is an overall framework diagram of a three-dimensional reconstruction method for continuous tissue sections in one embodiment of the present invention; Figure 2 This is a flowchart illustrating the three-dimensional reconstruction method for continuous tissue sections according to another embodiment of the present invention. Figure 3 This is a flowchart of continuous slice image registration in one embodiment of the present invention; Figure 4 This is a schematic diagram of a method for determining the spatial position of a continuous slice image in one embodiment of the present invention; Figure 5 This is a schematic diagram of the three-dimensional reconstruction result of continuous slices in one embodiment of the present invention; Figure 6 This is a flowchart of continuous slice cross-modal registration and three-dimensional reconstruction in one embodiment of the present invention. Detailed Implementation

[0024] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. In the absence of conflict, the technical features in the embodiments of the present invention can be combined with each other; steps, conditions, parameters or software operations not described in detail can all be implemented by conventional technical means in the art.

[0025] In one embodiment, see Figure 1 This invention provides a method for three-dimensional reconstruction of continuous slices based on CT spatial reference constraints, comprising: acquiring a sequence of CT scan images of paraffin-embedded tissue before slicing; constructing spatial reference positions corresponding to each CT scan image in the CT scan image sequence; acquiring continuous slice images corresponding to the paraffin-embedded tissue; performing cross-modal image registration on the CT scan images and the continuous slice images to obtain registered continuous slice images; calibrating the spatial position of the registered continuous slice images according to the spatial reference positions corresponding to the CT scan images; and overlaying the calibrated continuous slice images or their segmentation masks to complete three-dimensional reconstruction. In the above method, before cross-modal image registration, the CT scan images and continuous slice images can be converted into images with a unified structure. The specific processes of tissue region extraction, binarization processing, downsampling spatial registration, high-resolution restoration, segmentation mask generation, and three-dimensional model construction can be performed according to the following embodiments.

[0026] like Figure 2 As shown, the operation process of this embodiment includes the following steps: constructing a three-dimensional geometric model of the tissue using MRI images and performing tissue segmentation, paraffin embedding, CT scanning, serial sectioning, and HE staining accordingly; segmenting, labeling, and target segmenting the stained section images to generate and stitch together a full-view binary segmentation mask; using the CT scan image as a spatial reference, performing cross-modal registration and spatial position calibration of the serial section images and the corresponding CT scan images; and finally, superimposing the calibrated images or segmentation mask to complete the three-dimensional reconstruction of the tissue.

[0027] In one embodiment, before acquiring the CT scan image sequence, the method further includes constructing an appearance morphology model of the biological tissue corresponding to the paraffin-embedded tissue.

[0028] Construction of the morphological model: Based on MRI imaging data, a three-dimensional geometric model of the biological tissue was constructed using Mimics 21.0 software. First, the original MRI sequence of the biological tissue was acquired and imported into Mimics 21.0 software in DICOM format. Image filtering algorithms were applied to preprocess the original MRI sequence for smoothing, denoising, and contrast enhancement. In the target region extraction stage, based on the grayscale distribution characteristics of the biological tissue, a threshold segmentation method was used to construct an initial two-dimensional mask. When there was severe grayscale overlap, a region growing algorithm was used to automatically extract connected regions, and for areas with blurred or discontinuous boundaries, layer-by-layer polygon drawing and hole filling were performed manually. Finally, based on the processed continuous two-dimensional mask, the three-dimensional calculation module of Mimics 21.0 software was used to execute a reconstruction algorithm to construct the three-dimensional geometric model of the biological tissue.

[0029] In this embodiment, the biological tissue appearance morphology model is used to determine the appearance outline of the biological tissue and provide an appearance outline reference for subsequent tissue segmentation processing; each tissue block after tissue segmentation can be paraffin embedded to form at least one paraffin-embedded tissue, so as to obtain the corresponding CT scan image sequence and continuous slice image respectively.

[0030] In one embodiment, acquiring the CT scan image sequence of paraffin-embedded tissue before slicing and determining the spatial reference position corresponding to the CT scan images includes: performing a CT scan on the paraffin-embedded tissue paraffin block before it is continuously sliced, obtaining a CT scan image sequence composed of multi-slice CT scan images; determining the spatial reference position corresponding to each CT scan image based on the interslice spacing of each CT scan image in the CT scan image sequence and its slice position in the tissue slice normal direction. Considering that a complete and continuous volumetric data structure can be obtained from the paraffin-embedded tissue block using CT scans before slicing, this embodiment uses the spatial reference position corresponding to the CT scan images to perform spatial positioning calibration on the subsequently registered continuous slice images. When the same whole tissue sample is processed in blocks, the CT scan image sequence corresponding to each tissue block is also used to provide the spatial relative relationship required for inter-block stitching, so that the reconstructed blocks obtained by stacking the images of each tissue block can be aligned and stitched according to the CT spatial reference.

[0031] In one embodiment, obtaining the continuous slice images includes tissue preprocessing.

[0032] Pre-treatment: First, the collected fresh biological tissue samples undergo standardized histological processing, including tissue segmentation and paraffin embedding, forming paraffin-embedded tissue. Tissue segmentation is performed when the whole tissue sample is large, making it difficult to balance section quality and operational feasibility with direct whole-section slicing. Each tissue block is paraffin-embedded separately to form multiple paraffin-embedded tissue blocks. The paraffin-embedded tissue is then serially sectioned to obtain serial sections with section numbers. Subsequently, the serial sections are stained with hematoxylin and eosin (HE), and the stained slides are imaged using a high-throughput digital slide workstation to acquire gigapixel-level detail SVS format raw data. Finally, according to the 3D reconstruction requirements, the SVS format raw data is batch-converted to a common image format (BMP or PNG) to obtain serial section images. The section numbers correspond to the preparation order of the serial sections, and the preparation thickness of the serial sections is used for spatial positioning calibration of the registered serial section images.

[0033] In one embodiment, the generation of the segmentation mask includes image block cropping, image information annotation, and segmentation.

[0034] Image Block Cropping: To address the issue of insufficient video memory / memory computing resources in processing raw continuous slice images (BMP or PNG) at the tens of thousands of pixels level, this study employs a sliding window strategy to perform standardized block cropping processing on the raw continuous slice images. A script program was written using Python's PIL library to divide the entire raw continuous slice image into fixed-size sub-blocks (cell sub-images: 1024px × 1024px; blood vessel sub-images: 2048px × 2048px), and a certain proportion of overlap was set between adjacent sub-blocks (the overlap area for cell sub-images is 50px, accounting for approximately 1 / 20; the overlap area for blood vessel sub-images is 500px, accounting for approximately 1 / 4).

[0035] Image annotation and segmentation: LabelMe was used to annotate sub-tiles, and the outlines of cells and blood vessels were manually delineated using the Polygon tool. Then, semantic labels were assigned to each annotated object, and the annotation results were saved as JSON files, ensuring that each sub-tile corresponded to an independent JSON annotation file. Next, the JSON files generated by LabelMe were converted to the COCO standard dataset format, and 20 manually annotated sub-tiles were completed as the standard dataset. Finally, YOLOv11 was selected as the object detection and instance segmentation model. After configuring the model parameters, hyperparameters, and data paths, training was started, and the converged model was used to complete the automated segmentation and annotation of the remaining sub-tiles. YOLOv11 instance segmentation first detects the target bounding boxes and simultaneously outputs pixel-level masks for each instance, distinguishing different individuals of the same category (such as multiple cells or multiple blood vessels), achieving "detection + segmentation" in one step. In one embodiment, the number of annotated images corresponding to cells and blood vessels were 200 each, and the standard images were randomly selected after being cropped from continuous slices. After completing manual annotation and dataset construction, the trained YOLOv11 instance segmentation model is used to automatically segment and annotate the remaining sub-plots, generating independent binary segmentation masks for cells and blood vessels. Once all sub-plots are segmented, the segmented sub-plot masks are stitched together according to the splitting rules and parameters used during block cropping, restoring the original full-view image scale to obtain the segmentation mask.

[0036] In this embodiment, the continuous slice image is segmented to generate a binary segmentation mask corresponding to the target structure. When the continuous slice image is processed into blocks, the binary segmentation masks corresponding to each block image are stitched back to the full-view scale according to the spatial position of the block image in the continuous slice image, resulting in a full-view binary segmentation mask. When the continuous slice image is not processed into blocks, the binary segmentation mask corresponding to the target structure is used as the full-view binary segmentation mask. Based on the registration results and spatial position calibration results of the continuous slice image, a correspondence is established between the full-view binary segmentation mask and the calibrated continuous slice image, resulting in a segmentation mask corresponding to the calibrated continuous slice image.

[0037] The aforementioned image block cropping, image information annotation, and segmentation processes are used to address the challenge of balancing high-resolution tissue slice data processing with spatial consistency. To address the limitations of computational resources and the difficulty in maintaining spatial continuity during the processing of ultra-high-resolution slice images, this embodiment employs a block processing and consistent stitching strategy to achieve efficient processing while preserving complete spatial information.

[0038] In one embodiment, before performing cross-modal image registration between the CT scan image and the continuous slice image, the CT scan image and the continuous slice image are first converted into images with a unified structure. After obtaining their respective binary structure images, matching features are extracted based on the binary structure images to construct global registration relationship information between the CT scan image and the continuous slice image. Rigid and non-rigid transformations are then jointly optimized on the continuous slice image to obtain the registered continuous slice image. The overall process includes the following three main steps: ① Unified structure image conversion: In the same tissue sample, the CT scan slice spacing is 10 μm, while the slice thickness is 3 μm. Given the large CT slice spacing, this study employs a "nearest neighbor matching" strategy: for each CT scan image, based on the slice position of the CT scan image, the slice number of the consecutive slice images, and the preparation thickness of the consecutive slices, the nearest consecutive slice image in the tissue slice normal direction is determined as the corresponding image. This strategy is based on the assumption that tissue slices have morphological stability within a local range, meaning that the structural changes between adjacent slices are relatively small, thus ensuring the rationality and stability of single-layer matching. Since consecutive slice images and CT scan images differ significantly in resolution, scale range, and background features, unified preprocessing operations are required to improve the stability and computational efficiency of subsequent registration. First, the width of the corresponding images is uniformly scaled to 1024 px, and the height is scaled proportionally. The background is usually pure white or light gray; threshold segmentation can be used to extract the tissue region. The segmentation results are then binarized, with the foreground tissue region assigned a value of 1 and the background region assigned a value of 0, obtaining the binary segmentation results of the consecutive slice images. Secondly, redundant backgrounds in the CT scan images were cropped, retaining only the tissue regions. The images were scaled to a uniform width of 1024px, with the height scaled proportionally. Since CT scan images typically contain complex background noise and overlapping grayscale distributions between the foreground and background, a VM-UNet network structure was used for CT image segmentation. One hundred paraffin block CT scan images of tissue were selected, and precise manual annotation was performed using the LabelMe tool. The input images were downsampled to 572px × 572px to match the original U-Net design resolution. After network output, interpolation was used to restore the resolution to 1024px × 1024px, ultimately obtaining the binary segmentation results of the CT scan images. The resulting binary segmentation results of the continuous slice images and the CT scan images were used as their respective binary structure images, forming a unified structure image for subsequent cross-modal image registration.

[0039] ②Image matching: The VALIS tool was used to register the binary segmentation results of CT scan images with the binary segmentation results of continuous slice images. VALIS is an automated, high-precision registration framework for whole-slice image sequence registration, widely used in digital pathology and spatial omics research. Designed for ultra-high resolution tissue slice data, it achieves stable alignment of large-scale images while ensuring computational efficiency through a multi-resolution pyramid strategy and block processing mechanism. The registration process consists of four steps: key point detection and feature description, establishing matching relationships based on feature similarity, removing mismatched points, and estimating the translation-rotation-scaling transformation matrix. The VALIS framework employs a complementary strategy of Daisy (based on manually designed geometric descriptors) and Disk (an end-to-end feature extraction method based on deep learning) dual feature detectors to enhance registration robustness and improve adaptability to complex tissue morphologies. Simultaneously, a global relationship network between slices is constructed through feature point matching and graph structure modeling, optimized jointly by rigid and non-rigid transformations. Based on the estimated translation-rotation-scaling transformation matrix and the non-rigid transformation results, the continuous slice images are transformed to obtain the registered continuous slice images. The non-rigid transformation employs a B-spline or thin-plate spline deformation model to correct tissue deformation and slice distortion. In other words, this embodiment uses the binary structural image of the CT scan image and the binary structural image of its corresponding continuous slice image as input, extracts the matching features between the two, and establishes global registration relationship information based on the matching features; the rigid transformation is used to achieve translation, rotation, and scaling alignment, while the non-rigid transformation is used to correct tissue deformation and slice distortion, ultimately obtaining the registered continuous slice image.

[0040] The aforementioned cross-modal image registration process addresses the error accumulation problem caused by relying solely on information from the slice images themselves for registration. Existing techniques typically involve registering adjacent slices one by one and then cascading them, which easily leads to registration errors propagating between slices and affecting overall reconstruction accuracy. This embodiment uses the binary segmentation results of the CT scan image and the continuous slice images for registration, and combines rigid and non-rigid transformations for joint optimization to reduce error accumulation during the registration process.

[0041] The aforementioned joint optimization process of rigid and non-rigid transformations is used to address the impact of nonlinear deformation of tissue sections and sample preparation artifacts on registration accuracy. Deformations such as stretching, compression, wrinkling, and tearing during the slicing process reduce registration accuracy. This embodiment improves the ability to correct for complex deformations through cross-modal registration combined with rigid and non-rigid optimization methods.

[0042] ③ Image restoration: The registration process is performed in the downsampled image space, and the registration result in the downsampled image space needs to be mapped back to the original high-resolution space. Specific steps include: binarizing the low-resolution registration result to obtain the image width and height. Subsequently, the foreground target point set is extracted from the binarized low-resolution registration result; then, the minimum bounding rectangle of the foreground target point set is calculated to obtain the center coordinates ( , ),size and rotation angle Then, based on the width and height (WH, HH) of the original high-resolution image and the aspect ratio of the longer side of the low-resolution image, the global scaling factor s is calculated:

[0043] Next, create a scaled-up white canvas (width and height (W, H)) based on the global scaling factor:

[0044] Then, based on the center coordinates, rotation angle, and global scaling factor, the original high-resolution continuous slice image (Im) is processed. Perform translation and rotation transformations to obtain a high-resolution continuous slice image (Im) after geometric transformation. ):

[0045]

[0046]

[0047] Finally, the high-resolution continuous slice image after geometric transformation is pasted onto the white canvas to obtain a high-resolution continuous slice image with complete registration and restoration. This strategy avoids resolution loss caused by multiple interpolations and ensures complete preservation of structural details.

[0048] Under the condition that the cross-modal image registration is performed in the downsampled image space, the registration result in the downsampled image space can be used as downsampled registration transformation information; after mapping the downsampled registration transformation information to the original high-resolution space, high-resolution registration transformation information is obtained, and spatial transformation is performed on the continuous slice image without downsampling based on the high-resolution registration transformation information to obtain the high-resolution registered continuous slice image. The high-resolution registered continuous slice image can be used as the registered continuous slice image to enter the subsequent spatial position calibration step.

[0049] like Figure 3As shown, during registration, the stained slide image and CT scan image are first scaled proportionally to the same width and the tissue region is extracted to form a binary contour. Then, the rigid registration shown in the figure is completed based on the CT binary contour, and it is determined whether to match again according to the contour overlap. After the matching meets the requirements, the obtained scaling, rotation and translation parameters are mapped to the original high-resolution stained slide image, and the transformed image is placed in the corresponding canvas to obtain the restored high-resolution registered image.

[0050] In one embodiment, see Figure 6 The continuous slice cross-modal registration and 3D reconstruction process includes "Step 1, multimodal imaging of embedded tissue", "Step 2, slice alignment and segmentation", "Step 3, CT baseline registration", and "Step 4, coordinate system alignment and original image registration and reconstruction". In the first step, multimodal imaging of embedded tissue, the embedded tissue is imaged using a CT scanner, resulting in images 899 x 974 pixels with a total of 800 images, denoted as Ic. The embedded tissue is then sliced ​​and scanned using a microscope for high-resolution imaging, resulting in images 12400 x 12000 pixels with a total of 1000 images, denoted as Is.

[0051] In the second step of slice alignment and segmentation: 1. Extract the nth image from Is, where n starts from 1, and denote the extracted continuous slice image as Is(n); 2. Calculate the data corresponding to Is(n) from the slices in Ic, and denote it as Ic(n); 3. Perform segmentation using residual unet to obtain the corresponding binary images Ic_s(n) and Is_s(n). Here, Ic(n) and Is(n) represent the corresponding CT scan image and continuous slice image at the nth processing stage, respectively, and Ic_s(n) and Is_s(n) represent the binary structure images formed after tissue region segmentation and binarization. Figure 6 The residual unet in the equation represents the segmentation network processing step, which in this embodiment corresponds to the aforementioned unified structure image conversion process.

[0052] In the third step, during the calculation of transformation parameters using the ICP algorithm with the CT image center as the reference coordinate, Is_s(n) is registered to Ic_s(n) using Ic_s(n) as the reference and the image center of Ic_s(n) as the reference coordinate system. The registration method adopts the standard ICP method, and the transformation parameters are obtained using ICP, including translational displacement. Rotation angle (θ) and scaling ratio The ICP processing is used to achieve the aforementioned rigid transformation parameter estimation, and the obtained rigid transformation results can be combined with the aforementioned non-rigid transformation results for registration optimization of continuous slice images.

[0053] In the fourth step of coordinate system alignment and original image registration and reconstruction, 1) the original image is transformed and scaled while maintaining a fixed resolution. and Then, calculate , and then calculate That is, transform the original graph Is(n) into ; , Without changing the original image, three parameters are used to scale, rotate, and translate it. Finally, after cropping, single-image registration is completed, and the image is restored according to the CT coordinate system. An arbitrary resolution is set, thus achieving stepless scaling and registration. This represents the scaling parameter after mapping to a fixed resolution or any target resolution. The translation parameters after mapping are represented by the three parameters: translation displacement, rotation angle, and scaling ratio. 2) Repeat step two until the images on the entire dataset are registered, thus forming a registered 3D dataset; any resolution can be set to achieve infinite scaling and registration. The registered 3D dataset serves as the input for subsequent spatial positioning calibration, inter-layer overlay, and 3D reconstruction based on the spatial reference positions corresponding to the CT scan images.

[0054] In one embodiment, based on the spatial reference position corresponding to the CT scan image, the registered continuous slice image is spatially calibrated, and the calibrated continuous slice image or its segmentation mask is overlaid to complete three-dimensional reconstruction.

[0055] Image overlay: After acquiring high-resolution and registered sequential slice images, it is necessary to determine the spatial position of each registered sequential slice image, and then achieve the structural reconstruction of the tissue block by slice overlay. To solve the problem of inaccurate spatial positioning due to slice discontinuity and sample loss in tissue slice preparation, this study proposes a method for guiding slice spatial positioning using CT scan images. This study uses a sequence of CT scan images obtained from complete and continuous scanning of the tissue block as the spatial positioning reference. This data has no missing samples and fixed spatial positions, and the CT scan slice spacing is uniformly set to 10 μm. First, in the tissue slice normal direction, the registered sequential slice image closest to any CT scan image slice position is used as the matching sequential slice image, and the matching sequential slice image is calibrated to the spatial reference position of the corresponding CT scan image; the registered sequential slice images that are not used as matching sequential slice images are determined as unmatched sequential slice images. Then, based on the slice number of the continuous slice image, and with the actual preparation thickness of the slice at 3μm as the spatial interval, the spatial position of the unmatched continuous slice image is interpolated and filled along the normal direction of the tissue slice according to the preparation order. Based on the spatial reference position and the interpolation and filling results, the spatial position of each registered continuous slice image is determined. Subsequently, according to the spatial positions of each continuous slice image determined after spatial positioning calibration, the calibrated continuous slice images or their segmentation masks are sliced ​​and superimposed to form a continuous slice image sequence or two-dimensional mask sequence arranged according to the spatial positions. For multiple tissue blocks obtained from the same whole tissue block, after spatial positioning calibration is completed based on the CT scan image sequence of each tissue block, the continuous slice image sequence or two-dimensional mask sequence corresponding to each tissue block is spliced ​​between blocks according to the CT spatial reference position, so that the tissue blocks obtained by stacking CT images are spatially continuous and avoid inaccurate superposition caused by the inconsistency between the direction of tissue curvature change and the direction of slice normal. The continuous slice image sequence or two-dimensional mask sequence is converted into three-dimensional voxel data using three-dimensional reconstruction software. In this embodiment, the three-dimensional reconstruction software is Amira and Mimics Medical software. A three-dimensional surface model is generated based on the three-dimensional voxel data using the Marching Cubes isosurface extraction algorithm.

[0056] In the aforementioned spatial location calibration process, the matched continuous slice images calibrated to the corresponding spatial reference position can form a reference anchor slice. Based on the spatial reference position of the reference anchor slice, the preparation order of the continuous slice images, and the slice thickness of the continuous slices, the spatial positions of the remaining registered continuous slice images are determined along the tissue slice normal direction, forming a spatially calibrated continuous slice image sequence. For cases where multiple paraffin-embedded tissues correspond to different tissue blocks, a three-dimensional reconstruction model can be generated based on the three-dimensional volume data or the stitched three-dimensional volume data.

[0057] like Figure 4 As shown, the CT scan image has a known interslice spacing and a defined spatial position, and the continuous slice image has a corresponding slice number and a slice thickness of 3μm. First, the CT scan image with the corresponding morphology is selected and the continuous slice image is anchored to the corresponding CT slice position. Then, based on the spatial distance between adjacent anchored layers, the slice number and the slice thickness, the spatial position of the remaining continuous slice images is determined sequentially along the tissue slice normal direction, thus forming a continuous slice sequence constrained by the CT spatial reference.

[0058] like Figure 5 As shown, after completing cross-modal image registration, spatial location calibration, and image overlay processing of continuous slice images, a three-dimensional reconstruction result corresponding to the spatial morphology of paraffin-embedded tissue can be obtained. This three-dimensional reconstruction result is generated based on the three-dimensional volume data formed by the spatially calibrated continuous slice images or their segmentation masks, and can display the overall appearance contour and interlayer continuity of the target tissue in three-dimensional space. The three-dimensional spatial frame shown in the figure represents the spatial reference range of the reconstructed model, and R, L, A, P, and S are used to represent the orientation reference of the reconstruction result in three-dimensional space. This result figure intuitively demonstrates that, under the constraint of the spatial reference position corresponding to the CT scan image, this invention can enable continuous slice images to form a three-dimensional reconstruction model with spatial continuity along the tissue slice normal direction, thereby reducing the problems of overall morphological shift, interlayer misalignment, or local structural discontinuity caused by relying solely on slice image layer-by-layer registration and overlay.

[0059] The aforementioned spatial location calibration process addresses the problem of inaccurate spatial positioning caused by the lack of a unified spatial reference for continuous slices. Due to issues such as slice discontinuities and missing slices during slice preparation, existing methods struggle to accurately determine the true spatial location of each slice. This embodiment calibrates matched continuous slice images based on the spatial reference position corresponding to the CT scan image, and uses the slice number and actual slice preparation thickness to determine the spatial location of mismatched continuous slice images, thus completing the spatial location calibration of each registered continuous slice image. In the scenario of whole-tissue block reconstruction, the spatial location calibration also provides a unified spatial reference for splicing different tissue blocks, reducing spatial misalignment between blocks.

[0060] The aforementioned image overlay and 3D reconstruction process is also used to address the issues of overall structural offset and morphological distortion during 3D reconstruction. Due to the lack of macroscopic structural constraints, existing reconstruction methods are prone to overall morphological offset or structural distortion. In this embodiment, based on the spatial positions of each continuous slice image determined after spatial calibration, the calibrated continuous slice images or their segmentation masks are overlaid, and a 3D surface model is generated from the 3D voxel data. For arc-shaped tissues such as the meniscus, without spatial calibration, the block reconstruction results may result in inaccurate inter-block overlay because the tissue curvature does not extend monotonically along the slice normal direction. This embodiment utilizes the CT spatial reference position to constrain the overlay order and splicing position of each tissue block, thereby improving the problem of unnatural inter-block connections.

[0061] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Any modifications, equivalent substitutions, improvements, or combinations made to the above embodiments within the spirit and principles of the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A method for three-dimensional reconstruction of continuous slices based on CT spatial reference constraints, characterized in that, include: Obtain CT scan image sequences of paraffin-embedded tissue before sectioning, and construct the spatial reference positions corresponding to each CT scan image in the CT scan image sequence; Acquire sequential slice images corresponding to the paraffin-embedded tissue; perform cross-modal image registration between the CT scan image and the sequential slice image to obtain the registered sequential slice image; and perform spatial position calibration on the registered sequential slice image based on the spatial reference position corresponding to the CT scan image. The calibrated continuous slice images or their segmentation masks are overlaid to complete the 3D reconstruction.

2. The method for continuous slice three-dimensional reconstruction based on CT spatial reference constraints according to claim 1, characterized in that, The steps for obtaining the continuous slice images include: The collected biological tissue samples are divided into tissue blocks and embedded in paraffin to form at least one paraffin-embedded tissue; the paraffin-embedded tissue is then serially sectioned, stained, and imaged on a glass slide to obtain serially sectioned images with section numbers; the section numbers are used to characterize the preparation order of the serially sectioned images.

3. The method for continuous slice three-dimensional reconstruction based on CT spatial reference constraints according to claim 1, characterized in that, Before performing cross-modal image registration on the CT scan image and the continuous slice image, the method further includes converting the CT scan image and the continuous slice image into a unified structure image. The conversion steps include: determining the continuous slice image corresponding to the CT scan image in the tissue slice normal direction according to the spatial reference position corresponding to the CT scan image and the preparation order of the continuous slice image; extracting tissue regions from the CT scan image and the corresponding continuous slice image respectively and performing binarization processing to obtain their respective binary structure images.

4. The method for continuous slice three-dimensional reconstruction based on CT spatial reference constraints according to claim 3, characterized in that, The step of obtaining the registered continuous slice image includes: extracting matching features between the CT scan image and the continuous slice image based on the binary structure image of the CT scan image and the binary structure image of the corresponding continuous slice image; constructing global registration relationship information between the CT scan image and the continuous slice image based on the matching features; and performing joint optimization of rigid transformation and non-rigid transformation on the continuous slice image based on the global registration relationship information to obtain the registered continuous slice image; the rigid transformation is used to realize translation, rotation and scaling alignment, and the non-rigid transformation is used to correct tissue deformation and slice distortion.

5. The method for continuous slice three-dimensional reconstruction based on CT spatial reference constraints according to claim 4, characterized in that, Under the condition that the cross-modal image registration is performed in the downsampled image space, obtaining the registered continuous slice image further includes: acquiring downsampled registration transformation information obtained by joint optimization of the rigid transformation and the non-rigid transformation; mapping the downsampled registration transformation information to the original high-resolution space to obtain high-resolution registration transformation information; and performing spatial transformation on the continuous slice image that has not undergone downsampling processing based on the high-resolution registration transformation information to obtain the high-resolution registered continuous slice image.

6. The method for continuous slice three-dimensional reconstruction based on CT spatial reference constraints according to claim 1, characterized in that, The spatial location calibration step includes: calibrating the registered continuous slice images corresponding to the CT scan images to the corresponding spatial reference positions to form reference anchor slices; based on the spatial reference positions of the reference anchor slices, the preparation order of the continuous slice images and the slice thickness of the continuous slices, determining the spatial positions of the remaining registered continuous slice images along the tissue slice normal direction to obtain a spatially calibrated continuous slice image sequence.

7. The method for continuous slice three-dimensional reconstruction based on CT spatial reference constraints according to claim 1, characterized in that, The steps for generating the segmentation mask include: performing target structure segmentation on the continuous slice image to generate a binary segmentation mask corresponding to the target structure; under the condition that the continuous slice image is processed into blocks, performing target structure segmentation on each block image to obtain a binary segmentation mask corresponding to each block image, and stitching the binary segmentation masks corresponding to each block image back to the full-view scale according to the spatial position of the block images in the continuous slice image to obtain a full-view binary segmentation mask; under the condition that the continuous slice image is not processed into blocks, using the binary segmentation mask corresponding to the target structure as the full-view binary segmentation mask; and establishing the correspondence between the full-view binary segmentation mask and the calibrated continuous slice image based on the registration result and spatial position calibration result of the continuous slice image to obtain a segmentation mask corresponding to the calibrated continuous slice image.

8. The method for continuous slice three-dimensional reconstruction based on CT spatial reference constraints according to claim 1 or 7, characterized in that, The steps of the three-dimensional reconstruction include: according to the spatial position of the calibrated continuous slice images, performing interlayer overlay on the calibrated continuous slice images or their corresponding segmentation masks to generate three-dimensional volume data; under the condition that multiple paraffin-embedded tissues correspond to different tissue blocks, based on the spatial reference position of the CT scan images corresponding to each tissue block, performing inter-block stitching on the three-dimensional volume data corresponding to each tissue block to obtain stitched three-dimensional volume data; and generating a three-dimensional reconstruction model based on the three-dimensional volume data.

9. The method for continuous slice three-dimensional reconstruction based on CT spatial reference constraints according to claim 1, characterized in that, Before acquiring the CT scan image sequence before obtaining the paraffin-embedded tissue section, the method further includes: acquiring the MRI image sequence of the biological tissue used to form the paraffin-embedded tissue; segmenting the tissue region of the MRI image sequence and performing three-dimensional reconstruction based on the tissue region segmentation results to obtain a biological tissue appearance morphology model; determining the appearance contour of the biological tissue according to the biological tissue appearance morphology model, and providing an appearance contour reference for the tissue block processing of the biological tissue.

Images