Visualization method of biological tissue three-dimensional full-information atlas

Abstract

The invention relates to the technical field of optics, and discloses a visualization method of a biological tissue three-dimensional full-information atlas, which comprises the following steps of: acquiring image data, electron microscope data, mechanical data, protein distribution, omics information and histological staining images of biological tissues by using various biological visualization technologies, and correspondingly processing the acquired data to obtain a three-dimensional full-information atlas of the biological tissues; finally, organization information fusion and digital display are achieved. According to the method, a cross-scale feature correlation algorithm framework is innovatively established, and multi-source heterogeneous data fusion is performed on medical image topological data (CT / MRI), ultrastructure scanning data (SEM / TEM), molecular positioning information (immunofluorescence, immunohistochemistry and enzyme immunoassay), omics maps (space transcriptome / proteome) and high-resolution tissue staining data. The data island dilemma caused by traditional single-mode analysis is broken through, and full-scale visualization from nanoscale molecular distribution to centimeter-level organ tissue structure-function relationship is realized for the first time.

Description

Technical Field

[0001] The present invention relates to the field of optical technology, and in particular to a method for visualizing a three-dimensional full-information atlas of biological tissues. Background Art

[0002] In recent years, with breakthroughs in biological microscopy and exponential increases in computer processing power, a multimodal, cross-scale technology ecosystem has emerged in the field of biological tissue visualization. Existing technologies primarily encompass three broad categories: traditional visualization techniques based on morphological characterization, including histochemical staining (e.g., H&E staining, Masson's trichrome), electron microscopy (SEM / TEM), and confocal microscopy; molecular biology-based information analysis techniques, such as immunofluorescence labeling (IF) and spatial transcriptomics; and medical imaging techniques, including macroscopic imaging methods such as MRI, CT, and ultrasound.

[0003] The complexity of biological tissues is reflected not only in the interscalar spatial architecture between nanoscale molecular assembly and millimeter-scale organ structure, but also in the coordinated regulation of multidimensional information flows during dynamic physiological processes. Traditional two-dimensional characterization techniques, while offering analytical advantages at specific scales, are constrained by inherent limitations: conventional tissue section staining inevitably causes mechanical disruption of spatial topology; electron microscopy, while capable of nanoscale resolution, struggles to achieve three-dimensional reconstruction across a wide field of view; and molecular labeling, while capable of precisely localizing biomarkers, faces technical bottlenecks such as phototoxicity and crosstalk between multiple labels. The rise of three-dimensional digital reconstruction technology offers a new path to overcome this dimensional barrier, with advanced algorithmic models providing strong technical support for the integration and visualization of these biological tissue information. Current mainstream three-dimensional reconstruction methods rely on spatial registration and voxel-based reconstruction based on serial section staining images. This reconstruction paradigm, based on traditional staining templates, suffers from systemic flaws such as a single information dimension and difficulty fusing multimodal data. This makes it difficult to achieve multi-scale correlation analysis from molecular distribution to tissue structure, a key technical bottleneck hindering the decoding of tissue structure and function.

[0004] The current biological tissue visualization technology system still has multi-dimensional technical limitations. Its fundamental contradiction lies in the separation between the discrete technical path and the need for holistic cognition of biological systems. Traditional methods generally adopt a single-modality analysis strategy, which has caused morphological characterization, molecular localization and functional analysis to be in a data silo state for a long time: although histochemical staining can present local microstructural characteristics, it cannot associate molecular expression spectrum information; although spatial transcriptomics technology can draw gene expression maps, it is limited by resolution and cannot accurately locate subcellular structures; although medical imaging has the advantage of non-destructive testing, its microscopic resolution capability is significantly reduced below the micrometer level. This fragmentation of multi-source data means that researchers can only obtain fragmented information about the complex regulatory networks of biological tissues, making it difficult to construct cross-scale spatial mapping models and to achieve coordinated visualization of structural-functional characteristics in the complete tissue dimension.

[0005] 3D reconstruction techniques based on traditional staining face technical limitations. The cumulative deformation caused by sample syneresis during serial section preparation and the wax embedding process constitutes the most significant spatial topological distortion in the histological staining process, resulting in millimeter- to micron-scale spatial misalignment between staining batches. Existing registration methods lack knowledge of the topographic contours of the paraffin-embedded intermediates, making it difficult to effectively correct for the irregular deformation caused by anisotropic tissue shrinkage. Consequently, the stained images lack a contour reference for registration and alignment.

[0006] Existing visualization models generally suffer from a cognitive flaw of prioritizing structure over function. While they can reproduce the static geometric configuration of tissues, they lack information about their mechanical functions. This decoupled structure-function modeling approach fails to characterize the mechanisms by which the viscoelastic properties of the extracellular matrix regulate tissue differentiation, nor can it reveal the cross-scale mechanical coupling between microscopic stress distribution and macroscopic tissue deformation. This significantly hinders in-depth analysis of the synergistic mechanisms of structure-function in biological tissues. Summary of the Invention

[0007] Based on the above series of technical problems, the present invention provides a visualization method for the three-dimensional full-information map of biological tissues, aiming to promote the transformation of biomedical research and clinical diagnosis from traditional two-dimensional observation to a comprehensive three-dimensional stereoscopic paradigm, and lay a key technical foundation for the development of precision medicine and personalized medicine.

[0008] To achieve the above object, the present invention provides a method for visualizing a three-dimensional full-information atlas of biological tissues, which comprises the following steps:

[0009] (1) Information collection on the same biological tissue

[0010] Acquiring imaging data of biological tissue using one of magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound imaging (US);

[0011] Use SEM electron microscope to obtain electron microscopic data of biological tissues;

[0012] Perform mechanical simulations on biological tissue using finite element software (e.g., ANSYS, COMSOL, etc.) to generate mechanical data for different regions of the tissue, including at least one of stress, strain, and displacement;

[0013] Using tissue clearing technology to obtain distribution data of various cell types in biological tissues;

[0014] Using enzyme immunoassay and immunostaining techniques, the content of different proteins in different areas of biological tissues is obtained to obtain protein content data;

[0015] Obtain omics information of biological tissues including genome, transcriptome, proteome and metabolome;

[0016] The biological tissue is divided into blocks, and the blocks are dehydrated, transparentized, and wax-embedded. The wax-embedded blocks are then CT scanned to obtain a deformed tissue block model; the wax-embedded blocks are scanned using an industrial camera to obtain a paraffin section image with a clear box boundary; the paraffin sections are histologically stained (HE staining), and the histologically stained (HE staining) sections are scanned using a digital section scanner to obtain a tissue staining image;

[0017] (2) Image data model reconstruction

[0018] The imaging data was preprocessed, including denoising, registration, and segmentation. Features were then extracted from the preprocessed imaging data, and semantic segmentation of the target tissue was performed using the U-Net deep learning model. The segmentation results were optimized using a region growing algorithm and manual contour editing within the 3D Slicer platform. Finally, topological features were extracted from the segmented mask data, and an initial triangular mesh was generated using the marching cubes algorithm. This mesh was then optimized using the MeshLab tool to construct a reconstructed 3D image geometry model of the native tissue.

[0019] (3) SEM data reconstruction

[0020] Based on the tissue fiber images obtained from the electron microscope data images, a single-layer model simulating the orientation of the collagen fibers in biological tissue is constructed using 3D modeling software. Then, based on the structural characteristics of different tissue layers, each layer model is precisely spliced ​​together to ultimately obtain a 3D image of the collagen fiber arrangement and distribution in the complete tissue.

[0021] (4) Mechanical cloud map construction

[0022] Process the mechanical data, convert it into a plotting format like CSV or TXT, remove noise and outliers, then import the processed data into a plotting tool, select the mechanical parameters to be plotted, set a color gradient based on the data range, plot the color distribution on the geometric model (e.g., from blue to red for low to high stress), form a cloud map, and add annotations;

[0023] (5) Construction of cell type distribution map

[0024] Acquire three-dimensional imaging data of biological tissues based on tissue clearing technology (such as CLARITY or CUBIC processing). Use CellProfiler or Imaris software to perform automated cell segmentation and classification identification on multi-channel images of DAPI nuclear staining and immunofluorescence labeling (such as CD45+ immune cells and α-SMA+ fibroblasts). Color-code the segmented cell clusters according to preset cell type labels (such as neurons labeled with green RGB (0, 255, 0) and endothelial cells labeled with red RGB (255, 0, 0)). Different cell types are labeled with different colors to obtain a distribution map of different cell types in the tissue.

[0025] (6) Construction of protein distribution map

[0026] Set a color gradient based on the protein content data range, draw a color distribution cloud on the geometric model, and splice it according to the marked areas to obtain the content and distribution characteristics of different proteins in the entire tissue;

[0027] (7) Construction of omics information graph

[0028] The genome (SNP chip data), transcriptome (RNA-seq expression), proteome (mass spectrometry quantitative data), and metabolome (LC-MS metabolite concentration) were quantile normalized to eliminate batch effects, and the dimensions were unified by Z-score standardization to obtain standardized multi-omics data. The standardized multi-omics data were input into the multi-omics factor analysis (MOFA) model to extract shared potential factors of genomic variation, gene expression, protein abundance, and metabolite concentration. Cross-omics association features were screened based on factor loading weights and mapped to the Cytoscape software platform. By integrating the gene-protein interaction relationship of the STRING database, the protein-metabolite association of the KEGG metabolic pathway, and the cross-omics Spearman correlation analysis results, a multi-layer molecular interaction network with MOFA factors as the hub was constructed with a threshold of |ρ|>0.6 and p<0.05. Multi-omics features including gene mutations, differentially expressed genes, differentially expressed proteins, and metabolites were further extracted from the network. The random forest algorithm (Python) was used to analyze the correlation between the gene-protein interaction relationship and the protein-metabolite association of the KEGG metabolic pathway. scikit-learn library) to rank feature importance, screen the top 10% high-weight features as candidate biomarkers, and input these biomarkers into the Metascape platform. Pathway enrichment analysis (hypergeometric test, FDR < 0.05) was performed in conjunction with the KEGG and Reactome databases. Core regulatory pathways (such as the PI3K-AKT signaling pathway) were identified based on the enrichment score. Finally, based on the gene expression matrix (CSV format) of the key molecules in the above pathways, it was converted into spatial coordinate data, with its X / Y / Z axes corresponding to samples, genes, and expression levels. The PDB structure files of related proteins were simultaneously obtained from the UniProt database. After being parsed by Biopython, the three-dimensional structures were rendered in Cartoon+Surface mode using PyMOL software, and the gene expression levels were mapped to the corresponding protein spatial positions using gradient shading to generate interactive three-dimensional visualization images linked to the molecular interaction network and pathway analysis results.

[0029] (8) Continuous Slice 3D Modeling

[0030] 8a) Tissue Information Extraction

[0031] With the help of Python's PIL library, a script program was written to split the tissue staining image into several small images according to a fixed size of 1024*1024. LabelMe was used to annotate the small images. The slice staining image file was opened; the polygon tool Polygon was used to manually outline the contours of cells and blood vessels; a label (such as "cell" or "vessel") was assigned to each annotated object, and the annotation results were saved as a JSON file. Each JSON file corresponds to the annotation information of an image; the JSON file generated by LabelMe was then converted to COCO format; 20 cases were manually annotated on these small images, and the model was trained and run using YOLO. The model segmentation was used to complete the annotation of all images, and YOLOv11 was selected as the target detection and segmentation model. The configuration file of YOLOv11 was prepared, the model parameters, training hyperparameters, and data path were set, and the training process was started using the command line: python train.py --data data.yaml --cfg yolov11.yaml --weights yolov11.pt --epochs100 --batch-size 16,

[0032] --data: specifies the data configuration file path;

[0033] --cfg: specifies the model configuration file path;

[0034] --weights: specifies the path of the pre-training weight file;

[0035] --epochs: set the number of training rounds;

[0036] --batch-size: set the batch size;

[0037] Use TensorBoard to monitor the loss function and accuracy during training.

[0038] Evaluate model performance on the validation set, calculate mAP (mean Average Precision) and IoU (Intersection over Union) metrics, and adjust model parameters or data augmentation strategies based on the evaluation results;

[0039] Use the trained model to predict the test image: Python detect.py --source test_images / --weights runs / train / exp / weights / best.pt --conf 0.25, where

[0040] --source: specifies the test image path;

[0041] --weights: specifies the path of the trained model weight file;

[0042] --conf: set the confidence threshold;

[0043] The prediction results are saved in the runs / detect / exp directory, including images with detection boxes and segmentation contours. The prediction results are analyzed to check the segmentation accuracy of cells and blood vessels, and post-processing (such as morphological operations) is performed to optimize the segmentation effect.

[0044] Apply morphological operations (such as opening and closing) to remove noise or fill holes; use Gaussian filtering or bilateral filtering to smooth segmentation contours, and adjust model hyperparameters (such as learning rate and batch size) based on the prediction results; fine-tune the model on a specific dataset to improve performance on a specific task;

[0045] After the thumbnails are annotated, the image is deeply segmented using the Yolov11 instance segmentation algorithm. The Yolov11 algorithm can quickly and accurately identify annotated objects, such as cells and blood vessels, and separate them from the background image to generate independent segmentation masks. After the segmentation process is completed, the Python PIL library (Pillow library) is called again. Based on the previous segmentation rules and parameters, the multiple segmented 1024*1024 images are reassembled back to the original size to ensure that the segmentation results can be fully restored to the original image scale, providing complete data support for subsequent 3D reconstruction.

[0046] 8b) Image Registration

[0047] First, the tissue block model acquired by CT scanning was imported into Mimics software as a spatial reference framework for slice alignment. Second, based on the paraffin section images acquired by an industrial camera, the internal border of the slice was used as the reference line to perform preliminary registration of the serial sections. PS software was used for fine registration. By selecting blood vessels as registration markers, image processing algorithms were used to precisely adjust the position, rotation, and scaling parameters of the images to ensure accurate alignment of different images in the spatial coordinate system. For the registered images, vascular and cell information was extracted. The segmented images were imported into the Amira software environment, and the 2D image sequences were converted into DICOM data format. Subsequently, the generated DICOM data was imported into the Mimics software platform to obtain a 3D digital model of the serial sections with cell and vascular distribution.

[0048] (9) Organizational information integration and digital display

[0049] 9a) In Mimics software, fine-tune the 3D digital model created by serial slices by moving, rotating, and resizing it to accurately match it with the existing image geometry model; finally, save the matched 3D model in STL format;

[0050] 9b) converting the distribution map of different cell types in the tissue obtained by tissue clearing and the stereoscopic image obtained by tissue clearing into STL format and directly importing them into the three-dimensional digital model function layer of the serial section modeling;

[0051] 9c) importing the 3D model image of collagen fiber arrangement and distribution of the intact tissue reconstructed from the SEM image into the 3D digital model of the serial section modeling in STL format to generate a new 3D model functional layer;

[0052] 9d) When processing and drawing data, the 3D mechanics cloud map is directly drawn, and the 3D digital model modeled by continuous slicing is imported in STL format to generate a new 3D model functional layer;

[0053] 9e) The protein quantitative and distribution data obtained from the protein distribution map are constructed using the same method as the mechanical cloud map. After the protein distribution cloud map is obtained, it is imported into the 3D digital model of the serial section modeling in STL format to generate a new 3D model functional layer;

[0054] 9f) The 3D image generated by the omics data is finally imported into the 3D digital model of the serial slice modeling in STL format to generate a new 3D model functional layer;

[0055] 9g) Associating the initial values ​​collected in step (1) (such as protein molecules, growth factors and different types of cell content, vascular distribution, mechanical parameters, etc.) with the fused three-dimensional digital model points, surfaces or regions; then using interactive visualization tools (such as Plotly, ParaView, Mayavi) or programming to implement mouse event monitoring, when the mouse points to or clicks a specific location, the event is triggered and a dialog box pops up to display the corresponding quantitative information.

[0056] The method for visualizing a three-dimensional full-information atlas of biological tissues of the present invention, using the above technical solution, can achieve the following beneficial effects:

[0057] 1) Innovatively establish a cross-scale feature association algorithm framework to fuse multi-source heterogeneous data, including medical imaging topology data (CT / MRI), ultrastructural scanning data (SEM), molecular localization information (immunofluorescence, immunohistochemistry, and enzyme immunoassay), omics maps (spatial transcriptome / proteome), and high-resolution tissue staining data. This breaks through the data silos caused by traditional single-modality analysis and achieves full-scale visualization of structure-activity relationships from nanoscale molecular distribution to centimeter-scale organ tissues for the first time.

[0058] 2) Integrate mechanical microenvironment parameters (Young's modulus, stress distribution) and dynamic mechanical environment simulation data into the 3D reconstructed digital model. By establishing a micro-macro cross-scale mechanical transmission model, a bidirectional coupling simulation of the viscoelastic properties of the extracellular matrix and the macroscopic deformation of the tissue is achieved, breaking through the structural limitations of traditional 3D models and reproducing the synergistic mechanism of biological tissue structure and function in a digital model for the first time.

[0059] 3) A three-dimensional reference framework for paraffin embedding intermediates was creatively introduced. By performing micron-level CT scanning on the paraffin blocks during the embedding and solidification stage, combined with high-precision optical imaging after slicing, an intermediate registration model was formed to improve the cumulative deformation caused by sample dehydration shrinkage and wax embedding process during the preparation of serial sections. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0061] Figure 1 is a flow chart of the present invention;

[0062] Figure 2 CT scan result of natural biological tissue in an embodiment of the present invention (taking the meniscus as an example);

[0063] Figure 3 This is a scan of a paraffin section of a natural biological tissue in an embodiment of the present invention (taking the meniscus as an example);

[0064] Figure 4 The blood vessels (top) and cells (bottom) segmented using a computer algorithm in an embodiment of the present invention;

[0065] Figure 5 Schematic diagram of the three-dimensional reconstruction result of tissue information in an embodiment of the present invention (taking the reconstruction of meniscus tissue cells and blood vessels as an example).

[0066] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0067] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. Terms such as "upper," "lower," "left," "right," "center," and "one" used in the preferred embodiments are for ease of description and are not intended to limit the scope of the present invention. Changes or adjustments to these relative terms, without substantially altering the technical content, are also considered within the scope of the present invention.

[0068] like Figure 1As shown, the present invention provides a method for visualizing a three-dimensional full-information atlas of biological tissue. Taking meniscus biological tissue as an example, the specific method is as follows:

[0069] (1) Information collection of biological tissue (meniscus)

[0070] Acquiring imaging data of biological tissue using one of magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound imaging (US);

[0071] Use SEM electron microscope to obtain electron microscopic data of biological tissues;

[0072] Perform mechanical simulations on biological tissue using finite element software (e.g., ANSYS, COMSOL, etc.) to generate mechanical data for different regions of the tissue, including at least one of stress, strain, and displacement;

[0073] Using tissue clearing technology to obtain distribution data of various cell types in biological tissues;

[0074] Using enzyme immunoassay and immunostaining techniques, the content of different proteins in different areas of biological tissues is obtained to obtain protein content data;

[0075] Obtain omics information of biological tissues including genome, transcriptome, proteome and metabolome;

[0076] The biological tissue is divided into blocks, and the blocks are dehydrated, transparentized and wax-embedded. The wax-embedded blocks are then CT scanned to obtain a deformed tissue block model ( Figure 2 ); Use an industrial camera to scan the wax-embedded tissue blocks to obtain a paraffin section with a clear box boundary ( Figure 3 ); performing histological staining (HE staining) on ​​the paraffin sections, and scanning the histologically stained (HE staining) sections using a digital slice scanner to obtain a tissue staining image;

[0077] (2) Image data model reconstruction

[0078] The imaging data was preprocessed, including denoising, registration, and segmentation. Features were then extracted from the preprocessed imaging data, and semantic segmentation of the target tissue was performed using the U-Net deep learning model. The segmentation results were optimized using a region growing algorithm and manual contour editing within the 3D Slicer platform. Finally, topological features were extracted from the segmented mask data, and an initial triangular mesh was generated using the marching cubes algorithm. This mesh was then optimized using the MeshLab tool to construct a reconstructed 3D image geometry model of the native tissue.

[0079] (3) SEM data reconstruction

[0080] Based on the tissue fiber images obtained from the electron microscope data images, a single-layer model simulating the orientation of the collagen fibers in biological tissue is constructed using 3D modeling software. Then, based on the structural characteristics of different tissue layers, each layer model is precisely spliced ​​together to ultimately obtain a 3D image of the collagen fiber arrangement and distribution in the complete tissue.

[0081] (4) Mechanical cloud map construction

[0082] Process the mechanical data, convert it into a plotting format like CSV or TXT, remove noise and outliers, then import the processed data into a plotting tool, select the mechanical parameters to be plotted, set a color gradient based on the data range, plot the color distribution on the geometric model (e.g., from blue to red for low to high stress), form a cloud map, and add annotations;

[0083] (5) Construction of cell type distribution map

[0084] Acquire three-dimensional imaging data of biological tissues based on tissue clearing technology (such as CLARITY or CUBIC processing). Use CellProfiler or Imaris software to perform automated cell segmentation and classification identification on multi-channel images of DAPI nuclear staining and immunofluorescence labeling (such as CD45+ immune cells and α-SMA+ fibroblasts). Color-code the segmented cell clusters according to preset cell type labels (such as neurons labeled with green RGB (0, 255, 0) and endothelial cells labeled with red RGB (255, 0, 0)). Different cell types are labeled with different colors to obtain a distribution map of different cell types in the tissue.

[0085] (6) Construction of protein distribution map

[0086] Set a color gradient based on the protein content data range, draw a color distribution cloud on the geometric model, and splice it according to the marked areas to obtain the content and distribution characteristics of different proteins in the entire tissue;

[0087] (7) Construction of omics information graph

[0088] The genome (SNP chip data), transcriptome (RNA-seq expression), proteome (mass spectrometry quantitative data), and metabolome (LC-MS metabolite concentration) were quantile normalized to eliminate batch effects, and the dimensions were unified by Z-score standardization to obtain standardized multi-omics data. The standardized multi-omics data were input into the multi-omics factor analysis (MOFA) model to extract shared potential factors of genomic variation, gene expression, protein abundance, and metabolite concentration. Cross-omics association features were screened based on factor loading weights and mapped to the Cytoscape software platform. By integrating the gene-protein interaction relationship of the STRING database, the protein-metabolite association of the KEGG metabolic pathway, and the cross-omics Spearman correlation analysis results, a multi-layer molecular interaction network with MOFA factors as the hub was constructed with a threshold of |ρ|>0.6 and p<0.05. Multi-omics features including gene mutations, differentially expressed genes, differentially expressed proteins, and metabolites were further extracted from the network. The random forest algorithm (Python) was used to analyze the correlation between the gene-protein interaction relationship and the protein-metabolite association of the KEGG metabolic pathway. scikit-learn library) to rank feature importance, screen the top 10% high-weight features as candidate biomarkers, and input these biomarkers into the Metascape platform. Pathway enrichment analysis (hypergeometric test, FDR < 0.05) was performed in conjunction with the KEGG and Reactome databases. Core regulatory pathways (such as the PI3K-AKT signaling pathway) were identified based on the enrichment score. Finally, based on the gene expression matrix (CSV format) of the key molecules in the above pathways, it was converted into spatial coordinate data, with its X / Y / Z axes corresponding to samples, genes, and expression levels. The PDB structure files of related proteins were simultaneously obtained from the UniProt database. After being parsed by Biopython, the three-dimensional structures were rendered in Cartoon+Surface mode using PyMOL software, and the gene expression levels were mapped to the corresponding protein spatial positions using gradient shading to generate interactive three-dimensional visualization images linked to the molecular interaction network and pathway analysis results.

[0089] (8) Continuous Slice 3D Modeling

[0090] 8a) Tissue Information Extraction

[0091] LabelMe software was used to label the data and annotate the cells and blood vessels in the staining images ( Figure 4), considering that the size of the original HE staining image is large, Python's PIL library is used to write a script program to split the tissue staining image into several small images according to a fixed size of 1024*1024. LabelMe is used to annotate the image on the small image, and the slice staining image file is opened; the polygon tool Polygon is used to manually outline the contours of cells and blood vessels; a label (such as "cell" or "vessel") is assigned to each annotated object, and the annotation results are saved as a JSON file, each JSON file corresponding to the annotation information of an image; then the JSON file generated by LabelMe is converted to COCO format; 20 cases are manually annotated on these small images, and the model is trained and run with YOLO. The model segmentation is used to complete the annotation of all images, and YOLOv11 is selected as the target detection and segmentation model. The configuration file of YOLOv11 is prepared, the model parameters, training hyperparameters, and data path are set, and the training process is started using the command line: python train.py --data data.yaml --cfg yolov11.yaml --weights yolov11.pt --epochs 100 --batch-size 16,

[0092] --data: specifies the data configuration file path;

[0093] --cfg: specifies the model configuration file path;

[0094] --weights: specifies the path of the pre-training weight file;

[0095] --epochs: set the number of training rounds;

[0096] --batch-size: set the batch size;

[0097] Use TensorBoard to monitor the loss function and accuracy during training.

[0098] Evaluate model performance on the validation set, calculate mAP (mean Average Precision) and IoU (Intersection over Union) metrics, and adjust model parameters or data augmentation strategies based on the evaluation results;

[0099] Use the trained model to predict the test image: Python detect.py --source test_images / --weights runs / train / exp / weights / best.pt --conf 0.25, where

[0100] --source: specifies the test image path;

[0101] --weights: specifies the path of the trained model weight file;

[0102] --conf: set the confidence threshold;

[0103] The prediction results are saved in the runs / detect / exp directory, including images with detection boxes and segmentation contours. The prediction results are analyzed to check the segmentation accuracy of cells and blood vessels, and post-processing (such as morphological operations) is performed to optimize the segmentation effect.

[0104] Apply morphological operations (such as opening and closing) to remove noise or fill holes; use Gaussian filtering or bilateral filtering to smooth segmentation contours, and adjust model hyperparameters (such as learning rate and batch size) based on the prediction results; fine-tune the model on a specific dataset to improve performance on a specific task;

[0105] After the thumbnails are annotated, the image is deeply segmented using the Yolov11 instance segmentation algorithm. The Yolov11 algorithm can quickly and accurately identify annotated objects, such as cells and blood vessels, and separate them from the background image to generate independent segmentation masks. After the segmentation process is completed, the Python PIL library (Pillow library) is called again. Based on the previous segmentation rules and parameters, the multiple segmented 1024*1024 images are reassembled back to the original size to ensure that the segmentation results can be fully restored to the original image scale, providing complete data support for subsequent 3D reconstruction.

[0106] 8b) Image Registration

[0107] First, the tissue block model acquired by CT scanning was imported into Mimics software as a spatial reference framework for slice alignment. Second, based on the paraffin section images acquired by an industrial camera, the internal border of the slice was used as the reference line to perform preliminary registration of the serial sections. PS software was used for fine registration. By selecting blood vessels as registration markers, image processing algorithms were used to precisely adjust the position, rotation, and scaling parameters of the images to ensure accurate alignment of different images in the spatial coordinate system. For the registered images, vascular and cell information was extracted. The segmented images were imported into the Amira software environment, and the 2D image sequences were converted into DICOM data format. Subsequently, the generated DICOM data was imported into the Mimics software platform to obtain a 3D digital model of the serial sections with cell and vascular distribution.

[0108] (9) Organizational information integration and digital display

[0109] 9a) In Mimics software, the 3D digital model of the continuous slice model is moved, rotated, and resized to match it with the existing image geometry model with high precision; finally, the matched 3D model is saved in stl format ( Figure 5 );

[0110] 9b) converting the distribution map of different cell types in the tissue obtained by tissue clearing and the stereoscopic image obtained by tissue clearing into STL format and directly importing them into the three-dimensional digital model function layer of the serial section modeling;

[0111] 9c) importing the 3D model image of collagen fiber arrangement and distribution of the intact tissue reconstructed from the SEM image into the 3D digital model of the serial section modeling in STL format to generate a new 3D model functional layer;

[0112] 9d) When processing and drawing data, the 3D mechanics cloud map is directly drawn, and the 3D digital model modeled by continuous slicing is imported in STL format to generate a new 3D model functional layer;

[0113] 9e) The protein quantitative and distribution data obtained from the protein distribution map are constructed using the same method as the mechanical cloud map. After the protein distribution cloud map is obtained, it is imported into the 3D digital model of the serial section modeling in STL format to generate a new 3D model functional layer;

[0114] 9f) The 3D image generated by the omics data is finally imported into the 3D digital model of the serial slice modeling in STL format to generate a new 3D model functional layer;

[0115] 9g) Associating the initial values ​​collected in step (1) (such as protein molecules, growth factors and different types of cell content, vascular distribution, mechanical parameters, etc.) with the fused three-dimensional digital model points, surfaces or regions; then using interactive visualization tools (such as Plotly, ParaView, Mayavi) or programming to implement mouse event monitoring, when the mouse points to or clicks a specific position, triggering the event to pop up a dialog box, displaying the corresponding quantitative information, thereby realizing visualization of the three-dimensional full information map of the meniscus.

[0116] Although specific embodiments of the present invention are described above, those skilled in the art should understand that these are merely examples and that various changes or modifications may be made to the embodiments without departing from the principles and essence of the present invention. The scope of protection of the present invention is limited only by the appended claims.

Claims

1. A method for visualizing a three-dimensional full-information atlas of biological tissues, characterized in that: The following steps are involved: (1) Information collection on the same biological tissue Use various biological visualization technologies to collect imaging data, electron microscopy data, mechanical data, protein distribution, omics information, and histological staining images of biological tissues; (2) Image data model reconstruction After preprocessing the image data, features were extracted and semantic segmentation of the target tissue was performed using the U-Net deep learning model. The segmentation results were optimized using a region growing algorithm and manual contour editing in the 3D Slicer platform. Topological structural features were extracted from the segmented data, and an initial triangular mesh was generated using the marching cubes algorithm. This mesh was optimized using the MeshLab tool to construct a reconstructed 3D image geometric model of the natural tissue. (3) SEM data reconstruction Based on the tissue fiber images obtained from the electron microscope data images, a single-layer model simulating the orientation of the collagen fibers in biological tissue is constructed using 3D modeling software. Then, based on the structural characteristics of different tissue layers, each layer model is precisely spliced ​​together to ultimately obtain a 3D image of the collagen fiber arrangement and distribution in the complete tissue. (4) Mechanical cloud map construction Process the mechanical data, convert it into a plotting format like CSV or TXT, remove noise and outliers, then import the processed data into the plotting tool, select the mechanical parameters to be plotted, set the color gradient according to the data range, plot the color distribution on the geometric model, form a cloud map, and add annotations; (5) Construction of cell type distribution map Based on the three-dimensional imaging data of biological tissues obtained by tissue clearing technology, CellProfiler or Imaris software is used to automatically segment and classify cells in DAPI nuclear staining and immunofluorescence labeling multi-channel images. The segmented cell clusters are color-coded according to preset cell type labels, and different cell types are marked with different colors to obtain a distribution map of different cell types in the tissue. (6) Construction of protein distribution map Set the color gradient according to the protein content data range, draw the color distribution cloud map on the geometric model, and splice it according to the marked areas to obtain the content and distribution characteristics of different proteins in the entire tissue; (7) Construction of omics information graph The 3D structure was rendered using PyMOL software in Cartoon+Surface mode, and gene expression levels were mapped to corresponding protein spatial positions using gradient shading, generating interactive 3D visualization images linked to molecular interaction networks and pathway analysis results. (8) Continuous Slice 3D Modeling A three-dimensional digital model with serial section modeling of cell and blood vessel distribution was obtained using histological soft color images; (9) Organizational information integration and digital display The information obtained from steps (2) to (8) is saved in the format of stl, and a model is obtained based on imaging. Each stl file is imported and converted into a unified spatial coordinate system to achieve topological alignment in the three-dimensional digital model of continuous slice modeling, thereby ensuring the spatial consistency of data with different resolutions and realizing layer superposition; then the initial value collected in step (1) is associated with the point, surface or area of ​​the three-dimensional digital model formed after fusion; and then a visualization tool or programming that supports interaction is used to implement mouse event monitoring. When the mouse points to or clicks a specific position, an event is triggered and a dialog box pops up to display the corresponding quantitative information.

2. The method for visualizing a three-dimensional full-information atlas of biological tissue according to claim 1, characterized in that: Methods for collecting imaging data, electron microscopy data, mechanical data, protein distribution, omics information, and histological staining images of biological tissues using various biological visualization technologies, specifically including: Acquiring imaging data of biological tissue using one of magnetic resonance imaging, computerized tomography, and ultrasound imaging; Use SEM electron microscope to obtain electron microscopic data of biological tissues; Performing mechanical simulation on biological tissue using finite element software to generate mechanical data including at least one of stress, strain, and displacement for different regions of the tissue; Using tissue clearing technology to obtain distribution data of various cell types in biological tissues; Using enzyme immunoassay and immunostaining techniques, the content of different proteins in different areas of biological tissues is obtained to obtain protein content data; Obtain omics information of biological tissues including genome, transcriptome, proteome and metabolome; The biological tissue is divided into blocks, and the blocked tissues are dehydrated, transparentized, and wax-embedded. The wax-embedded blocked tissues are then CT scanned to obtain a deformed tissue block model; the wax-embedded blocked tissues are scanned using an industrial camera to obtain a paraffin section image with a clear square boundary; the paraffin sections are histologically stained, and the histologically stained sections are scanned using a digital section scanner to obtain a tissue staining image.

3. The method for visualizing a three-dimensional full-information atlas of biological tissue according to claim 1, characterized in that: In the construction of the omics information map, the genome and proteome are processed in the following ways: The genome, transcriptome, proteome and metabolome were quantile normalized to eliminate batch effects, and the dimensions were unified through Z-score standardization to obtain standardized multi-omics data. The standardized multi-omics data were input into the multi-omics factor analysis model to extract shared potential factors of genomic variation, gene expression, protein abundance and metabolite concentration. The cross-omics association features were screened based on the factor loading weights and mapped to the Cytoscape software platform. By integrating the gene-protein interaction relationship of the STRING database, the protein-metabolite association of the KEGG metabolic pathway and the cross-omics Spearman correlation analysis results, a MOFA factor was constructed with the threshold value |ρ|>0.6, p<0.

05. A multi-layer molecular interaction network of the hub was constructed; multi-omics features including gene mutations, differentially expressed genes, differentially expressed proteins and metabolites were further extracted from the network, and the feature importance was ranked using the random forest algorithm. The top 10% high-weight features were screened as candidate biomarkers, and these biomarkers were input into the Metascape platform. Pathway enrichment analysis was performed in conjunction with the KEGG and Reactome databases, and the core regulatory pathways were locked according to the enrichment scores; finally, based on the gene expression matrix of the key molecules in the above pathways, it was converted into spatial coordinate data, with its X / Y / Z axes corresponding to samples, genes, and expression levels. The PDB structure files of the relevant proteins were simultaneously obtained from the UniProt database and parsed by Biopython.

4. The method for visualizing a three-dimensional full-information atlas of biological tissue according to claim 1, characterized in that: In the process of continuous section 3D modeling, the method of obtaining a three-dimensional digital model of continuous section modeling with cell and blood vessel distribution using a histological soft color map specifically includes: First, the tissue block model obtained by CT scanning was imported into Mimics software as the spatial reference framework for slice alignment. Second, based on the paraffin section images captured by an industrial camera, the internal border of the slice was used as the reference line for preliminary registration of the serial sections. PS software was used for fine registration. By selecting blood vessels as registration markers, the image position, rotation, and scaling parameters were precisely adjusted using image processing algorithms to ensure accurate alignment of different images in the spatial coordinate system. For the registered images, vascular and cell information was extracted. The segmented images were imported into the Amira software environment, and the two-dimensional image sequences were converted into DICOM data format. Subsequently, the generated DICOM data was imported into the Mimics software platform to obtain a three-dimensional digital model with serial slice modeling of cell and blood vessel distribution.

5. The method for visualizing a three-dimensional full-information atlas of biological tissue according to claim 4, characterized in that: The serial section 3D modeling process also includes the tissue information extraction step, as follows: With the help of Python's PIL library, a script program was written to split the tissue staining image into several small images according to a fixed size of 1024*1024. LabelMe was used to annotate the small images. The slice staining image file was opened; the polygon tool Polygon was used to manually outline the contours of cells and blood vessels; labels were assigned to each annotated object and the annotation results were saved as JSON files. Each JSON file corresponds to the annotation information of an image; the JSON file generated by LabelMe was then converted to COCO format; 20 cases were manually annotated on these small images, the model was trained and run using YOLO, and the model segmentation was used to complete the annotation of all images. YOLOv11 was selected as the target detection and segmentation model, the YOLOv11 configuration file was prepared, the model parameters, training hyperparameters, and data path were set, and the training process was started using the command line: python train.py --data data.yaml --cfg yolov11.yaml --weights yolov11.pt --epochs 100 --batch-size 16, where --data: specifies the data configuration file path; --cfg: specifies the model configuration file path; --weights: specifies the path of the pre-training weight file; --epochs: set the number of training rounds; --batch-size: set the batch size; Use TensorBoard to monitor the loss function and accuracy during training. Evaluate model performance on the validation set, calculate mAP and IoU metrics, and adjust model parameters or data augmentation strategies based on the evaluation results; Use the trained model to predict the test image: Python detect.py --source test_images / --weights runs / train / exp / weights / best.pt --conf 0.25, where --source: specifies the test image path; --weights: specifies the path of the trained model weight file; --conf: set the confidence threshold; The prediction results are saved in the `runs / detect / exp` directory, including images with detection boxes and segmentation contours. The prediction results are analyzed to check the segmentation accuracy of cells and blood vessels, and post-processing is performed to optimize the segmentation effect. Apply morphological operations to remove noise or fill holes; use Gaussian filtering or bilateral filtering to smooth segmentation contours and adjust model hyperparameters based on prediction results; fine-tune the model on a specific dataset to improve performance on a specific task; After completing the thumbnail annotation, the image is deeply segmented using the Yolov11 instance segmentation algorithm. The Yolov11 algorithm can quickly and accurately identify labeled objects, such as cells and blood vessels, and separate them from the background image to generate independent segmentation masks. After the segmentation process is completed, Python's PIL library is called again to reassemble multiple segmented 1024*1024 images back to the original size based on the previous splitting rules and parameters, ensuring that the segmentation results can be completely restored to the original image scale, providing complete data support for subsequent 3D reconstruction.

6. The method for visualizing a three-dimensional full-information atlas of biological tissue according to claim 1, characterized in that: In the process of organizing information fusion and digital display, the specific steps to achieve image overlay include: 9a) In Mimics software, fine-tune the 3D digital model created by serial slices by moving, rotating, and resizing it to accurately match it with the existing image geometry model; finally, save the matched 3D model in STL format; 9b) converting the distribution map of different cell types in the tissue obtained by tissue clearing and the stereoscopic image obtained by tissue clearing into STL format and directly importing them into the three-dimensional digital model function layer of the serial section modeling; 9c) importing the 3D model image of collagen fiber arrangement and distribution of the intact tissue reconstructed from the SEM image into the 3D digital model of the serial section modeling in STL format to generate a new 3D model functional layer; 9d) When processing and drawing data, the 3D mechanics cloud map is directly drawn, and the 3D digital model modeled by continuous slicing is imported in STL format to generate a new 3D model functional layer; 9e) The protein quantitative and distribution data obtained from the protein distribution map are constructed using the same method as the mechanical cloud map. After the protein distribution cloud map is obtained, it is imported into the 3D digital model of the serial section modeling in STL format to generate a new 3D model functional layer; 9f) The three-dimensional image generated by the omics data is finally imported into the three-dimensional digital model of continuous section modeling in STL format to generate a new three-dimensional model functional layer.

Images