Tumor heterogeneity assessment devices, methods and electronic equipment
By leveraging the collaborative work of the acquisition, processing, and evaluation modules, three-dimensional local image patches are extracted and habitat cluster-labeled volumetric maps are generated. This addresses the lack of three-dimensional spatial structural information and interpretability in tumor heterogeneity assessment, enabling voxel-level fine analysis and stable evaluation results.
Patent Information
- Application Number
- CN202611045833.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-14
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies struggle to fully utilize the continuous information of tumors in three-dimensional space, lacking voxel-level fine analysis and interpretable quantitative indicators, resulting in unstable and insufficiently interpretable tumor heterogeneity assessment results.
The acquisition module obtains the image to be evaluated and the tumor ROI mask, the processing module extracts three-dimensional local image patches and maps them into feature embedding vectors to generate a three-dimensional habitat cluster label volume map, and the evaluation module calculates the number of three-dimensional connected regions and the maximum connected region volume to generate a heterogeneity score and a habitat cluster label volume map.
It enables voxel-level fine analysis and interpretability quantification in three-dimensional space, improving the spatial expressive power and interpretability of tumor heterogeneity assessment.
Smart Images

Figure CN123118448A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to a device, method and electronic equipment for assessing tumor heterogeneity. Background Technology
[0002] Tumor intratumoral heterogeneity refers to the differences in cellular composition, metabolic state, microenvironment characteristics, blood supply, and imaging manifestations among different regions within the same tumor. It is closely related to tumor development, invasion, metastasis, treatment sensitivity, and prognostic assessment. How to non-invasively, quantitatively, stably, and interpretably assess tumor intratumoral heterogeneity based on medical imaging has become an important research direction in the fields of medical image analysis and intelligent tumor diagnosis and treatment.
[0003] The relevant technologies mainly include the following categories: The first category is texture analysis or radiomics methods based on two-dimensional slices, which usually extract manual features such as grayscale, texture, and morphology within the tumor region or selected slices to construct heterogeneity indicators or risk scoring models. This is a relatively mature approach, but it is difficult to fully utilize the continuity information between tumor slices. The second category is habitat analysis methods based on local region division, which attempt to divide the tumor region into several sub-regions with different imaging features to enhance the locality and interpretability of the assessment. The third category is representation methods based on deep learning, which can automatically learn image features and output classification labels or overall feature vectors. The fourth category is heterogeneity quantification methods based on clustering labels, which calculate scores based on the differences in category proportions or feature distributions.
[0004] However, many related technologies are based on two-dimensional slices or overall statistics, making it difficult to fully utilize the three-dimensional spatial continuity of tumors, especially for lesions with irregular shapes or distributed across multiple layers, which may lead to the loss of spatial information; deep learning methods mostly output overall representations, making it difficult to directly establish voxel-level local representations within ROIs, and thus unable to support subsequent voxel-level habitat clustering and three-dimensional habitat reconstruction; traditional handcrafted features or radiomics features are sensitive to imaging conditions, and differences in grayscale distribution between different cases may lead to a decrease in the comparability of local features, affecting the stability of habitat segmentation; related technologies, such as self-supervised or contrastive learning methods, rarely incorporate the spatial neighbors within the tumor ROI. Using proximity and distance differences as sample constraints makes it difficult for the encoder's learned embeddings to fully serve habitat clustering. Habitat clustering methods often use two-dimensional regions or coarse-grained superpixels as objects, making it difficult to form voxel-level label volumetric maps that can be backfilled into the original three-dimensional space, which is not conducive to observing the spatial continuity and transition relationships of habitat regions. Heterogeneity scores rarely consider the three-dimensional connectivity structure and the degree of spatial fragmentation, making it difficult to truly reflect the complexity of the internal spatial tissue of tumors. If the evaluation results are output as a single numerical value, it is difficult to intuitively reflect the spatial location and local structural differences corresponding to the score, and the interpretability is still insufficient, which urgently needs to be addressed. Summary of the Invention
[0005] This application provides a tumor heterogeneity assessment device, method, and electronic device to address the problems in tumor heterogeneity assessment, such as the lack of three-dimensional spatial structural information, voxel-level fine analysis, and interpretable quantitative indicators.
[0006] The first aspect of this application provides a tumor heterogeneity assessment device, comprising: The acquisition module is used to acquire the image to be evaluated and the tumor ROI mask of the image to be evaluated; The processing module is used to extract three-dimensional local image patches from the image to be evaluated based on the tumor ROI mask, map the three-dimensional local image patches into feature embedding vectors, and generate a three-dimensional habitat cluster label volume map based on the feature embedding vectors. The evaluation module is used to determine the number of three-dimensional connected domains and the maximum connected domain volume for each habitat category based on the three-dimensional habitat cluster label volume map, calculate the heterogeneity score based on the number of three-dimensional connected domains and the maximum connected domain volume for each habitat category, and obtain the tumor heterogeneity assessment result based on the heterogeneity score and / or the three-dimensional habitat cluster label volume map.
[0007] Optionally, the processing module includes: The preprocessing unit is used to preprocess the image to be evaluated based on the tumor ROI mask to obtain a preprocessed three-dimensional image. The mapping unit is used to extract the three-dimensional local image patch from the preprocessed three-dimensional image, using the voxel in the tumor ROI mask as the target center point, extracting the three-dimensional local image patch of a preset size from the preprocessed three-dimensional image, and inputting the three-dimensional local image patch into the trained three-dimensional encoder to obtain the feature embedding vector corresponding to the target center point; The generation unit is used to cluster the feature embedding vectors, determine the habitat category label of each voxel, and generate a three-dimensional habitat cluster label volume map based on the habitat category label.
[0008] Optionally, the generation unit includes: The backfill subunit is used to backfill the habitat category labels into the ROI space of the image to be evaluated based on the original three-dimensional coordinates, so as to generate the three-dimensional habitat cluster label volume map.
[0009] Optionally, the preprocessing unit includes: A sub-unit is defined to determine the minimum bounding box of the tumor region in three-dimensional space based on the coordinates of the tumor voxels in the tumor ROI mask. The cropping subunit is used to expand a preset number of voxels outward in the first preset direction, the second preset direction and the third preset direction of the minimum bounding box, respectively, and to simultaneously crop the image to be evaluated and the tumor ROI mask based on the expanded bounding box, so as to obtain the cropped image to be evaluated and the cropped tumor ROI mask. The standardization subunit is used to perform grayscale standardization on the cropped image to be evaluated within the tumor ROI range corresponding to the cropped tumor ROI mask, so as to obtain the preprocessed three-dimensional image.
[0010] Optionally, it also includes: a training module, wherein, The training module is specifically used to: construct an initial 3D encoder and a training dataset; Multiple training cases are selected from the training dataset, and voxels are sampled within the tumor ROI of the training cases. Three-dimensional local image patches for training are extracted with each voxel as the center. Multiple triplet training samples are determined based on the three-dimensional local image patches used for training, wherein each triplet training sample includes an anchor sample, a positive sample, and a negative sample; The training 3D local image patches corresponding to the anchor point samples, the positive samples, and the negative samples are input into the initial 3D encoder to obtain the feature embedding vectors corresponding to the triplet training samples; The loss value is calculated based on the preset triplet loss function and the feature embedding vector corresponding to the triplet training sample, and the parameters of the initial 3D encoder are updated according to the loss value until the preset qualification conditions are met, thus obtaining the trained 3D encoder.
[0011] Optionally, each of the triplet training samples includes an anchor sample, a positive sample, and a negative sample, wherein, The anchor point sample is a training three-dimensional local image block corresponding to any voxel within the tumor ROI of any training case. The positive sample is a training three-dimensional local image patch corresponding to a voxel that belongs to the same training case as the anchor sample and whose three-dimensional spatial distance does not exceed the first preset threshold. The negative samples include cross-case negative samples and long-distance negative samples within the same case. The cross-case negative samples are training 3D local image patches corresponding to voxels within the ROI of other cases. The long-distance negative samples within the same case are training 3D local image patches corresponding to voxels that belong to the same training case as the anchor sample and whose distance in the third preset direction is not less than the second preset threshold.
[0012] Optionally, the evaluation module includes: The calculation unit is used to calculate a heterogeneity score based on the number of three-dimensional connected components and the maximum connected component volume for each habitat category, wherein the formula for calculating the heterogeneity score is: ; in, For the heterogeneity score, The total prime number of the tumor ROI. For the number of habitat categories, For the first The largest connected component volume among all habitat categories For the first The number of three-dimensional connected domains for each habitat category.
[0013] A second aspect of this application provides a method for assessing tumor heterogeneity, employing a tumor heterogeneity assessment device as described in any of the above claims, comprising the following steps: Obtain the image to be evaluated and the tumor ROI mask of the image to be evaluated; Based on the tumor ROI mask, three-dimensional local image patches are extracted from the image to be evaluated, and the three-dimensional local image patches are mapped into feature embedding vectors. A three-dimensional habitat cluster label volume map is generated based on the feature embedding vectors. Based on the three-dimensional habitat cluster label volume map, the number of three-dimensional connected domains and the maximum connected domain volume of each habitat category are determined, and a heterogeneity score is calculated based on the number of three-dimensional connected domains and the maximum connected domain volume of each habitat category. The tumor heterogeneity assessment result is obtained based on the heterogeneity score and / or the three-dimensional habitat cluster label volume map.
[0014] Optionally, the step of extracting three-dimensional local image patches from the image to be evaluated based on the tumor ROI mask, mapping the three-dimensional local image patches to feature embedding vectors, and generating a three-dimensional habitat cluster label volume map based on the feature embedding vectors includes: Based on the tumor ROI mask, the image to be evaluated is preprocessed to obtain a preprocessed three-dimensional image; The three-dimensional local image patch is extracted from the preprocessed three-dimensional image. The voxel in the tumor ROI mask is used as the target center point. The three-dimensional local image patch of a preset size is extracted from the preprocessed three-dimensional image. The three-dimensional local image patch is input into the trained three-dimensional encoder to obtain the feature embedding vector corresponding to the target center point. Cluster the feature embedding vectors to determine the habitat category label for each voxel, and generate a three-dimensional habitat cluster label volume map based on the habitat category label.
[0015] A third aspect of this application provides an electronic device, including: a tumor heterogeneity assessment device as described in any of the preceding claims.
[0016] Therefore, this embodiment of the application acquires the image to be evaluated and its tumor ROI mask through an acquisition module; the processing module extracts three-dimensional local image patches from the image to be evaluated based on the tumor ROI mask, maps the three-dimensional local image patches to feature embedding vectors, and generates a three-dimensional habitat cluster label volume map based on the feature embedding vectors; the evaluation module determines the number of three-dimensional connected components and the maximum connected component volume for each habitat category based on the three-dimensional habitat cluster label volume map, calculates a heterogeneity score based on the number of three-dimensional connected components and the maximum connected component volume for each habitat category, and obtains the tumor heterogeneity assessment result based on the heterogeneity score and / or the three-dimensional habitat cluster label volume map. This solves the problems of lacking three-dimensional spatial structure information, voxel-level fine analysis, and interpretable quantitative indicators in tumor heterogeneity assessment.
[0017] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0018] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a schematic diagram of a tumor heterogeneity assessment device provided according to an embodiment of this application; Figure 2 This is a flowchart of a tumor heterogeneity assessment method provided according to an embodiment of this application. Detailed Implementation
[0019] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.
[0020] Before introducing the tumor heterogeneity assessment method of the embodiments of this application, let me briefly introduce the tumor heterogeneity assessment methods in related technologies.
[0021] Specifically, tumor heterogeneity refers to the differences in cellular composition, metabolic state, microenvironment characteristics, blood supply, degree of necrosis, and imaging manifestations among different regions within the same tumor. Tumor heterogeneity is closely related to tumor development, invasion and metastasis, treatment sensitivity, and prognostic assessment. Therefore, how to non-invasively, quantitatively, stably, and interpretably assess tumor heterogeneity based on medical imaging has become an important research direction in the fields of medical image analysis and intelligent tumor diagnosis and treatment.
[0022] With the development of three-dimensional medical imaging technologies such as CT (Computed Tomography), MRI (Magnetic Resonance Imaging), and PET (Positron Emission Tomography), the methods for assessing tumor heterogeneity based on imaging data are gradually increasing. Among these technologies, the more common methods mainly include the following categories: The first category is texture analysis or radiomics analysis methods based on two-dimensional slices. These methods typically extract handcrafted features such as grayscale, texture, and morphology from the tumor region or selected slices, and further construct heterogeneity indicators, classification models, or risk scoring models. These methods are relatively mature and have been widely applied in tasks such as pathological type differentiation, efficacy prediction, and prognostic analysis. However, these methods usually rely on the statistical characteristics of two-dimensional slices or the lesion as a whole, making it difficult to fully utilize the interlayer continuity of the tumor in three-dimensional space, and also difficult to reflect the spatial tissue relationships between different local regions within the tumor.
[0023] The second category is habitat analysis methods based on local region segmentation or cluster analysis. These methods typically attempt to divide the tumor region into several sub-regions with different imaging features to reflect the distribution patterns of different microenvironments within the tumor. Compared to methods that only output overall statistics, habitat analysis can enhance the locality and interpretability of heterogeneity assessment to some extent. However, related habitat analysis methods often rely on manually designed radiomics features, grayscale features, texture features, or two-dimensional region segmentation results, making it difficult to establish stable voxel-level local representations within a three-dimensional tumor region of interest (ROI) and to reconstruct continuous three-dimensional habitat label volumetric maps.
[0024] The third category is medical image representation methods based on deep learning. Deep learning methods can automatically learn features from images and demonstrate good feature representation capabilities in tasks such as tumor classification, segmentation, and risk prediction. Related deep learning methods are typically geared towards case-level, slice-level, or lesion-level overall tasks, with outputs often including classification labels, risk scores, segmentation masks, or overall feature vectors. However, for tumor heterogeneity assessment, obtaining only an overall lesion-level representation is often insufficient to describe the fine-grained differences between different voxels or local regions within a region of interest (ROI), making it difficult to directly apply to the reconstruction of intratumoral habitat regions and spatial heterogeneity analysis.
[0025] The fourth category is heterogeneity quantification methods based on clustering labels or region partitioning results. These methods typically calculate a heterogeneity score based on category proportions, differences in characteristic distributions, the largest sub-region volume, or other statistical indicators after obtaining several categories or sub-regions within the tumor. While these methods can reflect the compositional differences among different regions within the tumor to some extent, if only the number or proportion of categories is considered without further taking into account the connectivity, degree of division, and fragmentation of each category's regions in three-dimensional space, they may not fully reflect the complexity of the tumor's internal spatial structure.
[0026] Overall, tumor heterogeneity assessment techniques are evolving from two-dimensional slice analysis to three-dimensional volumetric analysis, from manual features to deep representation learning, and from overall statistical output to locally interpretable habitat modeling. However, there is still room for improvement in areas such as voxel-level local representation learning within three-dimensional tumor ROIs, local feature constraints based on spatial relationships, three-dimensional reconstruction of habitat labels, and heterogeneity scoring incorporating three-dimensional connectivity structures.
[0027] Although related technologies have enabled a certain degree of quantitative analysis of tumor heterogeneity based on medical images, the following shortcomings still exist from the perspective of fine-grained habitat modeling within three-dimensional tumor ROIs and spatial heterogeneity scoring: (1) Most of the related technologies are based on two-dimensional slices or overall statistics, which makes it difficult to make full use of the three-dimensional spatial continuity of tumors.
[0028] Many heterogeneity analysis methods still rely heavily on single-layer or limited two-dimensional slices, focusing primarily on grayscale distribution, texture variations, or localized differences within the slice. While these methods are relatively simple to implement, they struggle to fully utilize the continuity information of the tumor along inter-layer directions and fail to accurately reflect the tumor's internal three-dimensional spatial structure. Especially for tumor lesions with irregular shapes, complex internal structures, or continuous distribution across multiple layers, relying solely on two-dimensional slices may result in the loss of spatial information, thus affecting the completeness and stability of the heterogeneity assessment results.
[0029] (2) Related deep learning methods mostly output overall representations, making it difficult to directly establish voxel-level local representations within the ROI.
[0030] While deep learning methods can automatically learn medical image features, they are mostly geared towards tasks such as classification, segmentation, and risk prediction, typically outputting case-level, slice-level, or lesion-level overall results. For assessing intratumoral heterogeneity, it is necessary to further characterize the differences between different voxels or local regions within the ROI. If only overall lesion features are obtained, it is difficult to describe the fine-grained changes in different microenvironments within the tumor, and it is also difficult to support subsequent voxel-level habitat clustering and three-dimensional habitat reconstruction.
[0031] (3) Traditional manual features or radiomics features are more sensitive to parameters and imaging conditions, and the stability of local characterization is insufficient.
[0032] Traditional radiomics methods typically rely on predefined, handcrafted features such as grayscale, texture, and morphology. These features have limited expressive power and are easily affected by scanning equipment, reconstruction parameters, slice thickness, window width and level, image noise, and feature extraction parameters. For intratumoral heterogeneity analysis, differences in grayscale distribution and imaging conditions between different cases can reduce the comparability of local features, thus affecting the stability of habitat segmentation and heterogeneity scoring. Therefore, how to obtain stable three-dimensional local representations while reducing reliance on handcrafted feature design remains a problem that related technologies need to solve.
[0033] (4) There are few self-supervised or comparative learning methods related to the technology that combine the internal spatial relationship of the tumor ROI for sample constraints.
[0034] Some related deep representation learning methods employ self-supervised learning, contrastive learning, or metric learning to acquire image features. However, their sample construction often relies primarily on data augmentation, random cropping, or case-level label relationships, failing to fully utilize the spatial proximity and long-distance differences between voxels within a tumor region of interest (ROI). For assessing intratumoral heterogeneity, spatially proximate local regions typically exhibit some continuity, while spatially distant or cross-case local regions may show more significant representational differences. If such spatial priors are not introduced during training, the embeddings learned by the encoder may be insufficient to adequately serve subsequent habitat clustering and local heterogeneity differentiation.
[0035] (5) Related technology habitat clustering methods are difficult to form voxel-level label volume maps that can be backfilled into the original three-dimensional space.
[0036] While some related techniques can cluster or partition tumor regions, the clustered objects are often two-dimensional regions, coarse-grained superpixel regions, or overall statistical features, making it difficult to map the local representation corresponding to each voxel back to the original three-dimensional ROI space. The results obtained in this way typically only provide category proportions or local statistical information, making it difficult to form a three-dimensional habitat cluster label volumetric map consistent with the original image spatial coordinates. This also hinders physicians or researchers from observing the spatial continuity, division, fusion, and transition relationships of different habitat regions within the tumor.
[0037] (6) The related technology heterogeneity score rarely considers the three-dimensional connectivity structure and the degree of spatial fragmentation.
[0038] Most existing heterogeneity quantification metrics focus on differences in texture statistics, category proportions, or feature distributions, but rarely consider the connectivity, fragmentation, and de-fragmentation of different habitat categories in three-dimensional space. In reality, even two tumors with similar category proportions can have significantly different internal spatial organization: a particular category may be continuously concentrated or fragmented into multiple discrete smaller regions. Without incorporating three-dimensional connected domain structures into the analysis, it is difficult to accurately reflect the spatial complexity of tumor internal heterogeneity.
[0039] (7) The heterogeneity of related technologies still has insufficient interpretability of the results.
[0040] Heterogeneity assessment results, if output only as a single numerical value, category proportion, or overall risk score, are insufficient to intuitively reflect the differences in the internal spatial location and local structure of the tumor corresponding to that score. For applications requiring observation of the distribution of the tumor's internal habitat, more valuable results should include both the heterogeneity score and a habitat-labeled volumetric map that can be backfilled into the original 3D image space for further display, analysis, and validation. Related technologies still have shortcomings in the complete chain of "voxel-level local characterization—habitat clustering—3D label reconstruction—spatial connectivity scoring."
[0041] In summary, while existing technologies have enabled some assessment of tumor heterogeneity from the perspectives of radiomics, deep learning, and cluster analysis, a suitable approach is still lacking. This approach can establish voxel-level local embeddings within a 3D tumor region of interest (ROI), train a 3D encoder based on internal tumor spatial relationships, generate a 3D habitat cluster label volumetric map based on the embeddings, and further calculate heterogeneity scores by incorporating 3D connected domain structures. Therefore, it is necessary to propose a tumor heterogeneity assessment method based on 3D local representation learning and habitat clustering to improve the 3D spatial representation, local discriminative power, and interpretability of tumor heterogeneity assessment results.
[0042] This application addresses the aforementioned problems by proposing a tumor heterogeneity assessment device. In this embodiment, an acquisition module acquires the image to be assessed and its tumor ROI mask. A processing module extracts three-dimensional local image patches from the image based on the tumor ROI mask, maps these patches to feature embedding vectors, and generates a three-dimensional habitat cluster label volume map based on the feature embedding vectors. An assessment module determines the number of three-dimensional connected components and the maximum connected component volume for each habitat category based on the three-dimensional habitat cluster label volume map, calculates a heterogeneity score based on these scores, and obtains the tumor heterogeneity assessment result based on the heterogeneity score and / or the three-dimensional habitat cluster label volume map. This solves the problems of lacking three-dimensional spatial structure information, voxel-level fine analysis, and interpretable quantitative indicators in tumor heterogeneity assessment.
[0043] Specifically, Figure 1 This is a schematic diagram of a tumor heterogeneity assessment device provided in an embodiment of this application.
[0044] like Figure 1 As shown, the tumor heterogeneity assessment device 10 includes: an acquisition module 100, a processing module 200, and an assessment module 300.
[0045] The acquisition module 100 is used to acquire the image to be evaluated and the tumor ROI mask of the image to be evaluated. The processing module 200 is used to extract three-dimensional local image patches from the image to be evaluated based on the tumor ROI mask, map the three-dimensional local image patches into feature embedding vectors, and generate a three-dimensional habitat cluster label volume map based on the feature embedding vectors. The evaluation module 300 is used to determine the number of three-dimensional connected domains and the maximum connected domain volume for each habitat category based on the three-dimensional habitat cluster label volume map, calculate the heterogeneity score based on the number of three-dimensional connected domains and the maximum connected domain volume for each habitat category, and obtain the tumor heterogeneity assessment result based on the heterogeneity score and / or the three-dimensional habitat cluster label volume map.
[0046] Specifically, this embodiment of the application achieves a complete processing flow from 3D medical image input to heterogeneity quantification output through the collaborative work of three core modules. The acquisition module 100 is responsible for the data entry, acquiring the 3D medical image to be evaluated and its corresponding tumor ROI mask, thus defining the precise range of the tumor in 3D space for subsequent analysis. The processing module 200 undertakes the core feature extraction and habitat reconstruction tasks: first, it extracts 3D local image patches centered on each voxel within the ROI; then, it uses a trained 3D encoder to map each image patch into a low-dimensional feature embedding vector, compressing high-dimensional image information into clusterable semantic representations; finally, it performs cluster analysis on the embedding vectors of all voxels, grouping voxels with similar local representations into the same habitat category and generating a 3D habitat cluster label volume map, realizing the spatial visualization reconstruction of different habitat regions within the tumor. The assessment module 300 performs quantitative analysis on the aforementioned label volume map, conducting three-dimensional connected component analysis for each habitat category, counting the number of connected components and the largest connected component volume, and calculating the heterogeneity score (ITHScore) based on this. This score comprehensively reflects the diversity and spatial fragmentation of habitat categories within the tumor. Finally, the assessment module 300 outputs the heterogeneity score and / or a three-dimensional habitat cluster label volume map, providing multi-level assessment results from a single numerical indicator to three-dimensional spatial distribution, combining quantitative accuracy and spatial interpretability.
[0047] Further, the patient's three-dimensional medical image (image to be evaluated) and its corresponding tumor ROI mask are read. The three-dimensional medical image can be CT, MRI, PET, or other medical images that can form three-dimensional volume data, preferably CT images. The ROI mask is a binary mask corresponding to the spatial location of the original image, where the tumor region is set to 1 and the background region is set to 0.
[0048] In a preferred embodiment, this application supports reading NIfTI format images and also supports reading DICOM sequences and reconstructing them into a three-dimensional image matrix. After reading, the image data is converted into a three-dimensional array arranged in Z, Y, X order, and the ROI mask is converted into a binary array of the same spatial size.
[0049] Optionally, in some embodiments, the processing module 200 includes: a preprocessing unit, used to preprocess the image to be evaluated based on a tumor ROI mask to obtain a preprocessed three-dimensional image; a mapping unit, used to extract three-dimensional local image patches from the preprocessed three-dimensional image, using voxels within the tumor ROI mask as target center points, extracting three-dimensional local image patches of a preset size from the preprocessed three-dimensional image, and inputting the three-dimensional local image patches into a trained three-dimensional encoder to obtain feature embedding vectors corresponding to the target center points; and a generation unit, used to cluster the feature embedding vectors, determine the habitat category label of each voxel, and generate a three-dimensional habitat cluster label volume map based on the habitat category label.
[0050] Optionally, in some embodiments, the generation unit includes: a backfill subunit, used to backfill habitat category labels into the ROI space of the image to be evaluated based on the original three-dimensional coordinates, so as to generate a three-dimensional habitat cluster label volume map.
[0051] Understandably, the system checks whether the spatial dimensions of the original image and the ROI mask are consistent. When their dimensions, voxel arrangement, or spatial correspondence are inconsistent, spatial alignment or resampling is performed first; when they are consistent, the ROI cropping process begins, and preprocessing is performed to obtain the preprocessed 3D image. After ROI normalization, each voxel within the ROI mask is used as the target center point, and a fixed-size 3D local patch is extracted from the normalized 3D image. This patch is used to describe the grayscale distribution, texture pattern, and local spatial structure within the neighborhood of the target voxel.
[0052] Let the target voxel coordinates be:
[0053] Construct a 3D local patch centered on this voxel:
[0054] Where r is the patch radius. Preferably, patch_size is 9 and patch radius is 4, that is, the size of each local patch is 9×9×9.
[0055] When the target voxel is located at the edge of the clipping region, causing the patch to exceed the clipping boundary, reflection padding is preferred for boundary expansion to ensure that all voxels obtain a 3D local patch of uniform size. For cases where the size is too small for reflection padding, zero-padding can be used. Through this method, each voxel within the tumor ROI corresponds to a 3D local patch, thereby refining the tumor heterogeneity analysis object from the entire lesion or a 2D slice region to a 3D voxel-level local region.
[0056] After completing the training of the 3D encoder, the trained encoder was used for heterogeneity assessment of the cases to be analyzed. First, the same image reading, ROI cropping, ROI z-score normalization, and voxel-centered 3D patch extraction steps as in the training phase were performed on the cases to be analyzed. Then, the patches corresponding to all voxels within the ROI were input into the trained 3D encoder in batches to obtain the embedding (feature embedding vector) corresponding to each voxel.
[0057] If there are N voxels within the ROI of the case to be analyzed, then the embedding set can be obtained: ; in, This is the L2 normalized embedding corresponding to the i-th ROI voxel.
[0058] In a preferred embodiment, to improve inference efficiency, the embedding can be extracted in batches.
[0059] Furthermore, after obtaining the embeddings of all voxels within the ROI, cluster analysis is performed on the embedding set to delineate different local habitat regions within the tumor. K-Means clustering is used; assuming the number of clusters is K, clustering the embedding set E yields the cluster label for each voxel: ; Among them, C( ) represents the clustering function. This indicates the habitat category to which the i-th voxel belongs.
[0060] After clustering is completed, the clustering labels of each ROI voxel are backfilled into the ROI space according to their original 3D coordinates to generate a 3D habitat cluster label volume map:
[0061] ; Wherein, L is a three-dimensional habitat cluster label volumetric map, the background label is 0, and the labels for different habitats within the tumor are 1 to K. Through this step, the present invention not only obtains a heterogeneity score, but also obtains a habitat label volumetric map corresponding to the original three-dimensional image space, thereby enhancing the spatial interpretability of the results.
[0062] Optionally, in some embodiments, the preprocessing unit includes: a determination subunit, configured to determine the minimum bounding box of the tumor region in three-dimensional space based on the coordinates of tumor voxels in the tumor ROI mask; a cropping subunit, configured to expand a predetermined number of voxels outward in a first preset direction, a second preset direction, and a third preset direction of the minimum bounding box, and simultaneously crop the image to be evaluated and the tumor ROI mask based on the expanded bounding box, to obtain the cropped image to be evaluated and the cropped tumor ROI mask; and a standardization subunit, configured to perform grayscale standardization on the cropped image to be evaluated within the tumor ROI range corresponding to the cropped tumor ROI mask, to obtain the preprocessed three-dimensional image.
[0063] Understandably, the minimum bounding box of the tumor region in three-dimensional space is determined based on the voxel coordinates with a value of 1 in the ROI mask. To preserve the tumor edge and its adjacent context information, a preset padding (preset number of voxels) is extended outward in the Z (first preset direction), Y (second preset direction), and X (third preset direction) directions of the bounding box, and the original image and ROI mask are simultaneously cropped using the extended bounding box. In a preferred embodiment, the padding value is 4 voxels. After completing the three-dimensional cropping, grayscale normalization is performed on the cropped tumor ROI. To reduce the impact of different cases, different scanning parameters, different reconstruction conditions, and differences in grayscale distribution on subsequent representation learning, this invention preferably adopts an in-ROI z-score normalization method.
[0064] Specifically, let the original cropped image be I, and the ROI mask be M, where M(x) = 1 indicates that voxel x belongs to the tumor region. First, calculate the mean and standard deviation of the voxel grayscale values within the ROI:
[0065] ; Then, the silhouette image is standardized:
[0066] ; in, is the lower limit of standard deviation, used to avoid numerical instability caused by excessively small gray-level variance within the ROI; c is the cutoff threshold. Preferably, =1e 3, c=5.0. By standardizing within the ROI, the impact of different image sources and grayscale differences on local patch embedding can be reduced, resulting in a preprocessed 3D image, thereby improving the stability of subsequent clustering and heterogeneity scoring.
[0067] Optionally, in some embodiments, the system further includes a training module, which is specifically used for: constructing an initial 3D encoder and a training dataset; selecting multiple training cases from the training dataset, sampling voxels within the tumor ROI of the training cases, and extracting 3D local image patches for training centered on each voxel; determining multiple triplet training samples based on the 3D local image patches for training, wherein each triplet training sample includes an anchor sample, a positive sample, and a negative sample; inputting the 3D local image patches for training corresponding to the anchor sample, the positive sample, and the negative sample into the initial 3D encoder to obtain the feature embedding vector corresponding to the triplet training sample; calculating a loss value based on a preset triplet loss function and the feature embedding vector corresponding to the triplet training sample, and updating the parameters of the initial 3D encoder based on the loss value until a preset qualification condition is met to obtain the trained 3D encoder.
[0068] Understandably, the initial 3D encoder is first constructed and the training dataset is prepared. In each training batch, multiple training cases are selected from the training dataset, voxels are sampled within the tumor ROI of each case, and 9×9×9 3D local image patches centered on each voxel are extracted as training samples. Triple training samples are constructed based on these training image patches. In each training batch, the system calculates the Euclidean distance between the anchor embedding and the candidate negative embedding, and selects semi-difficult negative samples online to participate in the triple loss calculation.
[0069] Let the distance between the anchor and the positive sample be: ; The distance between the anchor and the negative is: ; Semi-difficult negative samples satisfy: > And preferably select the negative that meets the above conditions and is closest to the anchor: ; If no negative satisfies the semi-hard condition, the hardest negative can be chosen as a fallback: ; This invention uses a preset triplet loss function to train a 3D encoder, which makes spatially adjacent and locally structurally similar patches closer together in the embedding space, and makes cross-case or same-case distant patches farther apart in the embedding space.
[0070] For the The training triplet has the following loss: ; in, Indicates the first The distance between each anchor and the positive embedding. Indicates the first The embedding distance between each anchor and the selected negative, where m is the margin.
[0071] The total loss within a batch is:
[0072] Where B represents the number of valid triples. Preferably, the margin is set to 1.0.
[0073] This loss function allows the encoder to learn embeddings related to the internal spatial structure of the tumor and local image differences during training, providing a more discriminative representation basis for subsequent voxel-level clustering. The loss value is calculated based on the triplet loss function to update the encoder parameters until the qualification conditions are met. The trained 3D encoder can then be used for voxel-level embedding extraction during the inference phase.
[0074] It should be noted that this invention constructs a 3D local encoder to map each 3D local patch to a low-dimensional embedding. The 3D local encoder preferably employs a lightweight 3D convolutional network, including 3D convolutional layers, normalization layers, nonlinear activation layers, downsampling layers, global pooling layers, and feature projection layers.
[0075] In a preferred embodiment, the 3D encoder includes: a stem module for initial 3D convolutional feature extraction of the input patch; a stage1 module for performing a first downsampling and extracting mid-to-low-level spatial structure features; a stage2 module for performing a second downsampling and extracting high-level local semantic features; a stage3 module for further extracting local spatial representations; an adaptive 3D average pooling layer for compressing the 3D feature map into a fixed-length vector; a linear projection layer for mapping the feature vector to a preset-dimensional embedding; and an L2 normalization layer for normalizing the embedding to a unit feature space. In a preferred embodiment, the 3D convolution module uses a combination of Conv3D, GroupNorm, and GELU activation functions.
[0076] Let the 3D encoder be Then, the embedding corresponding to the i-th voxel is L2 normalized: ; Preferably, the embedding dimension is 128, but it can also be set to 32, 64 or other dimensions depending on the data volume and application requirements.
[0077] Optionally, in some embodiments, each triplet training sample includes an anchor sample, a positive sample, and a negative sample. The anchor sample is a training 3D local image block corresponding to any voxel within the tumor ROI of any training case. The positive sample is a training 3D local image block corresponding to a voxel belonging to the same training case as the anchor sample and whose 3D spatial distance does not exceed a first preset threshold. The negative samples include cross-case negative samples and long-distance negative samples within the same case. Cross-case negative samples are training 3D local image blocks corresponding to voxels within the ROI of other cases. Long-distance negative samples within the same case are training 3D local image blocks corresponding to voxels belonging to the same training case as the anchor sample and whose distance in a third preset direction is not less than a second preset threshold.
[0078] Understandably, to enable the encoder to learn local representations suitable for tumor internal habitat analysis, this invention employs a triplet self-supervised learning strategy based on spatial distance gating. Each training sample set includes anchor samples, positive samples, and negative samples.
[0079] Specifically, the Anchor sample is a three-dimensional local patch corresponding to any voxel within the ROI of a tumor in a given case, denoted as: Its corresponding embedding is: .
[0080] Positive samples are selected from voxel patches within the ROI that belong to the same case as the anchor and are spatially adjacent. Preferably, positive samples are selected from patches located in adjacent or near-adjacent z-layers to take advantage of the continuity of the tumor in the inter-layer direction. In a preferred embodiment, the positive sample satisfies the condition that its three-dimensional coordinate L1 distance from the anchor does not exceed a preset threshold. ; in, For anchor voxel coordinates, For positive voxel coordinates, This is the threshold for the neighborhood distance of positive samples. Preferably, =5.
[0081] Furthermore, the system can set the cross-layer positive sampling probability, so that positive samples preferentially come from voxels in different z-layers from the anchor but spatially adjacent. Preferably, the cross-layer positive sampling probability is 0.7, and the minimum z-direction distance between layers is 1.
[0082] Negative samples are used to constrain the encoder to distinguish local representations of spatially distant regions in different cases or within the same case. The candidate negative samples in this invention include two categories: the first category is cross-case negative samples, i.e., 3D local patches from other cases' ROIs; the second category is long-distance negative samples within the same case, i.e., 3D local patches belonging to the same case as the anchor, but at a distance of not less than a preset threshold in the z-direction from the anchor.
[0083] The candidate negative sample set is defined as: ; in, Indicates the first The case to which each patch belongs. Indicates the case belonging to anchor. and Representing the candidate negative sample and the anchor respectively Direction coordinates For negative samples from distant locations of the same case Directional distance threshold. Preferably, =5.
[0084] Optionally, in some embodiments, the evaluation module includes: a calculation unit for calculating a heterogeneity score based on the number of three-dimensional connected domains and the maximum connected domain volume for each habitat category, wherein the formula for calculating the heterogeneity score is: ; in, For heterogeneity scoring, The total prime number of the tumor ROI. For the number of habitat categories, For the first The largest connected component volume among all habitat categories For the first The number of three-dimensional connected domains for each habitat category.
[0085] Understandably, after obtaining the 3D habitat cluster label volume map, a 3D connected component analysis is performed for each habitat category. Let the total prime number of the tumor ROI be: ; For the i-th habitat category, extract its corresponding three-dimensional binary region: ; right Perform a three-dimensional connected component analysis to obtain the number of connected components in this category. And calculate the volume of the largest connected component in this category. .
[0086] In the preferred embodiment, the three-dimensional connected component analysis uses a 26-neighbor approach, but 6-neighbor or 18-neighbor approaches can also be used. When a habitat category exhibits a continuous and concentrated distribution in three-dimensional space, its number of connected components is small, and the largest connected component volume is large, resulting in a lower heterogeneity score. When multiple habitat categories exhibit a fragmented, multifocal, or highly dispersed distribution in three-dimensional space, the number of connected components increases, the largest connected component volume decreases relatively, and the corresponding heterogeneity score increases. Therefore, this score not only reflects the presence of different habitat categories but also reflects the complexity and fragmentation of the internal spatial organization of the tumor.
[0087] It should be noted that after completing the ITHScore calculation, the embodiments of this application output one or more of the following results: tumor intra-tumor heterogeneity score (ITHScore); three-dimensional habitat cluster label volume map; voxel coordinate file within the ROI; embedding file of each voxel within the ROI; cluster label file of each voxel within the ROI; number of connected components and maximum connected component volume for each habitat category; analysis parameters, including patch size, embedding dimension, number of cluster categories, connected component adjacency method, encoder model path, etc.
[0088] In a preferred embodiment, the three-dimensional habitat cluster label volumetric map is saved as a medical image volumetric data file, such as an NIfTI file; the embedding, voxel coordinates, and cluster labels can be saved as an array file; and the ITHS score and statistical results for each category can be saved as JSON or CSV files. By saving the above results, the present invention can support subsequent visualization, inter-case comparison, result verification, and algorithm parameter traceability.
[0089] In summary, the present invention, through the above technical solution, can produce the following direct technical effects: First, by cropping the ROI and standardizing the z-score within the ROI, we can reduce the impact of irrelevant background areas and differences in grayscale distribution among different cases on subsequent representation learning, thereby improving the consistency of input data.
[0090] Secondly, by constructing a 9×9×9 three-dimensional local patch centered on a voxel, it is possible to simultaneously preserve intra-slice information and inter-layer continuity information in the local analysis unit, making fuller use of the three-dimensional spatial structure of the tumor compared to two-dimensional slice analysis methods.
[0091] Furthermore, by mapping each patch to an L2-normalized embedding through a 3D encoder, a comparable low-dimensional local representation can be established for each voxel within the ROI, thus providing a basis for voxel-level habitat clustering.
[0092] Furthermore, through the spatial distance-gated triplet training strategy, the encoder can utilize the spatial proximity and long-distance difference relationships within the tumor to make spatially adjacent patches tend to be closer in the embedding space, and make long-distance patches across cases or within the same case tend to be separated in the embedding space, thereby improving the ability of the embedding to express local differences within the tumor.
[0093] Furthermore, through online semi-hard negative mining, the system can dynamically select more valuable negative samples within each training batch, enhancing the encoder's ability to discriminate difficult local differences.
[0094] Furthermore, by performing KMeans clustering on all voxel embeddings within the ROI and backfilling the cluster labels into the original 3D space, a 3D habitat cluster label volume map can be generated, so that the heterogeneity results can be represented not only as numerical values but also as spatial partitioning results.
[0095] Finally, by performing three-dimensional connected domain analysis on the three-dimensional habitat cluster label volume map and calculating the ITHS score based on the number of connected domains, the largest connected domain volume, and the total tumor volume, the heterogeneity score can simultaneously reflect the differences in habitat categories and the degree of three-dimensional spatial fragmentation, thereby improving the ability of heterogeneity assessment to express the complexity of the internal spatial organization of tumors.
[0096] The following three embodiments illustrate the present invention: Example 1: Encoder Training Process Based on 3D Local Patch This embodiment illustrates a training method for a 3D local encoder based on self-supervised learning using spatial distance-gated triples. This training process is used to obtain a 3D encoder model capable of characterizing local regional differences within a tumor region of interest (ROI).
[0097] (1) Data preparation In this embodiment, each case includes a three-dimensional medical image and a corresponding tumor ROI mask. The three-dimensional medical image can be a NIfTI format file or a DICOM sequence folder; the tumor ROI mask is a binary mask file corresponding to the spatial location of the original image.
[0098] The system first reads the 3D medical image and the ROI mask, and checks whether their spatial dimensions are consistent. If there is a label value greater than 1 in the mask, it is converted into a binary mask, that is, the tumor area is assigned a value of 1, and the non-tumor area is assigned a value of 0.
[0099] (2) ROI trimming and standardization The system determines the three-dimensional bounding box of the tumor based on the position of the tumor voxels in the ROI mask, and extends the pre-set padding outward in the Z, Y, and X directions to preserve the tumor edge and its surrounding local context information. In a preferred embodiment, the padding is set to 4 voxels.
[0100] After completing the ROI cropping, the system calculates the mean and standard deviation of grayscale values within the tumor ROI and performs z-score standardization on the cropped 3D image. To avoid numerical instability caused by excessively small grayscale standard deviations within the ROI, a lower limit parameter for the standard deviation is set. =1e 3; Simultaneously, the standardized voxel values are cropped to [ Within the range of 5.0, 5.0. This step can reduce the impact of different cases, different scanning conditions, and different grayscale ranges on subsequent local representation learning.
[0101] (3) Construction of three-dimensional local patches After standardization, the system randomly samples voxels within the tumor region of interest (ROI) as the center points for training samples. Using each target voxel as the center, a fixed-size 3D local patch is extracted from the standardized 3D image. Preferably, the patch size is 9×9×9, corresponding to a patch radius of 4. This local patch contains the 3D grayscale distribution and spatial structure information surrounding the target voxel.
[0102] When the target voxel is located at the edge of the cropping area, causing the patch to exceed the boundary, it is preferable to use reflection padding for boundary expansion; when the image size is too small to be suitable for reflection padding, zero padding can also be used for filling.
[0103] (4) Construction of triplet samples This embodiment uses triplet samples to train a 3D encoder. Each triplet includes an anchor sample, a positive sample, and a negative sample.
[0104] Anchor samples are voxel centers randomly selected within the ROI of a tumor in a specific case.
[0105] Positive samples are selected from voxel patches within the same case's ROI that are spatially adjacent to the anchor. Preferably, positive samples satisfy the condition that their 3D L1 distance to the anchor does not exceed 5. Simultaneously, to utilize the continuity of the tumor in the inter-layer direction, the system sets a cross-layer positive sampling probability, so that positive samples preferentially come from regions in different z-layers but spatially adjacent to the anchor. In a preferred embodiment, the cross-layer positive sampling probability is 0.7, and the minimum cross-layer z-direction distance is 1.
[0106] Negative samples are divided into two categories: the first category is cross-case negative samples, which are local patches from other case ROIs; the second category is long-distance negative samples from the same case, which are local patches belonging to the same case as the anchor but at a distance of not less than a preset threshold in the z-direction. In a preferred embodiment, the z-direction distance threshold for long-distance negative samples from the same case is 5.
[0107] Through the above sample construction method, the encoder training process can utilize the spatial continuity and long-distance differences of local regions within the tumor to make adjacent regions tend to be close in the embedding space, and make distant regions across cases or within the same case tend to be separated in the embedding space.
[0108] (5) Structure of the three-dimensional encoder In this embodiment, the 3D encoder employs a lightweight 3D convolutional network. This network includes a stem module, multiple 3D convolutional stages, adaptive 3D average pooling layers, and linear projection layers. Each 3D convolutional module preferably includes Conv3D, GroupNorm, and GELU activation functions. The encoder ultimately outputs a fixed-dimensional embedding and performs L2 normalization on the embedding to make it suitable for subsequent distance metrics and clustering analysis.
[0109] In a preferred embodiment, the embedding dimension is set to 128; it can also be set to 32, 64 or other dimensions depending on the amount of data, memory conditions and application requirements.
[0110] (6) Online semi-difficult negative sample mining and model training Within each training batch, the system first calculates the embedding distance between the anchor and the positive. And calculate the embedding distance between the anchor and the candidate negative. The system prioritizes selecting those that meet the requirements. > The system identifies semi-hard negatives and selects the closest negative to the anchor from those that meet the criteria for triplet loss calculation. If no semi-hard negative exists in the batch, the system backtracks and selects the hardest negative closest to the anchor.
[0111] The training loss uses triplet loss: ; Where m is the margin, preferably set to 1.0. The average loss of effective triples within a batch is taken as the total loss. During training, the AdamW optimizer, cosine annealing learning rate scheduling, gradient pruning, and mixed precision training can be used to improve training stability and efficiency.
[0112] After training, the system saves the 3D encoder model weights, training configuration, training logs, and training / validation case segmentation information. This embodiment provides a 3D encoder suitable for representing local regions within a tumor region of interest (ROI), laying the model foundation for subsequent voxel-level embedding extraction and habitat clustering.
[0113] Example 2: Single-Case Tumor Heterogeneity Assessment Process This embodiment illustrates the specific process of using a trained 3D encoder to assess the heterogeneity of a single tumor.
[0114] (1) Input data The input includes the 3D medical image of the case to be analyzed, the tumor ROI mask, and the weight file of the trained 3D encoder. The 3D medical image can be an NIfTI file or a DICOM sequence folder, and the ROI mask is a binary mask corresponding to the image space.
[0115] (2) ROI trimming and standardization After reading the image of the case to be analyzed and the ROI mask, the system determines the three-dimensional bounding box of the tumor in the same manner as in the training phase, and extends it outward by 4 voxel paddings to obtain the cropped local tumor image and ROI mask. Subsequently, z-score normalization is performed within the ROI, and the normalized voxel values are cropped to […]. [5.0, 5.0]. Using a consistent standardization approach during the training and inference phases helps ensure a consistent encoder input distribution.
[0116] (3) Extraction of all voxel embeddings within the ROI The system acquires the coordinates of all tumor voxels within the ROI mask and extracts a 9×9×9 3D local patch centered on each voxel. These patches are then input into the trained 3D encoder in batches to obtain the embedding corresponding to each ROI voxel.
[0117] Through this step, each voxel within the ROI of the tumor to be analyzed obtains a low-dimensional normalized embedding, forming a voxel-level local representation set.
[0118] (4) Embedding clustering and habitat label volume graph generation The system performs cluster analysis on all voxel embeddings within the same ROI for each case. Preferably, the KMeans clustering algorithm is used, with the number of clusters K set to 5. After clustering, each ROI voxel receives a habitat category label.
[0119] Subsequently, the system backfills the clustering labels of each voxel into the cropped ROI space according to its three-dimensional spatial coordinates, generating a habitat label volume map within the cropped area; then, based on the position of the cropping frame, the label volume map is backfilled into the original image space to form a complete habitat label volume map consistent with the original three-dimensional medical image space.
[0120] In this habitat-labeled volumetric map, the background label is 0, and the labels for different habitat categories within the tumor range from 1 to K. This result can be used for subsequent visualization and spatial structure analysis.
[0121] (5) Three-dimensional connected component analysis and ITH Score calculation After obtaining the 3D habitat label volume map, the system performs 3D connectivity analysis on each habitat category. Preferably, a 26-adjacency method is used to determine 3D connectivity; alternatively, a 6-adjacency or 18-adjacency method can be used as needed.
[0122] For the For each habitat category, the system counts the number of its three-dimensional connected components. And calculate the maximum connected component volume in this category. Simultaneously, the total prime number V of the tumor ROI is calculated. Then, the ITHS score is calculated using the following formula:
[0123] When a certain habitat category is continuously and concentratedly distributed in three-dimensional space, its number of connected domains is small and its largest connected domain volume is large, resulting in a relatively low ITHS score. When different habitat categories are fragmented, multifocal, or dispersed within the tumor, the number of connected domains increases and the largest connected domain volume relatively decreases, leading to a relatively higher ITHS score. Therefore, this score can reflect the three-dimensional spatial complexity of habitat regions within the tumor.
[0124] (6) Output results The final output of this embodiment is as follows: Intratumor heterogeneity score (ITHS score); 3D habitat cluster tag volume diagram; ROI in-voxel coordinates; Voxel embedding within the ROI; Clustering labels for each voxel within the ROI; The number of connected components and the maximum connected component volume for each habitat category; Analyze parameters and model information.
[0125] In a preferred embodiment, the three-dimensional habitat label volume map is saved as an NIfTI file, the voxel coordinates, embeddings, and cluster labels are saved as an array file, the ITHS score and statistical information of each habitat category are saved as a JSON file, and the batch case results are saved as a CSV file.
[0126] Through the above output, users can not only obtain a single heterogeneity score, but also a three-dimensional habitat label volume map and intermediate calculation results consistent with the original image space, which facilitates subsequent visualization, verification and inter-case comparison.
[0127] Example 3: Batch Heterogeneity Assessment of Multiple Cases In another embodiment, the present invention can be used for batch processing of multiple cases. The system sequentially reads the 3D image and ROI mask of each case from the case image catalog, calculates the ITHS score for each case according to the process in Embodiment 2, and writes the evaluation results of each case into a summary result file.
[0128] For each case, the system establishes an independent output directory to store the 3D habitat label volumetric map, embedding, cluster labels, and ITHScore statistical file for that case. If a case lacks image files or ROI masks, the system can skip that case and record the error information; if an anomaly occurs during processing, the system can write the anomaly information to the error log for subsequent review.
[0129] Through this implementation method, the present invention can support automated heterogeneity assessment of multi-case datasets, improve processing efficiency, and reduce repetitive work caused by manual case-by-case operations.
[0130] In practical applications, the embodiments of this application provide the following specific details: (1) Explanation of patch size In a preferred embodiment, the patch size is set to 9×9×9. This size allows for control over the input scale and computational burden of the 3D encoder while preserving the local neighborhood information of the target voxel. If the patch size is too small, it may not adequately reflect the local spatial structure; if the patch size is too large, it may introduce too much non-local information and increase computational load. Therefore, 9×9×9 can be considered a preferred setting that balances local expressiveness and computational efficiency.
[0131] In other embodiments, the patch size can also be set to 5×5×5, 7×7×7, 11×11×11 or other odd-numbered sizes. As long as the three-dimensional local region can be extracted with the voxel as the center and input into the encoder, it can be used as an alternative embodiment of the present invention.
[0132] (2) Explanation of embedding dimensions In a preferred embodiment, the embedding dimension is set to 128. This dimension provides sufficient local representation capabilities while maintaining the feasibility of subsequent clustering calculations. Depending on the data size, model complexity, and computational resources, the embedding dimension can also be set to 32, 64, 256, or other dimensions.
[0133] (3) Explanation of three-dimensional connectivity methods In a preferred embodiment, the 3D connected component analysis employs a 26-adjacency approach. A 26-adjacency approach simultaneously considers the spatial connectivity of voxels along face, edge, and corner directions, making it more suitable for analyzing the continuity of habitat regions in a 3D labeled volumetric map. In other embodiments, 6-adjacency or 18-adjacency approaches can also be used to accommodate different spatial connectivity definitions and analysis requirements.
[0134] (4) Alternative solutions for model training methods This invention preferably employs a spatial distance-gated online semi-difficult triplet learning method to train the 3D encoder. However, without altering the basic concept of "voxel-level 3D local representation learning—embedding clustering—3D habitat reconstruction—connected component scoring," other self-supervised, contrastive, or metric learning methods can also be used to train the encoder. For example, SimCLR-type contrastive learning, MoCo-type momentum contrastive learning, BYOL-type self-distillation learning, or other training methods that can map 3D local patches to clusterable embeddings can be used.
[0135] (5) Alternatives to clustering algorithms This invention preferably employs the KMeans clustering algorithm to cluster voxel embeddings within the ROI. In other embodiments, Gaussian Mixture Model, spectral clustering, hierarchical clustering, DBSCAN, Mean Shift, or other clustering methods can also be used, as long as they can divide local regions within the ROI into different habitat categories based on voxel-level embeddings and generate a label volume map that can be backfilled into three-dimensional space, they can all be used as alternative embodiments of this invention.
[0136] (6) Explanation of input image modalities In this embodiment of the invention, CT images are preferably used, but the invention is not limited to CT. For MRI, PET, PET / CT, or other medical images that can form three-dimensional volumetric data, as long as a tumor ROI mask corresponding to the original image space can be obtained, the method of the present invention can be used for three-dimensional local representation learning, habitat clustering, and heterogeneity scoring.
[0137] (7) Explanation of ROI acquisition methods This invention uses a tumor region of interest (ROI) mask as one of the inputs. This ROI mask can be manually drawn by a physician, automatically generated by existing segmentation software, traditional image segmentation methods, or deep learning segmentation models, or obtained by modifying automatic segmentation results by a physician. The core of this invention is not limited to the specific method of obtaining the ROI, but rather lies in further performing three-dimensional local representation learning, habitat clustering reconstruction, and heterogeneity scoring based on a given ROI.
[0138] (8) Explanation of visualization applications The three-dimensional habitat cluster-labeled volumetric map generated by this invention can be overlaid on the original medical images. Users can observe the spatial distribution of different habitat categories within the tumor in axial, coronal, sagittal, or three-dimensional reconstructed views. The ITHS score can be output together with the habitat-labeled volumetric map, providing a corresponding spatial interpretation basis for the heterogeneity score.
[0139] In summary, through the training process of Example 1, the present invention can train a three-dimensional encoder by utilizing the spatial proximity and long-distance difference relationships of local regions within a tumor ROI, enabling the encoder to obtain local embeddings suitable for habitat clustering.
[0140] Through the single-case reasoning process in Example 2, the present invention can extract three-dimensional local embeddings for each voxel within the tumor ROI and generate a three-dimensional habitat-labeled volume map through clustering, thereby expanding the results of tumor internal heterogeneity from a single numerical value to spatially interpretable volume label results.
[0141] Through the multi-case batch processing process in Example 3, the present invention can automatically perform heterogeneity assessment on multiple cases and output scoring results, intermediate files and error logs in a unified format, thereby improving the efficiency of multi-case analysis and the traceability of results.
[0142] The specific implementation of this invention can fully realize the tumor heterogeneity assessment process from 3D image input, ROI standardization, 3D patch construction, encoder training, embedding extraction, habitat clustering, 3D label reconstruction to ITHScore output, providing an implementable, verifiable and scalable technical solution for tumor intra-tumor heterogeneity analysis.
[0143] Therefore, this invention provides a tumor heterogeneity assessment method based on three-dimensional local representation learning and habitat clustering. This method takes three-dimensional medical images and their tumor area of interest (ROI) masks as input. First, the tumor ROI is 3D-cropped and its grayscale normalized. Then, three-dimensional local patches are constructed centered on voxels within the ROI, and a three-dimensional local encoder is trained using a triplet self-supervised learning method with spatial distance constraints, enabling the encoder to map each three-dimensional local patch to a normalized embedding. During the inference phase, embeddings are extracted from all voxels within the ROI of the case to be analyzed, and these embeddings are clustered to obtain different habitat regions within the tumor. Subsequently, the clustering labels are backfilled into the original three-dimensional ROI space according to voxel coordinates to generate a three-dimensional habitat cluster label volume map. Finally, three-dimensional connected component analysis is performed based on the three-dimensional habitat cluster label volume map to calculate the tumor intra-tumor heterogeneity score (ITHScore).
[0144] Furthermore, the embodiments of this application can also summarize the key points of the tumor heterogeneity assessment device compared with related technologies, mainly involving the following aspects: 1. A voxel-level local representation learning framework based on 3D tumor ROI This invention does not perform heterogeneity analysis based on two-dimensional slices or overall lesion statistical features. Instead, it focuses on the tumor region of interest (ROI) in three-dimensional medical images, extracting three-dimensional local patches centered on each voxel within the ROI and establishing local characterizations for each voxel. This technique can characterize the local structural patterns of different regions within the tumor at the voxel level, providing a foundation for analysis.
[0145] 2. Self-supervised local representation learning mechanism based on 3D spatial priors In the process of 3D local representation learning, this invention preferably introduces a sample construction method based on 3D spatial positional relationships. This allows the model to utilize not only local image content during training, but also the spatial proximity, interlayer continuity, and long-distance differences between different patches within the tumor to constrain the representation space. This design differs from general deep learning representation methods that are only geared towards case-level tasks, and is more suitable for serving tumor internal modeling.
[0146] 3. A 3D habitat reconstruction method based on voxel-level embedding clustering This invention does not simply output a single classification result or an overall heterogeneity statistic. Instead, it clusters the embeddings corresponding to each voxel within the ROI and backfills the cluster labels into the original 3D space to generate a 3D habitat cluster label volumetric map. This volumetric map can directly characterize the distribution of different local habitat regions within the tumor in 3D space, enhancing the interpretability of the heterogeneity assessment results.
[0147] 4. A heterogeneity quantification method combining three-dimensional connected structures This invention, based on a three-dimensional habitat cluster label volume map, further performs three-dimensional connected component analysis on each habitat category and constructs the ITHscore using the number of connected regions, the maximum connected volume, and the degree of spatial fragmentation for each type of region. Compared with heterogeneity indicators based solely on grayscale statistical differences, texture features, or category proportions, this method can more realistically reflect the three-dimensional spatial tissue complexity of tumor internal heterogeneity.
[0148] 5. Integrate 3D local representation learning, habitat reconstruction, and spatial connectivity analysis into a unified heterogeneity assessment process. This invention organically integrates 3D image preprocessing, voxel center patch construction, local representation learning, voxel clustering, habitat volume map generation, and 3D connected domain heterogeneity scoring to form a complete 3D tumor intra-tumor heterogeneity assessment process. This overall technical approach differs from traditional single feature extraction or single statistical modeling methods, enabling a more complete, finer-grained, and more interpretable assessment of tumor heterogeneity.
[0149] Furthermore, the embodiments of this application preprocess the tumor ROI based on three-dimensional medical images and corresponding tumor ROI masks, construct three-dimensional local patches with voxel centers within the ROI, obtain voxel-level embeddings through local representation learning, cluster the embeddings to generate a three-dimensional habitat cluster label volume map, and perform three-dimensional connected component analysis and heterogeneity score calculation on this basis.
[0150] This application's embodiments establish voxel-level local characterization within a tumor region of interest (ROI), and then backfill this characterization into a three-dimensional space by clustering the voxel-level embeddings to generate a three-dimensional habitat cluster label volumetric map. This part is a key protective feature that distinguishes this invention from methods that only output overall statistics or single classification results.
[0151] This application embodiment describes a method for calculating the number of three-dimensional connected regions, the maximum connected volume, and the degree of spatial fragmentation for each habitat category in a three-dimensional habitat cluster label volume map, and constructing a tumor intratumoral heterogeneity score accordingly.
[0152] This application demonstrates the use of three-dimensional habitat cluster-labeled volumetric maps and heterogeneity scoring results in scenarios such as quantitative assessment of tumor intratumoral heterogeneity, invasiveness analysis, risk stratification, prognostic assessment, efficacy prediction, and decision support.
[0153] Those skilled in the art should understand that some functions of the embodiments of this application are not limited to the examples given above, and also include the following alternatives: 1. Alternatives to the 3D encoder structure In addition to lightweight 3D convolutional networks, the 3D encoder can also employ other network structures capable of extracting features from 3D local patches, such as deeper 3D convolutional networks, residual networks, 3D Transformer networks, or hybrid network structures combining convolution and Transformer.
[0154] 2. Alternatives to self-supervised training methods In addition to the preferred sample construction and training methods, other unsupervised or self-supervised training methods suitable for three-dimensional local representation learning can also be used, such as contrastive learning, self-distillation learning, representation learning methods based on positive and negative sample matching, or other training methods that can establish a local patch representation space.
[0155] 3. Alternatives to local patch parameters The size of the local patch is not limited to 9×9×9; it can be adjusted to other suitable sizes based on the spatial resolution of the medical image, the size of the tumor, and the needs of the target task. Boundary processing methods are not limited to reflection padding; duplicate padding, zero padding, or other boundary expansion methods can also be used.
[0156] 4. Alternatives to the embedding parameter The dimensions of embedding are not limited to 32, 64, or 128; they can be set to other dimensions according to training efficiency, task complexity, and clustering needs. The normalization method of embedding is not limited to L2 normalization; other normalization methods suitable for feature space constraints can also be used.
[0157] 5. Alternatives to clustering methods In addition to KMeans clustering, Gaussian mixture model clustering, hierarchical clustering, spectral clustering, density clustering, or other methods capable of classifying voxel-level embeddings can also be used. The number of clusters is not limited to a fixed value and can be adjusted based on data characteristics, tumor type, or analysis task.
[0158] 6. Alternatives to three-dimensional connected component analysis In addition to the preferred 26-adjacency approach, 6-adjacency, 18-adjacency, or other adjacency definitions suitable for three-dimensional connectivity analysis can also be used. Connectivity statistics, besides the number of connected regions and the maximum connected volume, can further include average connected volume, connected volume distribution, volume percentage, or other parameters that reflect the degree of spatial fragmentation.
[0159] 7. Alternatives to heterogeneity scoring methods The specific formula for constructing ITHscore can be adjusted according to different task requirements. In addition to the preferred formula, heterogeneity scoring indicators can also be constructed by linearly combining, nonlinearly combining, or weighted combining parameters such as the number of connected components, the proportion of the largest connected component, the volume distribution of various regions, and the degree of spatial fragmentation. As long as it can reflect the tissue complexity and heterogeneity of the tumor's internal habitat in three-dimensional space, it can be regarded as an equivalent alternative to the present invention.
[0160] To address the shortcomings of existing technologies in assessing tumor heterogeneity, this invention proposes a tumor heterogeneity assessment method based on 3D local representation learning and habitat clustering. This method takes 3D medical images and tumor region-of-interest (ROI) masks as input. It constructs voxel-centered 3D local patches within the tumor ROI, trains a 3D encoder using a self-supervised triplet learning approach with spatial distance constraints, and obtains voxel-level local embeddings within the ROI. Furthermore, the embeddings are clustered and backfilled into the original 3D ROI space to generate a 3D habitat cluster label volumetric map. Finally, a tumor heterogeneity score is calculated based on 3D connected component analysis.
[0161] The present invention mainly addresses the following problems.
[0162] (1) Solve the problem that existing methods cannot fully utilize the three-dimensional spatial continuity of tumors. Heterogeneity analysis methods in related technologies often rely on two-dimensional slices, local two-dimensional regions, or overall statistical features of lesions, easily neglecting the continuous structural information of tumors in the inter-slice directions. This invention directly processes tumor regions of interest (ROIs) in three-dimensional medical images and extracts three-dimensional local patches centered on voxels within the ROI. This ensures that each local analysis unit contains neighborhood information in the axial, coronal, and sagittal directions, thereby enabling the characterization of tumor internal structural features at the three-dimensional volume level.
[0163] (2) Solve the problem that related technologies are difficult to establish voxel-level local characterization within ROI. Existing deep learning methods are mostly geared towards case-level, slice-level, or lesion-level overall tasks, making it difficult to directly obtain the local representation corresponding to each voxel within a Region of Interest (ROI). This invention maps the center patch of each voxel to a low-dimensional embedding through a 3D encoder, so that each voxel within the tumor ROI corresponds to a local feature vector, thereby providing a representational basis for subsequent voxel-level habitat clustering and 3D heterogeneity analysis.
[0164] (3) Solve the problems of insufficient stability of traditional handmade features and strong dependence on manual feature design. Traditional radiomics or texture analysis methods typically rely on manually designed features, which are easily affected by scanning parameters, grayscale distribution, image noise, and feature extraction parameters. This invention performs grayscale normalization within the tumor region of interest (ROI) and employs a 3D encoder to automatically learn the depth representation of local patches, thereby reducing reliance on manual feature design and improving the comparability and stability of local representations across different cases.
[0165] (4) Addressing the issue that self-supervised or deep representation learning methods do not fully utilize the internal spatial relationships of tumor ROIs. Existing self-supervised or contrastive learning methods often rely on random augmentation, random pruning, or case-level label relationships, failing to fully utilize the spatial proximity and distance differences between voxels within the tumor. This invention introduces spatial distance constraints in the construction of triplet samples: using a voxel center patch within the ROI as the anchor, spatially proximate patches within the same case are considered positive samples, and patches from different cases or within the same case whose z-direction distance from the anchor exceeds a preset threshold are considered candidate negative samples. Effective negative samples are selected for training through online semi-difficult negative sample mining. This approach enables the 3D encoder to learn local embeddings more suitable for distinguishing the tumor's internal habitat.
[0166] (5) Solve the problem that it is difficult to reconstruct three-dimensional voxel-level habitat label maps in habitat analysis of related technologies. Related habitat analysis methods only output category proportions, local statistical results, or two-dimensional partition maps, making it difficult to generate a three-dimensional habitat label map consistent with the original tumor ROI spatial coordinates. This invention clusters the embeddings of all voxels within the ROI during the inference phase and backfills the cluster labels to the original three-dimensional ROI space according to the corresponding voxel coordinates, generating a three-dimensional habitat cluster label volume map. This allows for a direct reflection of the spatial distribution of different habitat regions within the tumor.
[0167] (6) Addressing the issue that related technologies have relatively few heterogeneous scoring considerations regarding three-dimensional connectivity and spatial fragmentation. Existing heterogeneity scoring techniques often focus on category proportions, texture differences, or feature distribution differences, failing to reflect whether the same habitat category is continuously and centrally distributed or fragmented in three-dimensional space. This invention, based on a three-dimensional habitat cluster label volume map, performs three-dimensional connected component analysis on each habitat category, statistically analyzing the number of connected regions, the largest connected volume, and the total tumor volume for each category, and calculates the ITHS score accordingly. This allows the heterogeneity score to reflect the complexity and fragmentation of the tumor's internal spatial structure.
[0168] (7) Addressing the problem of insufficient interpretability of heterogeneous results Existing methods often only output a single score or classification result, making it difficult for doctors or researchers to understand the specific spatial location and local structural differences corresponding to the score. This invention not only outputs a heterogeneity score but also a three-dimensional habitat cluster label volumetric map. This volumetric map can be overlaid with the original image or ROI mask, thereby intuitively displaying the spatial distribution of different habitat regions within the tumor and improving the interpretability of heterogeneity assessment results.
[0169] Compared to related technologies, this invention, by employing techniques such as 3D normalization within the ROI, 3D patch construction at voxel centers, self-supervised representation learning of triples with spatial distance constraints, voxel-level embedding clustering, 3D habitat cluster label volume graph backfilling, and 3D connected domain ITHScore calculation, can directly produce the following beneficial effects: (1) Improve the ability to utilize the three-dimensional spatial structure information of tumors. This invention uses three-dimensional medical images and tumor regions of interest (ROIs) as processing objects, extracting three-dimensional local patches centered on voxels within the ROI. Each patch contains not only local grayscale and texture information within a single slice, but also spatial context information in the inter-layer direction. Therefore, compared to heterogeneity analysis methods based on two-dimensional slices or single-layer local regions, this invention can more completely utilize the three-dimensional spatial continuity of the tumor, improving the ability of heterogeneity assessment to express the true spatial structure of the tumor.
[0170] (2) Improve the ability to fine-grained characterize the differences in the tumor microenvironment. This invention generates corresponding embeddings for each voxel within a Region of Interest (ROI) using a 3D encoder, refining the heterogeneity analysis object from the overall lesion or two-dimensional region to a voxel-level local region. Since each voxel has a corresponding local representation, subsequent clustering can segment different habitat regions within the tumor at a finer granular scale, thereby improving the ability to characterize the differences in the tumor's internal microenvironment.
[0171] (3) Improve the stability and comparability of local representations This invention calculates the mean and standard deviation of grayscale within the tumor region of interest (ROI) and performs z-score normalization and extreme value truncation on local images, which can reduce the impact of different cases, different scanning conditions, and different grayscale distributions on subsequent representation learning. Simultaneously, the 3D encoder automatically learns the deep representation of local patches, reducing reliance on manually designed radiomics features, thereby improving the stability of local representations and the comparability between different cases.
[0172] (4) Improve the ability of the three-dimensional encoder to characterize the spatial differences within the tumor. This invention introduces a triplet sample construction method with spatial distance constraints during encoder training. This method tends to make spatially adjacent local patches closer together in the embedding space, and to separate distant patches across cases or within the same case. Furthermore, by selecting more valuable negative samples through online semi-difficult negative sample mining, the encoder's ability to distinguish subtle local differences can be enhanced, making the obtained embeddings more suitable for subsequent habitat clustering.
[0173] (5) Improve the three-dimensional interpretability of habitat clustering results This invention does not merely output the cluster category ratio or overall score, but rather backfills the voxel-level embedding clustering results within the ROI to the original 3D ROI space, generating a 3D habitat cluster label volumetric map. This label volumetric map can directly reflect the spatial location, continuity, division, and fusion relationships of different habitat categories within the tumor, and can be used for subsequent 3D display, hierarchical observation, or clinical interpretation and analysis.
[0174] (6) Improve the ability of heterogeneity scores to reflect the complexity of spatial organization. This invention performs three-dimensional connected component analysis based on a three-dimensional habitat-labeled volume map, and calculates the ITHS score by combining the number of connected regions, the largest connected volume, and the total tumor volume for each habitat category. Therefore, this score not only reflects the presence of different habitat categories within the tumor, but also further reflects the connectivity and fragmentation degree of the same habitat category in three-dimensional space. Compared to heterogeneity indicators based solely on category proportion or statistical distribution, this invention can more fully reflect the spatial organizational complexity within the tumor.
[0175] (7) Enhance the visualization and interpretability of heterogeneity assessment results. The three-dimensional habitat cluster label volumetric map output by this invention can be overlaid with the original medical image and tumor ROI mask, allowing users to directly observe the distribution of different habitat regions in the image space. Simultaneously, the ITHS score can be output together with the three-dimensional habitat label map, providing a corresponding spatial interpretation basis for the heterogeneity score, thereby enhancing the visualization and clinical interpretability of the heterogeneity assessment results.
[0176] (8) Reduce reliance on large-scale, detailed manual annotation. This invention employs a self-supervised triplet learning approach based on spatial relationships to train a 3D encoder. The training process primarily utilizes the spatial proximity and long-distance difference relationships of local patches within the tumor region of interest (ROI) to construct positive and negative samples, eliminating the need for pre-labeling specific habitat categories for each voxel. Therefore, compared to supervised learning methods that rely on large-scale, fine-grained region annotation, this invention reduces manual annotation costs and improves the applicability of the method under different data conditions.
[0177] According to the tumor heterogeneity assessment device proposed in this application, the embodiment acquires the image to be assessed and its tumor ROI mask through an acquisition module; the processing module extracts three-dimensional local image patches from the image to be assessed based on the tumor ROI mask, maps the three-dimensional local image patches to feature embedding vectors, and generates a three-dimensional habitat cluster label volume map based on the feature embedding vectors; the assessment module determines the number of three-dimensional connected components and the maximum connected component volume for each habitat category based on the three-dimensional habitat cluster label volume map, calculates a heterogeneity score based on the number of three-dimensional connected components and the maximum connected component volume for each habitat category, and obtains the tumor heterogeneity assessment result based on the heterogeneity score and / or the three-dimensional habitat cluster label volume map. This solves the problems of lacking three-dimensional spatial structure information, voxel-level fine analysis, and interpretable quantitative indicators in tumor heterogeneity assessment.
[0178] Next, referring to the accompanying drawings, the tumor heterogeneity assessment method proposed according to the embodiments of this application is described, using the tumor heterogeneity assessment device as shown above.
[0179] Figure 2 This is a flowchart of a tumor heterogeneity assessment method according to an embodiment of this application.
[0180] In step S201, the image to be evaluated and the tumor ROI mask of the image to be evaluated are obtained.
[0181] In step S202, based on the tumor ROI mask, three-dimensional local image patches are extracted from the image to be evaluated, and the three-dimensional local image patches are mapped to feature embedding vectors. A three-dimensional habitat cluster label volume map is generated based on the feature embedding vectors.
[0182] In step S203, based on the three-dimensional habitat cluster label volume map, the number of three-dimensional connected domains and the maximum connected domain volume of each habitat category are determined, and a heterogeneity score is calculated based on the number of three-dimensional connected domains and the maximum connected domain volume of each habitat category. The tumor heterogeneity assessment result is obtained based on the heterogeneity score and / or the three-dimensional habitat cluster label volume map.
[0183] Optionally, in some embodiments, based on a tumor ROI mask, three-dimensional local image patches are extracted from the image to be evaluated, and the three-dimensional local image patches are mapped to feature embedding vectors. A three-dimensional habitat cluster label volume map is generated based on the feature embedding vectors. This includes: preprocessing the image to be evaluated based on the tumor ROI mask to obtain a preprocessed three-dimensional image; extracting three-dimensional local image patches from the preprocessed three-dimensional image, using voxels within the tumor ROI mask as target center points, extracting three-dimensional local image patches of a preset size from the preprocessed three-dimensional image, and inputting the three-dimensional local image patches into a trained three-dimensional encoder to obtain feature embedding vectors corresponding to the target center points; clustering the feature embedding vectors to determine the habitat category label for each voxel, and generating a three-dimensional habitat cluster label volume map based on the habitat category label.
[0184] It should be noted that the foregoing explanation of the tumor heterogeneity assessment device embodiment also applies to the tumor heterogeneity assessment method of this embodiment, and will not be repeated here.
[0185] According to the tumor heterogeneity assessment method proposed in this application, the method acquires the image to be assessed and its tumor ROI mask. Based on the tumor ROI mask, three-dimensional local image patches are extracted from the image to be assessed, and these patches are mapped to feature embedding vectors. A three-dimensional habitat cluster label volume map is generated based on the feature embedding vectors. Based on the three-dimensional habitat cluster label volume map, the number of three-dimensional connected components and the maximum connected component volume for each habitat category are determined. A heterogeneity score is calculated based on the number of three-dimensional connected components and the maximum connected component volume for each habitat category. The tumor heterogeneity assessment result is obtained based on the heterogeneity score and / or the three-dimensional habitat cluster label volume map. This solves the problems of lacking three-dimensional spatial structure information, voxel-level fine analysis, and interpretable quantitative indicators in tumor heterogeneity assessment.
[0186] This application also provides an electronic device for implementing the tumor heterogeneity assessment device as described above when executed.
[0187] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0188] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0189] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.
[0190] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0191] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.
Claims
1. A tumor heterogeneity assessment device, characterized in that, Includes the following steps: The acquisition module is used to acquire the image to be evaluated and the tumor ROI mask of the image to be evaluated; The processing module is used to extract three-dimensional local image patches from the image to be evaluated based on the tumor ROI mask, map the three-dimensional local image patches into feature embedding vectors, and generate a three-dimensional habitat cluster label volume map based on the feature embedding vectors. The evaluation module is used to determine the number of three-dimensional connected domains and the maximum connected domain volume for each habitat category based on the three-dimensional habitat cluster label volume map, calculate the heterogeneity score based on the number of three-dimensional connected domains and the maximum connected domain volume for each habitat category, and obtain the tumor heterogeneity assessment result based on the heterogeneity score and / or the three-dimensional habitat cluster label volume map.
2. The apparatus according to claim 1, characterized in that, The processing module includes: The preprocessing unit is used to preprocess the image to be evaluated based on the tumor ROI mask to obtain a preprocessed three-dimensional image. The mapping unit is used to extract the three-dimensional local image patch from the preprocessed three-dimensional image, using the voxel in the tumor ROI mask as the target center point, extracting the three-dimensional local image patch of a preset size from the preprocessed three-dimensional image, and inputting the three-dimensional local image patch into the trained three-dimensional encoder to obtain the feature embedding vector corresponding to the target center point; The generation unit is used to cluster the feature embedding vectors, determine the habitat category label of each voxel, and generate a three-dimensional habitat cluster label volume map based on the habitat category label.
3. The apparatus according to claim 2, characterized in that, The generation unit includes: The backfill subunit is used to backfill the habitat category labels into the ROI space of the image to be evaluated based on the original three-dimensional coordinates, so as to generate the three-dimensional habitat cluster label volume map.
4. The apparatus according to claim 2, characterized in that, The preprocessing unit includes: A sub-unit is defined to determine the minimum bounding box of the tumor region in three-dimensional space based on the coordinates of the tumor voxels in the tumor ROI mask. The cropping subunit is used to expand a preset number of voxels outward in the first preset direction, the second preset direction and the third preset direction of the minimum bounding box, respectively, and to simultaneously crop the image to be evaluated and the tumor ROI mask based on the expanded bounding box, so as to obtain the cropped image to be evaluated and the cropped tumor ROI mask. The standardization subunit is used to perform grayscale standardization on the cropped image to be evaluated within the tumor ROI range corresponding to the cropped tumor ROI mask, so as to obtain the preprocessed three-dimensional image.
5. The apparatus according to claim 1, characterized in that, Also includes: The training module, in which, The training module is specifically used to: construct an initial 3D encoder and a training dataset; Multiple training cases are selected from the training dataset, and voxels are sampled within the tumor ROI of the training cases. Three-dimensional local image patches for training are extracted with each voxel as the center. Multiple triplet training samples are determined based on the three-dimensional local image patches used for training, wherein each triplet training sample includes an anchor sample, a positive sample, and a negative sample; The training 3D local image patches corresponding to the anchor point samples, the positive samples, and the negative samples are input into the initial 3D encoder to obtain the feature embedding vectors corresponding to the triplet training samples; The loss value is calculated based on the preset triplet loss function and the feature embedding vector corresponding to the triplet training sample, and the parameters of the initial 3D encoder are updated according to the loss value until the preset qualification conditions are met, thus obtaining the trained 3D encoder.
6. The apparatus according to claim 5, characterized in that, Each of the triplet training samples includes an anchor sample, a positive sample, and a negative sample, wherein, The anchor point sample is a training three-dimensional local image block corresponding to any voxel within the tumor ROI of any training case. The positive sample is a training three-dimensional local image patch corresponding to a voxel that belongs to the same training case as the anchor sample and whose three-dimensional spatial distance does not exceed the first preset threshold. The negative samples include cross-case negative samples and long-distance negative samples within the same case. The cross-case negative samples are training 3D local image patches corresponding to voxels within the ROI of other cases. The long-distance negative samples within the same case are training 3D local image patches corresponding to voxels that belong to the same training case as the anchor sample and whose distance in the third preset direction is not less than the second preset threshold.
7. The apparatus according to claim 1, characterized in that, The evaluation module includes: The calculation unit is used to calculate a heterogeneity score based on the number of three-dimensional connected components and the maximum connected component volume for each habitat category, wherein the formula for calculating the heterogeneity score is: ; in, The heterogeneity score is given. The total prime number of the tumor ROI. For the number of habitat categories, For the first The largest connected component volume among all habitat categories For the first The number of three-dimensional connected domains for each habitat category.
8. A method for assessing tumor heterogeneity, employing the tumor heterogeneity assessment device as described in any one of claims 1-7, characterized in that, Includes the following steps: Obtain the image to be evaluated and the tumor ROI mask of the image to be evaluated; Based on the tumor ROI mask, three-dimensional local image patches are extracted from the image to be evaluated, and the three-dimensional local image patches are mapped into feature embedding vectors. A three-dimensional habitat cluster label volume map is generated based on the feature embedding vectors. Based on the three-dimensional habitat cluster label volume map, the number of three-dimensional connected domains and the maximum connected domain volume of each habitat category are determined, and a heterogeneity score is calculated based on the number of three-dimensional connected domains and the maximum connected domain volume of each habitat category. The tumor heterogeneity assessment result is obtained based on the heterogeneity score and / or the three-dimensional habitat cluster label volume map.
9. The method according to claim 8, characterized in that, The step of extracting three-dimensional local image patches from the image to be evaluated based on the tumor ROI mask, mapping the three-dimensional local image patches to feature embedding vectors, and generating a three-dimensional habitat cluster label volume map based on the feature embedding vectors includes: Based on the tumor ROI mask, the image to be evaluated is preprocessed to obtain a preprocessed three-dimensional image; The three-dimensional local image patch is extracted from the preprocessed three-dimensional image. The voxel in the tumor ROI mask is used as the target center point. The three-dimensional local image patch of a preset size is extracted from the preprocessed three-dimensional image. The three-dimensional local image patch is input into the trained three-dimensional encoder to obtain the feature embedding vector corresponding to the target center point. Cluster the feature embedding vectors to determine the habitat category label for each voxel, and generate a three-dimensional habitat cluster label volume map based on the habitat category label.
10. An electronic device, characterized in that, include: The tumor heterogeneity assessment device as described in any one of claims 1-7.
