A method for characterizing tumor heterogeneity based on medical images
Through the tumor heterogeneity characterization method based on medical imaging, the combination of mean and variance of feature maps is used to solve the limitations of tumor heterogeneity characterization in the prior art, and more accurate prediction of cervical cancer treatment response is achieved, supporting personalized treatment.
Patent Information
- Application Number
- CN202410374933.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-29
- Publication Date
- 2025-07-08
- Estimated Expiration
- 2044-03-29
AI Technical Summary
Existing tumor heterogeneity characterization methods have limitations in predicting cervical cancer treatment response, ignoring the interpretability of model decisions and the complexity of tumor behavior, resulting in poor treatment results.
Using a tumor heterogeneity characterization method based on medical imaging, the superpixel segmentation module, high-resolution characterization module and tumor heterogeneity characterization module are used to characterize tumor heterogeneity by using the combination of mean and variance of feature maps to avoid the limitations of traditional convolution and pooling operations.
It improves the accuracy and interpretability of tumor heterogeneity analysis, can predict the treatment response of cervical cancer more accurately, and supports personalized treatment planning.
Smart Images
Figure CN118172343B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of disease heterogeneity characterization, and particularly relates to a method for tumor heterogeneity characterization based on medical images. Background Art
[0002] Cervical cancer is a major global health problem, with its incidence and mortality ranking fourth among female diseases. Notably, developing countries account for a large proportion of the death cases. Despite the progress of treatment methods, such as external beam radiotherapy (EBRT) combined with chemotherapy, the 5-year overall survival rate of locally advanced cervical cancer (LACC) remains unsatisfactory. The survival rates of the International Federation of Gynecology and Obstetrics (FIGO) stage II, III, and IVA are only 65%, 40%, and 15% respectively. Studies have shown that up to one-third of cervical cancer patients have a high risk of recurrence within 18 months after treatment. These statistics highlight the urgent need for more accurate and real-time efficacy prediction models to evaluate treatment response and facilitate personalized treatment planning.
[0003] Current methods for predicting treatment response mainly focus on clinical parameters, such as tumor stage and patient characteristics. However, these methods may sometimes overlook the various complexities of tumor behavior, which may lead to unsatisfactory results. In contrast, leveraging the potential of medical imaging provides a more comprehensive and personalized perspective, integrating anatomical and functional insights. Currently, the methods used in medical image analysis for predicting treatment response mainly involve manually extracting features related to treatment response in images or directly extracting and classifying features using end-to-end deep neural networks. However, these methods often neglect the need for interpretable analysis of model decisions and fail to gain in-depth understanding of the reasons behind treatment response prediction.
[0004] Clinical studies have confirmed that the treatment response of cervical cancer radiotherapy is strongly correlated with tumor heterogeneity. Generally, the higher the degree of heterogeneity, the worse the treatment effect, while homogeneous tumors often show good results. Existing methods often use traditional convolutional kernel pooling operations to characterize tumor heterogeneity, which has certain limitations. Summary of the Invention
[0005] To solve the above problems, the present invention provides a method for tumor heterogeneity characterization based on medical images.
[0006] To achieve the above object, the present invention provides the following technical solutions:
[0007] A method for tumor heterogeneity characterization based on medical images, characterized by comprising the following steps:
[0008] Extract the tumor region in the medical images of cervical cancer patients before radiotherapy;
[0009] Input the tumor region in the medical image into a network for characterizing tumor heterogeneity, and output the heterogeneity characterization result; the network for characterizing tumor heterogeneity includes a superpixel segmentation module, a high-resolution characterization module, and a tumor heterogeneity characterization module; the process of inputting the tumor region in the medical image into the network for characterizing tumor heterogeneity and outputting the heterogeneity characterization result specifically includes:
[0010] Input the tumor region in the medical image into the superpixel segmentation module, and segment the tumor region into multiple sub-regions with similar attributes;
[0011] Extract the eigenvalue of multiple sub-regions through the high-resolution characterization module;
[0012] Calculate the average value and standard deviation of the eigenvalues of multiple sub-regions through the tumor heterogeneity characterization module, and use the weighted combination of the average value and the standard deviation as the sub-region descriptor, and characterize the heterogeneity of the tumor through the sub-region descriptor.
[0013] Preferably, the high-resolution characterization module includes an initial convolutional layer and multiple dilated convolutional layers. Each dilated convolutional layer contains multiple residual blocks. The initial convolutional layer can extract preliminary low-level features from the sub-region, and the low-level features include edges and textures; multiple residual blocks can extract hierarchical features from the sub-region.
[0014] Preferably, the initial convolutional layer is an initial convolutional layer with a standard 3x3 kernel and a stride of 1; the dilated convolutional layer also includes normalization and non-linear elements.
[0015] Preferably, the superpixel segmentation module uses the SLIC clustering method to segment the tumor region in the medical image into multiple sub-regions with similar attributes. The SLIC clustering method runs by minimizing a distance metric that takes into account the gray intensity similarity between pixels in the image. The formula of this distance metric is as follows:
[0016]
[0017] where, ΔI is the intensity difference, Δx and Δy represent spatial differences, and K I 、K x and K y are all scale factors;
[0018] By minimizing the distance metric, the SLIC clustering method groups pixels into superpixels with similar gray intensities, and realizes the segmentation of the gray-scale medical image.
[0019] Preferably, the pre-radiotherapy cervical medical image is a pre-radiotherapy cervical MRI image, and a manually delineated mask is used to extract the tumor region in the MRI image.
[0020] Preferably, the heterogeneity includes pathological complete response (pCR) and pathological non-complete response (Non-pCR) of the tumor.
[0021] The method for characterizing tumor heterogeneity based on medical images provided by the present invention has the following beneficial effects:
[0022] The present invention first extracts the tumor region from the medical image of cervical cancer after radiotherapy, and divides the tumor region into multiple sub-regions with similar attributes, and can use the differences between these sub-regions to analyze tumor heterogeneity; then uses a high-resolution characterization module to extract features for each sub-region to obtain the feature values of each sub-region; calculates the average value and standard deviation of the feature values of multiple sub-regions through the tumor heterogeneity characterization module, and uses the weighted combination of the average value and standard deviation as the sub-region descriptor, and characterizes the heterogeneity of the tumor through the sub-region descriptor, and uses the combination of the mean and variance of the feature map to characterize the heterogeneity of the tumor, avoiding the limitations brought by traditional convolution and pooling operations. Description of the Drawings
[0023] In order to more clearly illustrate the embodiments of the present invention and their design solutions, the accompanying drawings required for this embodiment will be briefly introduced below. The accompanying drawings in the following description are only partial embodiments of the present invention. For those of ordinary skill in the art, other accompanying drawings can be obtained based on these drawings without creative work.
[0024] Figure 1 It is a flowchart of the method for characterizing tumor heterogeneity based on medical images according to Embodiment 1 of the present invention;
[0025] Figure 2 It is a detailed diagram of the method for characterizing tumor heterogeneity based on medical images according to Embodiment 1 of the present invention. Detailed Embodiments
[0026] In order to enable those skilled in the art to better understand the technical solutions of the present invention and implement them, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and cannot be used to limit the protection scope of the present invention.
[0027] Embodiment 1
[0028] Based on the prior art, the present invention designs a network capable of characterizing tumor heterogeneity. Through tumor heterogeneity, the treatment response of cervical cancer can be predicted. This network includes several parts such as a superpixel segmentation module, a high-resolution characterization module, and a tumor heterogeneity characterization module. This deep network clarifies the basic principle behind treatment response prediction and visualizes the heterogeneity of the tumor.
[0029] Based on this, the present invention proposes a method for characterizing tumor heterogeneity based on medical images, as Figure 1 and Figure 2 shown, specifically including the following steps:
[0030] Step 1: Extract the tumor region from the medical images of cervical cancer patients before radiotherapy. Among them, the medical images are specifically MRI images.
[0031] Step 2: Input the tumor region in the medical image into the network for characterizing tumor heterogeneity, and output the heterogeneity characterization result. The heterogeneity includes pathological complete response pCR and non-pathological complete response Non-pCR of the tumor. Step 2 specifically includes the following sub-steps:
[0032] Step 21: Input the tumor region in the medical image into the superpixel segmentation module, and segment the tumor region into multiple superpixel segmentation sub-regions with similar attributes.
[0033] Specifically, the superpixel segmentation module is a sub-region clustering module. This module divides the tumor into a predefined number of sub-regions to obtain superpixel segmentation sub-regions, and uses the differences between these sub-regions to analyze tumor heterogeneity.
[0034] The superpixel segmentation module, as an integral part of the network architecture of the present invention, helps to segment the tumor into discrete sub-regions, enabling the present invention to comprehensively analyze tumor heterogeneity using the differences between these sub-regions.
[0035] In order to segment the acquired tumor, superpixel segmentation is required to obtain the segmentation result at the single sub-region level. Specifically, the initial step includes dividing the tumor region into sub-regions with similar attributes. In this embodiment, a clustering method is used to achieve this, namely the SLIC clustering method.
[0036] Using SLIC (Simple Linear Iterative Clustering) superpixel segmentation is a method for dividing an image into superpixels. In the context of medical images, which are usually grayscale images, the SLIC algorithm must be adapted to measure similarity based on grayscale intensity. SLIC runs by minimizing a distance metric that takes into account the grayscale intensity similarity between pixels in the image. The formula for this distance metric is as follows:
[0037]
[0038] where ΔI is the intensity difference, and Δx, Δy are the spatial differences. The scaling factors K I , K x and K y are determined according to the required number of superpixels and the image size.
[0039] By minimizing this distance metric, the improved SLIC effectively groups pixels into superpixels with similar grayscale intensities, thereby allowing meaningful segmentation of grayscale medical images.
[0040] Step 22: Extract high-resolution features of the superpixel segmentation sub-regions through a high-resolution characterization module (i.e., the high-resolution characterization module network in Figure 1 .
[0041] After superpixel segmentation divides pixels into regions with similar features, the present invention needs to construct a feature representation network, namely, a high-resolution characterization module. Since the present invention hopes that the sub-regions will not change too much during the feature extraction process, the present invention needs a network that preserves high-resolution feature maps rather than a typical downsampling classifier. Therefore, the present invention designs a high-resolution characterization module. It avoids typical downsampling methods and instead preserves the spatial resolution through the following components:
[0042] First, the network applies an initial convolutional layer with a standard 3x3 kernel and a stride of 1; this layer extracts preliminary low-level features such as edges and textures from the input while preserving its original resolution.
[0043] Next is a series of dilated convolutional layers, each of which further expands the receptive field coverage.
[0044] The dilation rate gradually increases, enabling the network to aggregate a wider context without losing resolution or density. The expanded receptive field enables the network to integrate multi-scale visual information.
[0045] Each dilated convolution contains multiple residual blocks, where there are shortcut connection convolutional layers. Residual connections allow gradient transmission through hundreds of layers to overcome the problem of vanishing gradients when training deep networks.
[0046] The above components also include batch normalization and non-linear elements. The coordinated architecture enables the construction of models with exceptional capacity and depth while preserving the representative resolution.
[0047] The high-resolution characterization module uses residual learning and dilated convolutions to extract hierarchical features without downsampling. This produces a discriminative high-resolution representation that is crucial for segmentation and classification.
[0048] Step 23: Calculate the mean and standard deviation of the feature values of multiple sub-regions through a tumor heterogeneity characterization module (i.e., the heterogeneity descriptor network in Figure 1 , and use the weighted combination of the mean and standard deviation as the sub-region descriptor to characterize the tumor heterogeneity through the sub-region descriptor.
[0049] The present invention designs a new method for representing tumor heterogeneity through a dedicated deep network. Different from traditional convolution and pooling operations, this network uses a unique combination of the mean and variance of feature maps to effectively characterize tumor heterogeneity.
[0050] After extracting the high-resolution feature representation of the superpixel segmentation sub-regions, the present invention needs to characterize the heterogeneity between the sub-regions to achieve accurate pathological modeling. Traditional methods usually use mean pooling of sub-region features, thus losing important heterogeneity clues. To overcome this problem, the present invention proposes a heterogeneity characterization module to explicitly simulate the differences between sub-regions.
[0051] Specifically, this module calculates the mean and standard deviation of the feature values of each sub-region. The standard deviation summarizes the distribution information about the pixel values of the sub-region and represents heterogeneity. The mean provides a description of the activation pattern of the sub-region. The weighted combination of the mean and standard deviation serves as a sub-region descriptor, replacing the traditional mean pooling descriptor.
[0052] By combining the summary statistics, the heterogeneity characterization module can generate descriptors that simultaneously describe the global sub-region behavior and its heterogeneity. The heterogeneity representation enables the model to identify abnormal sub-regions with high variability in tumors, which is crucial for predicting the radiotherapy effect based on tumor heterogeneity.
[0053] Therefore, the tumor heterogeneity characterization module proposed by the present invention improves the previous method by using the mean and standard deviation to represent each sub-region, rather than just the mean. This enables the high-resolution characterization module to simulate the regional heterogeneity crucial for pathological analysis of cancer tissues, and capturing heterogeneity can improve the detection and segmentation performance.
[0054] The tumor heterogeneity characterization module of the present invention uses a combination of the mean and variance of feature maps to characterize tumor heterogeneity, rather than traditional convolution and pooling operations. This network uses the variance of feature maps to represent the differences between sub-regions. Experimental results on two databases show that this method effectively visualizes the heterogeneity of patient tumors.
[0055] In summary, compared with the existing technologies, the present invention has the following advantages:
[0056] Based on the existing technologies, the present invention designs a network capable of characterizing tumor heterogeneity for predicting the treatment response of cervical cancer. This deep network clarifies the basic principle behind treatment response prediction and visualizes the heterogeneity of tumors.
[0057] The standard deviation provided by the present invention summarizes the distribution information of pixel values in sub-regions and represents heterogeneity. The weighted combination of the mean and the standard deviation serves as a sub-region descriptor, replacing the traditional mean pooling descriptor. Heterogeneity enables the model to identify abnormal sub-regions with high variability in tumors.
[0058] The characterization method proposed by the present invention uses a superpixel segmentation module to divide the tumor into a predefined number of sub-regions, which can more accurately analyze the heterogeneity of the tumor, being more detailed and comprehensive compared with traditional methods. An innovation is the introduction of a heterogeneity characterization module that uses a combination of the mean and variance of feature maps to characterize the heterogeneity of the tumor, avoiding the limitations brought by traditional convolution and pooling operations. The network uses the variance of feature maps to represent the differences between sub-regions, providing a more accurate and meaningful characterization. In the prediction of the efficacy of cervical cancer radiotherapy, the heterogeneity of the tumor can be used as a reference factor, thus having higher reliability in predicting treatment response, providing a more accurate and practical reference for doctors and researchers, and helping to improve personalized treatment planning.
[0059] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.
[0060] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.
[0061] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that realizes the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1The functions specified in one or more boxes.
[0062] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for implementing the steps of the functions specified in one Figure 1 one process or more processes and / or boxes Figure 1 or more boxes.
[0063] It should be noted that the above-described specific embodiments can enable those skilled in the art to understand the present invention more comprehensively, but do not limit the present invention in any way. Therefore, although the present specification and embodiments have described the present invention in detail, those skilled in the art should understand that the present invention can still be modified or equivalently replaced; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered by the protection scope of the patent of the present invention. Any reference signs in the claims should not be construed as limiting the claimed claims.
Claims
1. A method for characterizing tumor heterogeneity based on medical images, characterized in that, Including the following steps: Extracting the tumor region from the medical images of cervical cancer patients before radiotherapy; Inputting the tumor region in the medical image into a network for characterizing tumor heterogeneity, and outputting a heterogeneity characterization result; the network for characterizing tumor heterogeneity includes a superpixel segmentation module, a high-resolution characterization module, and a tumor heterogeneity characterization module; The step of inputting the tumor region in the medical image into the network for characterizing tumor heterogeneity and outputting a heterogeneity characterization result specifically includes: Inputting the tumor region in the medical image into the superpixel segmentation module to segment the tumor region into multiple sub-regions with similar attributes; Extracting the eigenvalue of multiple sub-regions through the high-resolution characterization module; Calculating the average value and standard deviation of the eigenvalues of multiple sub-regions through the tumor heterogeneity characterization module, and taking the weighted combination of the average value and the standard deviation as a sub-region descriptor, and characterizing the heterogeneity of the tumor through the sub-region descriptor; The high-resolution characterization module includes an initial convolutional layer and multiple dilated convolutional layers, each dilated convolutional layer contains multiple residual blocks, the initial convolutional layer can extract preliminary low-level features from the sub-region, and the low-level features include edges and textures; multiple residual blocks can extract hierarchical features from the sub-region; The superpixel segmentation module uses the SLIC clustering method to segment the tumor region in the medical image into multiple sub-regions with similar attributes, and the SLIC clustering method operates by minimizing the distance metric of the gray intensity similarity between pixels in the image, and the formula of this distance metric is as follows: ； Among them, ∆ I is the intensity difference, ∆ x , ∆ y represent spatial differences, K I , K x and K y are all scale factors; By minimizing the distance metric, the SLIC clustering method groups pixels into superpixels with similar gray intensities to achieve the segmentation of the medical image.
2. The method for characterizing tumor heterogeneity based on medical images according to claim 1, wherein The initial convolutional layer is an initial convolutional layer with a standard 3x3 kernel and a stride of 1.
3. The method for characterizing tumor heterogeneity based on medical images according to claim 1, wherein The medical images of cervical cancer patients before radiotherapy are cervical MRI images before radiotherapy, and a manually drawn mask is used to extract the tumor region in the MRI images.
4. The method for characterizing tumor heterogeneity based on medical images according to claim 1, wherein The heterogeneity includes complete pathological response pCR of the tumor and non-complete pathological response Non-pCR of the tumor.