An intestinal mirror image age-related phenotype interpretation method based on a conditional diffusion model

Abstract

The present application relates to the technical field of medical image explainability, and particularly relates to a colonoscopy image age-related phenotype explanation method based on a conditional diffusion model. The method comprises the following steps: screening a representative image from a colonoscopy age prediction model, obtaining a depth map through monocular depth estimation, constructing a diffusion model with double conditions of age and depth and training with noise prediction loss; in the inference stage, fixing the depth map and random seed, changing the age condition to generate the corresponding colonoscopy image, and directly revealing the age-related phenotypes such as mucosal texture and microvessels. The present application can eliminate structural interference, accurately explain the model prediction basis, is compatible with existing models and can quantitatively evaluate the intestinal aging state, and provides an objective and explainable auxiliary means for clinical use.

Description

Technical Field

[0001] This invention relates to the field of medical image interpretability technology, specifically to a method for interpreting age-related phenotypes in colonoscopy images based on a conditional diffusion model. Background Technology

[0002] Intestinal mucosal images typically contain a wealth of visual information, including mucosal texture, microvascular course, vascular density, and surface structure. While this information is not usually used as an independent quantitative indicator in routine clinical image interpretation, it may contain potential characteristics related to the degree of intestinal aging. Existing interpretable methods often employ heatmaps, attention visualization, or occlusion analysis, which can only roughly indicate the areas of interest for the model and are insufficient to directly demonstrate phenotypic changes in colonoscopy images corresponding to different age conditions while maintaining structural consistency.

[0003] Therefore, there is an urgent need to provide a new technical solution that can generate and compare colonoscopy images corresponding to different age conditions under fixed structural constraints based on the colonoscopy image age prediction model, so as to reveal key image phenotypes related to age prediction and improve the model's interpretability and the reliability of the results. Summary of the Invention

[0004] The purpose of this invention is to provide a method for interpreting age-related phenotypes in colonoscopy images based on a conditional diffusion model, in order to solve the problems of unintuitive interpretation methods and difficulty in comparing phenotype differences among different ages under the same structural conditions in existing intelligent analysis of colonoscopy images.

[0005] The objective of this invention can be achieved through the following technical solutions: A method for interpreting age-related phenotypic patterns in colonoscopy images based on a conditional diffusion model, comprising the following steps: Representative images representing age information from each colonoscopy report are selected from the trained colonoscopy age prediction model. Monocular depth estimation is performed on a representative image to obtain the corresponding depth map; Training samples are constructed based on representative images, depth maps, and age labels to train a conditional diffusion model that is jointly constrained by age conditions and depth structure conditions. During the inference phase, a fixed depth map and a random seed are used to generate colonoscopy images corresponding to different ages by changing the age conditions, in order to interpret the key image phenotypes on which the age prediction model is based.

[0006] Furthermore, the representative image is filtered by the attention weights output by the attention head of the age prediction model, and the image with the highest weight is selected as the representative image of the colonoscopy report.

[0007] Furthermore, the monocular depth estimation was performed using the MiDaS3.0 monocular depth estimator.

[0008] Furthermore, the conditional diffusion model employs a conditional image generation framework based on StableDiffusion.

[0009] Furthermore, age labels are used as age-conditional inputs to the backbone network of the diffusion model, while depth maps are used as structural conditions and injected into the diffusion model through ControlNet branches.

[0010] Furthermore, during the inference phase, colonoscopy images corresponding to different ages are generated by fixing the same depth map and random seed while changing the age condition input, so as to compare age-related phenotypic differences under the same structural constraints.

[0011] Furthermore, age-related phenotypes include one or more of the following: mucosal texture, microvascular course, vascular density, and surface structure.

[0012] Furthermore, during the inference stage, the colonoscopy image of the subject is input, the corresponding depth map is extracted and a random seed is fixed, a corresponding reference colonoscopy image is generated under the condition of the subject's actual age, and the aging status of the subject is evaluated based on the difference between the reference colonoscopy image and the real colonoscopy image.

[0013] Compared to existing solutions, the beneficial effects achieved by this invention are: Achieving generative interpretability with more intuitive interpretation results: Abandoning traditional indirect interpretation methods such as heatmaps and attention visualization, it directly generates colonoscopy images corresponding to different ages under fixed structural constraints, presenting the model's prediction basis intuitively with visual images, greatly improving interpretability and clinical readability.

[0014] Fixing the intestinal structure and eliminating structural interference: Structural constraints are introduced through monocular depth maps, and the generation process maintains the consistency of the overall morphology and spatial structure of the intestinal mucosa, showing only age-related phenotypic changes and avoiding structural differences from interfering with age characteristic analysis.

[0015] Precisely reveals age-related potential visual features: It can clearly observe the regular changes in mucosal texture, microvascular course, vascular density, and surface structure with age, clarify the effective biological features learned by the model, and verify the rationality of the prediction model. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0017] Figure 1This is a flowchart illustrating a method for interpreting age-related phenotypes in colonoscopy images based on a conditional diffusion model, according to an embodiment of the present invention. Figure 2 This is a schematic diagram illustrating the interpretation results of age-related image phenotypes based on a diffusion model according to an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more exemplary embodiments. Numerous specific details are provided in the following description to give a full understanding of exemplary embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, steps, etc., can be employed. In other instances, well-known structures, methods, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0020] This embodiment provides a method for interpreting age-related phenotypic patterns in colonoscopy images based on a conditional diffusion model. Figure 1 This is a flowchart illustrating the workflow of a method for interpreting age-related phenotypes in colonoscopy images based on a conditional diffusion model, according to an embodiment of the present invention. Figure 1 As shown, the method includes: Step 1: Select the representative image that best represents age information from each colonoscopy report from the trained colonoscopy age prediction model; First, it is necessary to select the image that best represents the age information of the examinee from each colonoscopy report from a large-scale, noisy dataset of colonoscopy reports. Specifically, this invention randomly selects images from the valid image set corresponding to each colonoscopy report. Zhang, forming an image bag. It employs an attention-based multiple instance learning (ABMIL) model. For image packets... For each colonoscopy image, an EfficientNetV2-S network pre-trained on ImageNet and fine-tuned on over 100,000 colonoscopy reports was used as a feature extractor to map the image into a corresponding feature vector. : ; in, For parameters The EfficientNet feature extractor, For image packs The first in Zhang Image , For feature dimensions.

[0021] To obtain the abundance of age information contained in each image, this implementation further employs an attention head to assign different weights to each image feature. For the i-th... Feature vector Their attention score It can be represented as: ; in, The attention scoring function, implemented using a multilayer perceptron with a linear layer of 256 neurons, a Tanh non-linear activation function, and a fully connected layer, followed by softmax, is used to calculate the importance score of the image frame based on the feature vector. Indicates the first The normalized attention weights of the images in the current image bag, satisfying: ; Assuming that for each colonoscopy report, there is a corresponding image package The attention weights corresponding to each image output by the attention head are: The image with the highest weight is selected as the representative image for this report, i.e.: ; in, This represents the most representative image from the colonoscopy report that best reflects age-related information. Using this method, one representative image can be extracted from each report in the training set, thus constructing a training sample set consisting of the representative image and its corresponding age label.

[0022] Step 2: Perform monocular depth estimation on the representative image to obtain the corresponding depth map; To preserve the spatial structure information of the intestinal image, this invention uses the MiDaS3.0 monocular depth estimator to estimate the depth of the representative image, obtaining the corresponding depth map. Therefore, each training sample can be represented as a triple. ,in Represents colonoscopy images, represent The corresponding depth map, This represents the corresponding age tag.

[0023] Step 3: Construct training samples based on representative images, depth maps, and age labels, and train a conditional diffusion model that is jointly constrained by age conditions and depth structure conditions. In terms of model construction, this invention adopts a conditional image generation framework based on StableDiffusion, using age labels... The distribution of colonoscopy images is modeled using conditional information. For real images Its forward diffusion process is achieved by gradually adding Gaussian noise to the image, which can be represented as: ; ; in, Indicates the first The noise image corresponding to each time step. Indicates the parameters for scheduling diffused noise. Represents the identity matrix. , This is the cumulative coefficient for the diffusion process.

[0024] Simultaneously, to introduce structural conditions into the generation process, this invention uses a ControlNet branch to inject structural information into StableDiffusion. During the reverse generation process, a noise prediction network is constructed. Based on age and depth conditions Using this as input, the noise at the current time step is predicted, and the target colonoscopy image is generated by progressively denoising. The conditional noise prediction process can be represented as follows: ; in, Indicates the current diffusion time step, age tag After embedding encoding, the data is input into the StableDiffusion backbone network to guide the generation of intestinal image distributions that correspond to the target age; depth map The control network is then injected into the generative network through the control network branch to constrain the spatial structure and morphological features of the generated images, so that the generated images can reflect the differences in image phenotypes under different age conditions while maintaining the rationality of the intestinal structure.

[0025] It is worth noting that during model training, t steps of noise are added to the forward direction of the images. This is the original image. Added a noise step. By adding t-step noise, gradually transforming clear images into pure noise, the model learns how to... Denoising images containing noise During the inference (generation) process, the model performs n steps of reverse denoising from pure noise to obtain a clean image.

[0026] The model training objective is to minimize the true noise. With prediction noise The difference between them can be expressed by the loss function as: ; By optimizing the above loss function, a colonoscopy image generation model under the joint constraints of age and depth conditions can be obtained, where, The sum of squares of the L2 norm and the vector elements is used to calculate the mean square error (MSE) between the actual noise and the predicted noise.

[0027] Step four involves fixing the depth map and random seed during the inference phase, and changing the age conditions to generate colonoscopy images corresponding to different ages, in order to interpret the key image phenotypes on which the age prediction model is based.

[0028] After the model is trained, colonoscopy images corresponding to different ages can be generated by changing the age condition while maintaining a fixed depth structure, thereby observing the differences in image phenotypes caused by age changes under the same structural constraints. In this way, the present invention can analyze key visual features related to age prediction and achieve generative interpretation of the aforementioned age prediction model.

[0029] Figure 2 This is a schematic diagram illustrating the interpretation results of age-related image phenotypes based on a diffusion model according to an embodiment of the present invention, as shown below. Figure 2 As shown, under the condition of fixing the same depth map and the same random seed, colonoscopy images are generated by inputting different age conditions. The age conditions are 20 years old, 30 years old, 40 years old, 50 years old, 60 years old, 70 years old, 80 years old and 90 years old. In the figure, Age represents age. This figure is used to illustrate that under the premise of maintaining the basic consistency of the overall intestinal structure, the colonoscopy images generated under different age conditions show different mucosal phenotypic features. Moreover, with the increase of age, the mucosal vascular texture shows a decreasing trend, thus indicating that the age prediction model has good generative interpretability.

[0030] In summary, this invention uses a fixed intestinal structural depth map and an age-conditional diffusion model to provide a generative visualization that intuitively explains the key phenotypes of the colonoscopy age prediction model, avoiding structural interference and accurately revealing age-related features such as mucosal texture and microvessels. It is also compatible with existing models, has low deployment costs, and can quantitatively assess the degree of deviation from intestinal aging, providing an objective and interpretable means of aging assessment for clinical practice. The generated images conform to the real physiological morphology of the intestine and have high clinical reference value.

[0031] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0032] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0033] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0034] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0035] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0036] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for interpreting age-related phenotypic patterns in colonoscopy images based on a conditional diffusion model, characterized in that, The method includes the following steps: Representative images representing age information from each colonoscopy report are selected from the trained colonoscopy age prediction model. Monocular depth estimation is performed on a representative image to obtain the corresponding depth map; Training samples are constructed based on representative images, depth maps, and age labels to train a conditional diffusion model that is jointly constrained by age conditions and depth structure conditions. During the inference phase, a fixed depth map and a random seed are used to generate colonoscopy images corresponding to different ages by changing the age conditions, in order to interpret the key image phenotypes on which the age prediction model is based.

2. The method for interpreting age-related phenotypic patterns in colonoscopy images based on a conditional diffusion model according to claim 1, characterized in that, The representative image is selected from the attention weights output by the attention head of the age prediction model, and the image with the highest weight is selected as the representative image of the colonoscopy report.

3. The method for interpreting age-related phenotypic patterns in colonoscopy images based on a conditional diffusion model according to claim 1, characterized in that, Monocular depth estimation was performed using the MiDaS3.0 monocular depth estimator.

4. The method for interpreting age-related phenotypic patterns in colonoscopy images based on a conditional diffusion model according to claim 1, characterized in that, The conditional diffusion model employs a conditional image generation framework based on StableDiffusion.

5. The method for interpreting age-related phenotypic patterns in colonoscopy images based on a conditional diffusion model according to claim 1, characterized in that, Age labels are used as age-conditional inputs to the backbone network of the diffusion model, while depth maps are used as structural conditions and injected into the diffusion model through ControlNet branches.

6. The method for interpreting age-related phenotypic patterns in colonoscopy images based on a conditional diffusion model according to claim 1, characterized in that, During the inference phase, colonoscopy images corresponding to different ages are generated by fixing the same depth map and random seed and changing the age condition input, so as to compare age-related phenotypic differences under the same structural constraints.

7. The method for interpreting age-related phenotypic patterns in colonoscopy images based on a conditional diffusion model according to claim 1, characterized in that, Age-related phenotypes include one or more of the following: mucosal texture, microvascular course, vascular density, and surface structure.

8. The method for interpreting age-related phenotypic patterns in colonoscopy images based on a conditional diffusion model according to claim 1, characterized in that, During the inference phase, the colonoscopy image of the subject is input, the corresponding depth map is extracted and the random seed is fixed, and a corresponding reference colonoscopy image is generated under the condition of the subject's actual age. Based on the difference between the reference colonoscopy image and the real colonoscopy image, the aging status of the subject is evaluated.

Images