Fundus image classification method and system using diffusion-model-based continual test-time adaptation

Abstract

The present application relates to a fundus image classification method and system using diffusion-model-based continual test-time adaptation, which relate to the technical field of medical image classification. The method comprises: first, performing forward noising on a fundus image by means of a diffusion model, so as to obtain a noised image; then, performing reverse denoising on the noised image by means of the diffusion model, wherein during the reverse denoising process, a plurality of types of gradient guidance are introduced, including content preservation guidance, consistency guidance and style alignment guidance; introducing gradient guidance data into a denoised image to perform gradient fusion, so as to obtain a reconstructed image; and finally, using a classifier to predict the final reconstructed image. In the present application, a diffusion model is introduced to optimize fundus images, thereby achieving structure preservation, style alignment and classification stability optimization for sequential unlabeled test images; and the images are caused to gradually approximate a source-domain distribution, thereby realizing image-level domain alignment without modifying source model parameters, and ensuring the system stability and deployability.

Description

A continuous-time adaptive fundus image classification method and system based on a diffusion model Technical fields:

[0001] This application relates to the field of medical image classification technology, and in particular to an adaptive fundus image classification method and system based on a diffusion model during continuous testing. Background technology:

[0002] In the field of medical image classification, with the rapid development of deep learning, image recognition methods based on convolutional neural networks (CNNs) have been widely used in disease-aided diagnosis, especially in fundus image analysis, demonstrating high accuracy in tasks such as glaucoma classification and diabetic retinopathy (DR) grading. However, current deep learning models generally suffer from sensitivity to domain shifts; that is, when there are differences in the distribution of training and testing data (such as different imaging devices or different hospital sources), model performance deteriorates significantly, limiting their practical deployment and promotion in clinical practice.

[0003] To address the aforementioned issues, existing technologies employ Test-Time Adaptation (TTA) methods. This method allows the model to be adjusted using unlabeled data from the testing phase without accessing labeled data in the target domain, thereby improving its generalization ability. Exemplary TTA methods include TENT (Entropy Minimization), EATA (Pseudo-label Self-Training), and CoTTA (Model Regularization). These methods typically use an auxiliary loss function to update model parameters during the testing phase, achieving transfer from the source domain to the target domain.

[0004] However, in Continuous Test-Time Adaptation (CTA) scenarios, where the model needs to adapt sequentially to multiple test data sequences from different domains, traditional TTA methods are prone to two problems: First, reliance on model parameter updates can easily lead to "catastrophic forgetting," meaning the model forgets knowledge from the old domain while adapting to the new domain, resulting in poor adaptability to changes in domain distribution and unstable learning. Second, over-reliance on pseudo-labels or entropy regularization may produce incorrect supervision when predictions are unstable, further weakening model performance and making it unsuitable for medical image processing, as well as failing to guarantee the preservation of medical structural information. Furthermore, current research on diffusion-based TTA models has not yet systematically explored the medical field of fundus image classification, particularly lacking designs that incorporate medical features in areas such as continuous adaptation, structural preservation, and semantic consistency.

[0005] It is evident that existing technologies have failed to apply diffusion models to adaptive fundus image classification, and suffer from problems such as poor adaptability and model instability. Summary of the Invention:

[0006] This application provides a continuous testing-time adaptive fundus image classification method and system based on a diffusion model to improve the stability and generalization ability of the model under different image quality conditions during the testing phase, focusing on iterative optimization of medical images. On the one hand, this application realizes the application of the diffusion model to the processing of medical images such as fundus images; on the other hand, the technical solution of this application uses the diffusion denoising process to guide the test images to align with the source domain distribution, and combines mechanisms such as anatomical structure preservation, style guidance, and consistency constraints to effectively improve the stability and robustness of the model in the face of multi-source data in the actual clinical environment, without modifying the source model parameters, ensuring system stability and deployability, and solving the problems of poor adaptability and model instability in the prior art.

[0007] In a first aspect, this application provides a continuously tested adaptive fundus image classification method based on a diffusion model, including:

[0008] Obtain fundus images to be classified;

[0009] The fundus image to be classified is subjected to forward diffusion processing at time steps using a preset diffusion model to obtain a noisy image, and each time step of the forward diffusion process is analyzed. The diffusion model is an unconditional diffusion model.

[0010] During the back diffusion process, for each time step in the forward diffusion process, the noisy image is processed by the diffusion model to obtain the intermediate image of the current time step.

[0011] Based on the intermediate image, guided recognition is performed, and based on the guided recognition result, gradient guided analysis is performed on the intermediate image to obtain a denoised intermediate image and gradient guided data. The gradient guided data includes content-preserving gradient information, consistency gradient information, and style-guided gradient information.

[0012] Based on the gradient-guided data, gradient fusion processing is performed using the denoised intermediate image, and the backdiffusion process is iterated at each time step to obtain the final target reconstructed image.

[0013] The reconstructed image of the target is input into a frozen classifier for prediction processing to obtain the fundus image classification result.

[0014] Optionally, the fundus image to be classified is subjected to forward diffusion processing at a preset diffusion model to obtain a noisy image, including:

[0015] The fundus image to be classified is input into the diffusion model, and according to... Perform a forward diffusion process to generate a noisy image.

[0016] Where ò indicates that the added noise follows a standard normal distribution. The cumulative retention ratio is the product of all retention coefficients from the first time step to the t-th time step.

[0017] Optionally, during the back-diffusion process, for each time step in the forward diffusion process, the noisy image is processed using the diffusion model to obtain an intermediate image for the current time step, including:

[0018] During the back diffusion process, based on each time step in the forward diffusion process, the noisy image is analyzed through the diffusion model to determine the randomness introduction information, which includes the randomness introduction intensity.

[0019] Based on the information introduced by randomness, the noisy image is processed by the diffusion model to obtain an intermediate image.

[0020] Optionally, based on the information introduced by randomness, the noisy image is processed by the diffusion model to obtain an intermediate image, including:

[0021] For each time step, based on the information introduced by the randomness, according to Use a diffusion model to generate an intermediate image for the current time step;

[0022] Among them, ò θ The neural network model representing the predicted noise, i.e., the diffusion model, σ tThe noise standard deviation at the current time step t is used to control whether to inject additional noise into the currently generated image. The intensity of the randomness introduction is based on σ. t Decision, and σ t The value of σ is determined based on the stage at time step t during scheduling. t When σ = 0, it represents a deterministic denoising process, which does not require the introduction of randomness. t When the value is greater than 0, it indicates a denoising step with random perturbation, which introduces noise perturbation on the basis of model prediction.

[0023] Optionally, guided recognition is performed based on the intermediate image, and gradient guided analysis is performed on the intermediate image based on the guided recognition result to obtain a denoised intermediate image and gradient guided data, including:

[0024] Based on the intermediate image, intermediate state estimation is performed, and based on the denoised intermediate image obtained from the intermediate state estimation, combined with the solid color image corresponding to the fundus image to be classified, image stability is determined through image distance analysis to obtain the guidance application result;

[0025] When the guidance application result is determined to apply guidance, the content preservation gradient is calculated based on the denoised intermediate image to obtain the content preservation gradient information.

[0026] Enhancement processing is performed on the denoised intermediate image to obtain an enhanced image, and gradient consistency prediction is performed on the denoised intermediate image and the enhanced image to obtain consistency gradient information.

[0027] The similarity between the denoised intermediate image and the semantic text of the multimodal prediction model is analyzed by a preset multimodal prediction model, and the denoised intermediate image is guided to approach the target style corresponding to the semantic text during the backdiffusion reconstruction process to obtain style guidance gradient information.

[0028] Optionally, intermediate state estimation is performed based on the intermediate image, and based on the denoised intermediate image obtained from the intermediate state estimation, combined with the solid color image corresponding to the fundus image to be classified, image stability is determined through image distance analysis to obtain the guidance application result, including:

[0029] according to Perform intermediate state estimation to obtain the denoised intermediate image.

[0030] according to Analyze the first distance information between the denoised intermediate image and the pure black and pure white images in the solid color image, and, based on... The pixel differences between the denoised intermediate image and the fundus image to be classified are analyzed to obtain the second distance information;

[0031] The first distance information and the second distance information are compared, and... When this happens, the application of guidance will be determined as the result of guidance application;

[0032] Where, x blk For a pure black image, x whi For a pure white image, when the first distance information is greater than or equal to the second distance information, it is determined that the denoised intermediate image forms a stable structure, and guidance is applied.

[0033] Optionally, enhancement processing is performed on the denoised intermediate image to obtain an enhanced image, and gradient consistency prediction is performed on the denoised intermediate image and the enhanced image to obtain consistency gradient information, including:

[0034] Image enhancement processing is performed on the denoised intermediate image to obtain the enhanced image;

[0035] Using the denoised intermediate image and the enhanced image as input, according to Predict the probability distribution;

[0036] according to Calculate the prediction consistency gradient;

[0037] Where H() represents the entropy that measures the uncertainty of prediction, and A k () represents a series of image enhancements. This represents the probability distribution of a single prediction made by the classifier for the denoised intermediate image.

[0038] Optionally, the similarity between the denoised intermediate image and the semantic text of the multimodal prediction model is analyzed using a preset multimodal prediction model, and the denoised intermediate image is guided to approach the target style corresponding to the semantic text during the backdiffusion reconstruction process to obtain style guidance gradient information, including:

[0039] The denoised intermediate image is input into the multimodal model, according to... The semantic text of the multimodal model is introduced into the denoised intermediate image, and the similarity between the denoised intermediate image and the semantic text is calculated.

[0040] according to Calculate style-guided gradient information;

[0041] Where r is the text input to the model, E i () is the image encoder of the model, E t () represents the text encoder of the model, and η is the parameter that controls the style-guided gradient.

[0042] Optionally, based on the denoised intermediate image, a content-preserving gradient is calculated to obtain content-preserving gradient information, including:

[0043] Using the denoised intermediate image as input, according to The calculation content preserves the gradient g content ;

[0044] Specifically, based on the gradient-guided data, gradient fusion processing is performed using the denoised intermediate image, and the backdiffusion process is iterated at each time step to obtain the final target reconstruction image, including:

[0045] according to The denoised intermediate image and gradient-guided data are fused together; λ is a parameter that controls the gradient.

[0046] Secondly, this application provides a continuously test-time adaptive fundus image classification system based on a diffusion model, comprising:

[0047] Image acquisition module, used to acquire fundus images to be classified;

[0048] The forward diffusion module is used to perform forward diffusion processing on the fundus image to be classified at time steps using a preset diffusion model to obtain a noisy image, and to analyze each time step of the forward diffusion process. The diffusion model is an unconditional diffusion model.

[0049] The back diffusion module is used to process the noisy image through the diffusion model at each time step in the forward diffusion process during the back diffusion process to obtain the intermediate image at the current time step.

[0050] The gradient guidance module is used to perform guided recognition based on the intermediate image, and perform gradient guidance analysis processing based on the intermediate image based on the guided recognition result to obtain a denoised intermediate image and gradient guidance data. The gradient guidance data includes content preservation gradient information, consistency gradient information and style guidance gradient information.

[0051] The fusion and reconstruction module is used to perform gradient fusion processing based on the gradient-guided data and the denoised intermediate image, and to iterate the backdiffusion process at each time step to obtain the final target reconstructed image.

[0052] The prediction module is used to input the reconstructed image of the target into the frozen classifier for prediction processing to obtain the fundus image classification result.

[0053] In summary, this application's embodiments introduce a diffusion model for guided image optimization of the target domain image during the testing phase. During this guided optimization process, firstly, the fundus image is forward-denoised using the diffusion model to obtain a noisy image. Then, the noisy image is reverse-denoised using the diffusion model. In this reverse denoising process, various gradient guidance methods are introduced, including content preservation guidance, consistency guidance, and style alignment guidance. Gradient fusion is performed on the denoised image using gradient-guided data. Through iterative reverse diffusion at each time step, the target reconstructed image is finally obtained. This achieves unsupervised correction of the image distribution, making it closer to the source domain distribution and effectively preventing the loss of medical image information. It also improves the classification model's generalization ability to continuous multi-domain data, without modifying the source model parameters throughout the process. Finally, a classifier is used to predict the reconstructed image. This application, based on the application of a diffusion model for processing fundus images, effectively improves the model's stability and robustness in real-world clinical environments with multi-source data, solving the problems of poor adaptability and model instability in existing technologies, and ensuring system stability and deployability. Attached Figure Description

[0054] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0055] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 is a flowchart illustrating an adaptive fundus image classification method based on a diffusion model during continuous testing, as provided in an embodiment of this application.

[0057] Figure 2 is a flowchart illustrating the steps of an adaptive fundus image classification method based on a diffusion model during continuous testing, provided in an optional embodiment of this application.

[0058] Figure 3 is a gradient guidance flowchart provided as an example in this application;

[0059] Figure 4 is a flowchart of an example of an adaptive fundus image classification method based on a diffusion model during continuous testing provided in this application.

[0060] Figure 5 is a flowchart of target domain image adaptation provided in an example of this application;

[0061] Figure 6 is a structural block diagram of a continuous test-time adaptive fundus image classification system based on a diffusion model provided in an embodiment of this application. Detailed implementation method:

[0062] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0063] To facilitate understanding of the embodiments of this application, further explanations and descriptions will be provided below in conjunction with the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of this application.

[0064] Figure 1 is a flowchart illustrating a continuously adaptive fundus image classification method based on a diffusion model provided in an embodiment of this application. As shown in Figure 1, the continuously adaptive fundus image classification method based on a diffusion model provided in this embodiment of the application may specifically include the following steps:

[0065] Step 110: Obtain the fundus image to be classified.

[0066] This embodiment proposes a diffusion-based continuous test-time adaptive fundus image classification method (DiffCTA), primarily targeting fundus images in medical imaging, addressing key issues and challenges in adaptive scenarios during continuous testing of fundus images. This embodiment can achieve continuous test classification of single or multiple fundus images.

[0067] The solution in this embodiment can be applied to, but is not limited to, tasks such as glaucoma classification and diabetic retinopathy grading.

[0068] Step 120: Perform forward diffusion processing on the fundus image to be classified using a preset diffusion model to obtain a noisy image, and analyze each time step of the forward diffusion process.

[0069] The diffusion model is an unconditional diffusion model.

[0070] In this specific implementation, this embodiment focuses on the in-depth study of the application of diffusion models in fundus image classification. When using the diffusion model to reconstruct and optimize fundus classification images (including forward diffusion and backward diffusion processes), the fundus image to be classified is input into the diffusion model. The diffusion model performs forward diffusion processing on the input image, which can be understood as a forward noise addition process (or image noise addition). Specifically, at the image level, the diffusion model gradually adds noise to the input image, simulating the process of degrading from a clear image to a random image. During the forward diffusion process, the diffusion model mainly uses continuous time steps t as a reference to perform the forward diffusion process, generating a noisy image. Furthermore, it analyzes each time step during the forward diffusion process to determine the total time step. For example, if the total time step is T, forward diffusion is performed starting from time step t = 1 and ending at time step t = T.

[0071] Step 130: In the back diffusion process, for each time step in the forward diffusion process, the noisy image is processed by the diffusion model to obtain the intermediate image of the current time step.

[0072] After obtaining the fully noisy image in this embodiment, the diffusion model performs a reverse diffusion process, which can be understood as a reverse denoising process, mainly to restore and optimize the noisy image.

[0073] Backdiffusion is an iterative optimization process, mainly including generating intermediate images for the current time step, gradient-guided analysis, and gradient fusion. When performing backdiffusion, it is primarily performed at each time step of the forward diffusion process.

[0074] For example, the forward diffusion process is from time step t=1 to t=T, and the reverse diffusion process is from time step t=T to t=1.

[0075] In this specific implementation, the backdiffusion process in the diffusion model framework adopted in this embodiment is a process of gradually restoring the target image from a purely noisy image. At each time step, the diffusion model predicts the noise residual of the noisy image, and then uses this residual to update the noisy image, generating an intermediate image. Specifically, in the first backdiffusion process at a time step (e.g., t=T), the diffusion model directly updates the noisy image based on the predicted noise residual to obtain the intermediate image. In subsequent backdiffusion processes (e.g., from t=T-1 to t=1), the reconstructed image obtained after the backdiffusion process at the previous time step can be used to predict the noise residual through the diffusion model and update the reconstructed image to obtain the intermediate image, thereby achieving iterative optimization.

[0076] Step 140: Perform guided recognition based on the intermediate image, and perform gradient guided analysis processing based on the intermediate image according to the guided recognition result to obtain a denoised intermediate image and gradient guided data.

[0077] The gradient guidance data includes content preservation gradient information, consistency gradient information, and style guidance gradient information. Content preservation gradient information is used to implement content preservation guidance; consistency gradient information is used to implement consistency guidance; and style guidance gradient information is used to implement style alignment guidance.

[0078] In its implementation, the diffusion model estimates intermediate states based on intermediate images to obtain denoised intermediate images. Then, guided recognition is performed based on these denoised intermediate images to determine whether the image has displayed basic structure. This can be understood as determining whether the image possesses a certain stable structural content. When the denoised intermediate image has not yet formed a stable structure, guided intervention can be temporarily suspended, and the generated denoised intermediate image can be used as the reconstructed image for the next time step of backdiffusion processing. When the denoised intermediate image is identified as possessing certain structural content, gradient guidance can be applied to obtain gradient guidance data, which is used for subsequent image fusion and reconstruction.

[0079] In practical implementation, this embodiment employs an anatomy-aware sampling optimization mechanism. Primarily, before applying guidance to the image, this mechanism's guidance recognition and judgment effectively avoids the instability caused by blind guidance when the image is in a high-noise stage. On one hand, this mechanism improves the smoothness and effectiveness of the reverse generation path, enhances the model's robustness in multi-domain continuous adaptation, and achieves more stable continuous adaptation. On the other hand, this guidance recognition and judgment is one of the key technical steps in achieving structural preservation and style fusion unity in the final reconstructed image.

[0080] Step 150: Based on the gradient-guided data, gradient fusion processing is performed in conjunction with the denoised intermediate image, and the backdiffusion process is iterated at each time step to obtain the final target reconstructed image.

[0081] In the specific implementation, for each time step of the backdiffusion process, if it is determined that the denoised intermediate image generated at the current time step requires gradient guidance, then the gradient guidance data is fused into the denoised intermediate image to achieve image gradient fusion and reconstruction, resulting in the reconstructed image at the current time step. If it is determined that the denoised intermediate image generated at the current time step does not require gradient guidance, then the denoised intermediate image is directly used as the reconstructed image at the current time step. This reconstructed image is then used for backdiffusion in the next time step. Using each time step as a reference, the backdiffusion process is iteratively completed to obtain the final target reconstructed image.

[0082] Therefore, this embodiment achieves the fusion of the above three types of optimization information (i.e., gradient-guided data) to adjust the current image generation result, making the final reconstructed image closer to the style of the source domain image and structurally complete. Specifically, in this embodiment, the diffusion model-driven image adaptation module uses an unconditional diffusion model to perform forward denoising and reverse denoising processes on the test image, gradually generating an image that better conforms to the distribution of the source domain during the reverse denoising process. The denoising diffusion probabilistic model (DDPM) is used to achieve image domain adaptation without relying on pseudo-label training or model parameter updates.

[0083] In this context, source domain images can be understood as the source domain medical image dataset used when training the classifier; source domain distribution is the image distribution corresponding to the source domain medical image dataset; and source domain style is the style of each image in the source domain medical image dataset.

[0084] Step 160: Input the reconstructed target image into the frozen classifier for prediction processing to obtain the fundus image classification result.

[0085] This embodiment inputs the finally recovered image, i.e., the target reconstructed image, into a pre-trained classifier (such as a frozen ResNet-50) for inference / prediction, obtaining the classification result of the fundus image output by the classifier. The entire classification and recognition process requires no parameter updates, realizing a test-time adaptive mechanism without modifying the source model parameters. The classifier is in a frozen state, with parameters consistent with those trained in the source domain. After training in the source domain is complete, the parameters are no longer updated, meaning no fine-tuning or retraining is required. During the testing phase, this frozen classifier is directly used as a stable discriminator to obtain the output classification result. By optimizing the input image quality and distribution, it bypasses the path of fine-tuning model parameters in traditional test-time adaptive methods, effectively avoiding problems such as model parameter drift and catastrophic forgetting. This achieves true test-time input alignment, offering practical application value with convenient deployment and rapid adaptation.

[0086] Specifically, this embodiment introduces a diffusion model to iteratively optimize the target domain image, making it gradually approach the source domain distribution. During the testing phase, it achieves domain alignment at the image level, avoiding parameter modifications to the original classification model and ensuring system stability and deployability.

[0087] Furthermore, the solution in this embodiment can effectively improve the stability and generalization ability of the classification model under different hospital, equipment and image quality conditions during the testing phase.

[0088] As can be seen, the solution of this application aims to solve the problems of strong model parameter dependence, sensitivity to false labels, and loss of structural information in existing test-time adaptive and continuous test-time adaptive methods in medical image classification tasks, especially in fundus image classification scenarios. This embodiment introduces a diffusion model to perform image-guided optimization of the target domain image during the test phase. In the guided optimization process, firstly, the fundus image is forward-denoised using the diffusion model to obtain a noisy image. Then, the noisy image is reverse-denoised using the diffusion model. During the reverse denoising process, it is determined whether gradient guidance should be performed. When gradient guidance is determined, various gradient guidance information, including content-preserving guidance, consistency guidance, and style alignment guidance, is used to introduce gradient guidance data into the denoised image for gradient fusion, reconstructing the image. The reverse denoising process is iterated until a target reconstructed image with a style close to the source domain image and a complete structure is obtained. Finally, a classifier is used to predict the final reconstructed image. Therefore, this embodiment realizes the application of the diffusion model to adaptive fundus image classification. The diffusion model does not modify the source model parameters during the optimization and reconstruction of the input image, solving the problems existing in the prior art in the medical image classification process.

[0089] Referring to Figure 2, a flowchart illustrating the steps of an optional embodiment of this application provides a continuously adaptive fundus image classification method based on a diffusion model. The method specifically includes the following steps:

[0090] Step 210: Obtain the fundus image to be classified.

[0091] For a description of step 210, please refer to step 110 above. This implementation will not be described in detail here.

[0092] Step 220: Perform forward diffusion processing on the fundus image to be classified using a preset diffusion model to obtain a noisy image, and analyze each time step of the forward diffusion process.

[0093] The diffusion model is an unconditional diffusion model.

[0094] In one optional embodiment, the fundus image to be classified is subjected to forward diffusion processing at a time step using a preset diffusion model to obtain a noisy image, including: inputting the fundus image to be classified into the diffusion model, and according to... Perform a forward diffusion process to generate a noisy image. Where ò indicates that the added noise follows a standard normal distribution. The cumulative retention ratio is the product of all retention coefficients from the first time step to the t-th time step.

[0095] In this embodiment, to simulate the process of a clear image degrading into a random image, a diffusion model is used to perform forward diffusion on the fundus images to be classified, according to the formula... Implement forward noise addition to an image.

[0096] It should be noted that the technical solution in this embodiment uses a fixed noise scheduling strategy to add noise to the image forward in order to construct a reversible diffusion trajectory. Its main function is to provide a reasonable initial state for subsequent image guidance and adaptive reconstruction. The key technical innovation of this invention mainly focuses on a series of guidance mechanisms introduced in the back-diffusion stage.

[0097] Step 230: During the back diffusion process, based on each time step in the forward diffusion process, the noisy image is analyzed using the diffusion model to determine the information introduced by randomness.

[0098] The randomness introduction information includes the randomness introduction intensity.

[0099] In its implementation, each time the diffusion model denoises an image, it first estimates the noise in the current image (initially a noisy image, and the image optimized in the previous time step is used as input in subsequent iterations) and analyzes the noise intensity coefficients to determine whether to introduce randomness and to what extent. Introducing randomness can be understood as whether to introduce noise perturbation to improve the diversity and generalization ability of the generated images.

[0100] Step 240: Based on the information introduced by randomness, the noisy image is processed by the diffusion model to obtain an intermediate image.

[0101] In the specific implementation, when it is determined that randomness should be introduced, the intensity of the introduced noise is determined based on the randomness introduction information, and the noisy image is processed by the diffusion model to introduce additional noise and obtain an intermediate image; when it is determined that randomness should not be introduced, the noisy image is directly processed by the diffusion model to generate an intermediate image.

[0102] Optionally, this embodiment uses the introduced randomness information as a basis to process the noisy image through the diffusion model to obtain an intermediate image. Specifically, it may include: for each time step, using the introduced randomness information as a basis, according to... The intermediate image at the current time step is generated using a diffusion model; where, ò θ The neural network model representing the predicted noise, i.e., the diffusion model, σ t The noise standard deviation at the current time step t is used to control whether to inject additional noise into the currently generated image. The intensity of the randomness introduction is based on σ. t Decision, and σ t The value of σ is determined based on the stage at time step t during scheduling. t When σ = 0, it represents a deterministic denoising process, which does not require the introduction of randomness. t When the value is greater than 0, it indicates a denoising step with random perturbation, which introduces noise perturbation on the basis of model prediction.

[0103] In this embodiment, a noise estimator based on a diffusion model is used to estimate the noise standard deviation σ at the current time step t. t , σ t Also known as noise intensity factor. Utilizing the formula... It accurately introduces a certain degree of randomness into a noisy image to obtain an intermediate image.

[0104] In this embodiment, each time step of the reverse denoising process updates the image based on the noise residual predicted by the model. Whether to introduce "new randomness" and the intensity of this introduction during image updating are primarily determined by the parameter σ. t Control. The following applies to σ. t Further explanation:

[0105] ①σ t To control whether to inject additional noise into the currently generated image, this implementation utilizes a randomness introduction strategy, combining the advantages of "determinism" and "randomness" as an important mechanism in the diffusion model to improve the quality of generated images, thereby maintaining the diversity, randomness, and robustness of the generated images; ②σ t Used to determine whether randomness is introduced, specifically, when σ t When σ = 0, it represents a deterministic denoising process, meaning the current step completely depends on the noise prediction results of the neural network, with no additional random perturbations; when σ = 0, it represents a deterministic denoising process. t When σ > 0, it represents a denoising step with random perturbation. Noise perturbation is introduced based on the model prediction to improve the diversity and generalization ability of the generated images. ③ The degree of randomness introduced is determined by σ. t Size determines: σ tThe value of σ is usually scheduled according to the stage of time step t (the closer to the initial image stage, the smaller the perturbation). In this embodiment, a larger σ is given in the initial time step (when t is closer to t=T). t This allows the image to retain a certain degree of "blur and plasticity," while in the later stages, as t approaches t=0, σ t The value is reduced or even approaches zero, thereby ensuring the stability and repeatability of the final output image.

[0106] Step 250: Perform guided recognition based on the intermediate image, and perform gradient guided analysis processing based on the intermediate image according to the guided recognition result to obtain a denoised intermediate image and gradient guided data.

[0107] The gradient guidance data includes content preservation gradient information, consistency gradient information, and style guidance gradient information.

[0108] In related technologies, for the specific task of fundus image classification, existing solutions using CTA methods exhibit more significant limitations in performance, mainly due to: 1. Limited sample size, making it difficult for training data to cover a sufficient range of domain distributions; 2. A small number of categories, leading to overconfidence in the model's predicted probability distribution and low entropy values, thus affecting the adaptive effect of entropy-based optimization. To address this technical problem, existing solutions, such as VPTTA methods, attempt to utilize a prompt-tuning mechanism based on Fourier transform to achieve passive model modification adaptation strategies. However, this still requires additional training of prompt parameters, limiting its practicality. Another emerging direction is image optimization strategies based on diffusion models, such as DDA (a mainstream test-time adaptive method). This method removes noise from test samples at the image level and guides them towards the source domain distribution, achieving "returning to the source" adaptation.

[0109] However, current research on diffusion models for TTA has not yet systematically explored the medical field of fundus image classification, especially lacking designs that incorporate medical features in terms of continuous adaptation, structure preservation, and semantic consistency.

[0110] Therefore, addressing the shortcomings of the existing technologies mentioned above, this embodiment proposes a continuous-test-time adaptive fundus image classification method based on a diffusion model. The aim is to achieve accurate fundus image classification without modifying the source model parameters, ensuring system stability and deployability. This embodiment primarily focuses on the diffusion denoising process, guiding the noisy images to align with the source domain distribution. Simultaneously, it incorporates mechanisms such as anatomical structure preservation, style guidance, and consistency constraints to effectively improve the model's stability and robustness against multi-source data in real-world clinical environments.

[0111] Optionally, the above-mentioned guided recognition based on the intermediate image, and gradient guided analysis processing based on the guided recognition result to obtain a denoised intermediate image and gradient guided data, may include the following sub-steps:

[0112] Sub-step 2501: Based on the intermediate image, perform intermediate state estimation, and based on the denoised intermediate image obtained from the intermediate state estimation, combine it with the solid color image corresponding to the fundus image to be classified, determine the image stability through image distance analysis, and obtain the guidance application result.

[0113] Solid color images include pure white images and pure black images.

[0114] In this embodiment, the gradient guidance flowchart shown in Figure 3 is referenced. The diffusion model first updates based on the intermediate image, estimating the intermediate state of the image to obtain the denoised intermediate image at the current time step. Then, using the currently obtained denoised intermediate image as a reference, the image distance is analyzed to determine the stability of the denoised intermediate image, thereby determining whether guidance should be applied. When the image is determined to be stable, it indicates that the currently obtained denoised intermediate image has a certain structural content and tends to be stable, making it suitable for gradient guidance. The determination to apply guidance can be taken as the result of guidance application. When the image has not yet formed a stable structure, guidance intervention can be temporarily suspended, and the absence of guidance can be taken as the result of guidance application.

[0115] In practical implementation, whether to apply guidance is mainly based on the stability judgment of the denoised intermediate image. This embodiment mainly combines the analysis of image stability using the denoised intermediate image, the solid color image, and the fundus image to be classified. Specifically, to improve the robustness of the guidance module in the early stage of image diffusion, this embodiment uses two extreme benchmark images (i.e., a pure black image and a pure white image) as reference judgment standards. The denoised intermediate image is compared with the fundus image to be classified and the solid color image, respectively. When comparing images, the pixel difference between the denoised intermediate image and the pure black and pure white images is mainly calculated to obtain the corresponding image distance (assumed to be L1 distance); the pixel difference between the denoised intermediate image and the original image is calculated to obtain the corresponding image distance (assumed to be L2 distance); and the application of gradient guidance is determined by comparing the image distances.

[0116] For example, when L1 is not less than L2, it means that the denoised intermediate image is closer to the original image and has a certain structural content, making it suitable for applying gradient guidance; when L1 is less than L2, it means that the denoised intermediate image is still closer to a pure black or pure white state, and it is considered that it has not yet formed a stable structure. In the current time step, guidance intervention can be temporarily suspended, and the application of gradient guidance will be re-judged in the back diffusion of the next time step.

[0117] This embodiment utilizes the gradient-guided judgment mechanism to construct a structure-aware sampling optimization mechanism. This mechanism effectively avoids the instability caused by blind guidance during high-noise stages, improving the smoothness and effectiveness of the reverse-diffusion path. It is one of the key technical aspects for achieving structure preservation and style fusion. Therefore, this embodiment determines whether to apply further guidance to the current image by judging whether the image generated by the current reverse diffusion has a perceptible structure, avoiding blind intervention in the early stages of diffusion and improving optimization efficiency and stability.

[0118] In an optional embodiment, this embodiment performs intermediate state estimation based on the intermediate image, and based on the denoised intermediate image obtained from the intermediate estimation, combined with the solid color image corresponding to the fundus image to be classified, determines image stability through image distance analysis to obtain the guidance application result, which may include: according to Perform intermediate state estimation to obtain the denoised intermediate image. according to Analyze the first distance information between the denoised intermediate image and the pure black and pure white images in the solid color image, and, based on... Analyze the pixel differences between the denoised intermediate image and the fundus image to be classified to obtain second distance information; compare the first distance information and the second distance information, and... When this happens, the application of guidance will be determined as the result of guidance application; where x blk For a pure black image, x whi For a pure white image, when the first distance information is greater than or equal to the second distance information, it is determined that the denoised intermediate image forms a stable structure, and guidance is applied.

[0119] In practical implementation, distance information is obtained by calculating the pixel differences between different images. This embodiment firstly... The pixels of the denoised intermediate image and the pixels of the solid color image are unfolded into vectors, and the first distance information L1 between the denoised intermediate image and the solid white and solid black images is calculated. Then, according to... The pixels of the denoised intermediate image and the pixels of the original image are unfolded into vectors, and the second distance information L2 between the images is calculated.

[0120] In this embodiment, the judgment logic is set as follows: only when the current denoised intermediate image is closer to the original image compared to the pure black / pure white image and the original image is it considered to have sufficient structure and can be guided. Specifically, it mainly judges the magnitude of the first distance information and the second distance information, such as when the formula... When this condition is met, it indicates that the first distance information L1 is not less than the second distance information L2. At this point, the current image is closer to the original image, and it already possesses certain structural content, making it suitable for applying content guidance, style guidance, or consistency optimization. Similarly, when... This indicates that the first distance information L1 is less than the second distance information L2. At this time, the current image is still closer to a pure black or pure white state, indicating that the image is in an overly blurred or unformed stage and has not yet formed a stable structure. Applying guidance too early at this time may introduce interference, so guidance intervention will not be performed for the time being.

[0121] Sub-step 2502: When the guidance application result is determined to apply guidance, the content preservation gradient is calculated based on the denoised intermediate image to obtain content preservation gradient information.

[0122] In its implementation, as shown in Figure 4, when guidance is applied, the diffusion model uses the denoised intermediate image as a reference to calculate content-preserving gradient information, which is used to achieve content gradient guidance or structure-preserving guidance. This content-preserving gradient information is used in subsequent image reconstruction to ensure that the spatial structure of the reconstructed image remains consistent with the source domain training image. Specifically, structure-preserving guidance mainly utilizes the structural consistency between the image and the original image during backdiffusion to preserve key anatomical features in the image (such as the optic disc and vascular structures), thereby improving the reliability of medical image classification.

[0123] In an optional embodiment, this application embodiment calculates the content-preserving gradient based on the denoised intermediate image to obtain content-preserving gradient information, which may include: taking the denoised intermediate image as input, and according to... The calculation content preserves the gradient g content .

[0124] In actual implementation, this embodiment applies the following gradient guidance: λ(g content +g style +g consist Among them, for content-preserving gradient g content Mainly using formulas To achieve gradient preservation in computation, the primary input is a denoised intermediate image, which is then used to calculate the gradient of the denoised intermediate image using a formula. The spatial distance between the image and the fundus image x0 to be classified is used to calculate the appropriate content-preserving gradient.

[0125] Sub-step 2503: Enhancement processing is performed on the denoised intermediate image to obtain an enhanced image, and gradient consistency prediction is performed on the denoised intermediate image and the enhanced image to obtain consistent gradient information.

[0126] In this specific implementation, referring to Figure 3, this embodiment applies a consistency gradient guide to the denoised intermediate image to ensure that the reconstructed image maintains a high degree of consistency with the source domain training image in terms of color distribution. To achieve this high consistency in color distribution, this embodiment first performs a series of image enhancement processes on the denoised intermediate image. Then, based on the enhanced image and the denoised intermediate image, consistency gradient prediction analysis is performed to analyze the differences in color distribution between the denoised intermediate image and the source domain training image, in order to calculate the consistency gradient information. Using the consistency gradient guide can optimize the image generation process towards a stable prediction direction, thereby improving the robustness of cross-domain classification.

[0127] In an optional embodiment, enhancement processing is performed on the denoised intermediate image to obtain an enhanced image, and gradient consistency prediction is performed on the denoised intermediate image and the enhanced image to obtain consistency gradient information. This may include: performing image enhancement processing on the denoised intermediate image to obtain an enhanced image; using the denoised intermediate image and the enhanced image as input, according to... Predict the probability distribution; based on Calculate the prediction consistency gradient; where H() represents the entropy that measures prediction uncertainty, and A k () represents a series of image enhancements. This represents the probability distribution of a single prediction made by the classifier for the denoised intermediate image.

[0128] In the specific implementation, firstly, according to the formula This process involves a series of image enhancements, calculating the predicted probability distribution of the image, and primarily using the denoised intermediate image as the basis for enhancement. Image enhancement procedures include, but are not limited to, image flipping and color perturbation. After each image enhancement, based on... The prediction classifier obtains the probability distribution of a single prediction of the denoised intermediate image through multiple image augmentations and single probability distribution predictions. Then, using the probability distribution as input, the entropy of prediction uncertainty is measured, according to... This allows for accurate analysis of the color distribution differences between the current image and the source domain, and the calculation of the appropriate uniformity gradient to be applied.

[0129] Sub-step 2504 involves analyzing the similarity between the denoised intermediate image and the semantic text of the multimodal prediction model using a preset multimodal prediction model, and guiding the denoised intermediate image to approach the target style corresponding to the semantic text during the backdiffusion reconstruction process to obtain style guidance gradient information.

[0130] Among them, the multimodal prediction model refers to the CLIP (Constrastive Language-Image Pre-training) multimodal prediction model. Semantic text can be associated with classified images; for example, when the classified image is a fundus image, the semantic text can carry the word "fundus".

[0131] In the specific implementation, referring to Figure 3, to ensure that the target image maintains consistency with the source domain in visual semantic information (or semantic features), this embodiment introduces a style guidance mechanism based on the CLIP multimodal model. This mechanism guides the image to continuously approach the "real fundus image style" during the back-diffusion reconstruction process by calculating the CLIP similarity between the currently generated image and the semantic text "fundus". Specifically, the denoised intermediate image and the semantic text are mapped to a unified semantic space. The input semantic text "fundus" generates a high-dimensional semantic vector, representing the visual style features of a typical fundus image. In each step of the back-diffusion process, the similarity between the current image and this semantic vector is calculated, and its gradient is used as an optimization signal to guide the image to optimize towards higher semantic consistency, generating style guidance gradient information.

[0132] Semantic style guidance is one of the key aspects of this embodiment. This embodiment introduces a style alignment guidance mechanism based on the CLIP model during image reconstruction using a diffusion model. The text content used, such as "fundus," serves as a semantic reference for image style. The system then tends to preserve the unique anatomical structure of the retina, making the generated image visually closer to the style of the source domain "fundus image," mitigating the visual distribution differences between the source and target domains, and unifying the image semantics with the source domain. The goal of this mechanism is to ensure that the image generated during diffusion reconstruction is more closely aligned with the statistical and semantic features of the source domain medical image at the style level, thereby improving the diagnosticability of the target domain image and the model's transfer and generalization capabilities.

[0133] Furthermore, the CLIP model used in this embodiment to analyze and calculate style-guided gradient information is described in detail:

[0134] ① Text guidance (taking "fundus" as an example): The introduced text "fundus" is mapped into a high-dimensional semantic feature vector by CLIP's text encoder, representing the "semantic style of a typical fundus image." During the backdiffusion stage, by comparing the similarity between the currently generated image and this semantic vector, the system can determine whether the image deviates from the style of a typical fundus image and adjust the image features through gradient guidance. ② Potential impacts of text guidance include: improved structural clarity: when the generated image structure is blurry, CLIP will capture the feature distribution that is inconsistent with the semantics of "fundus," thereby guiding the enhancement of structural details such as optic disc edges and vascular networks; color style alignment: CLIP can capture the implicit preferences of the "fundus image" in color channel distribution, such as enhancing red-green contrast and restoring vascular tones; ③ noise interference suppression: when there are textures or artifacts in the image that deviate from the real fundus style, they will also be regarded by CLIP as semantic mismatches, guiding the system to reduce related interference components.

[0135] Therefore, this embodiment effectively addresses the misleading supervision problem caused by inaccurate pseudo-labels. This embodiment introduces multiple explicit guidance signals (i.e., gradient guidance) during the diffusion process. First, it analyzes and calculates different gradient guidance signals to incorporate gradient guidance during image reconstruction, thereby improving the robustness of adaptation and reducing erroneous feedback caused by reliance on pseudo-labels.

[0136] Furthermore, this embodiment preserves key anatomical features in fundus images. Specifically, by introducing a pixel-level content preservation mechanism during diffusion, it ensures that important structures such as the optic disc and blood vessels are not destroyed during optimization, thereby guaranteeing the interpretability and clinical usability of medical diagnoses.

[0137] In an optional embodiment, this application embodiment analyzes the similarity between the denoised intermediate image and the semantic text of the multimodal prediction model using a preset multimodal prediction model, and guides the denoised intermediate image to approach the target style corresponding to the semantic text during backdiffusion reconstruction to obtain style guidance gradient information. Specifically, this may include: inputting the denoised intermediate image into the multimodal model, according to... The semantic text of the multimodal model is introduced into the denoised intermediate image, and the similarity between the denoised intermediate image and the semantic text is calculated. according to Calculate style-guided gradient information; where r is the text input to the model, E i () is the image encoder of the model, E t () represents the text encoder of the model, and η is the parameter that controls the style-guided gradient.

[0138] Step 260: Based on the gradient-guided data, gradient fusion processing is performed in conjunction with the denoised intermediate image, and the backdiffusion process is iterated at each time step to obtain the final target reconstructed image.

[0139] In this specific implementation, the gradient-guided data is combined with the denoised intermediate image to perform gradient fusion, reconstruct the denoised intermediate image, obtain the reconstructed image at the current time step, and complete the backdiffusion process at the current time step.

[0140] In practice, after completing the backdiffusion at the current time step, it can be determined whether there are subsequent time steps. For each subsequent time step, the backdiffusion process is performed. When performing backdiffusion at each subsequent time step, the reconstructed image obtained by backdiffusion at the previous time step is mainly used. This iterative backdiffusion process at each time step is achieved to obtain the final reconstructed target image.

[0141] In this embodiment, after the diffusion process is complete, the generated target domain image is aligned with the source domain feature distribution at multiple levels, including structure, style, and consistency. This image can then be input into a classifier trained on the source domain for inference.

[0142] Therefore, this embodiment focuses on the adaptive problem in the testing phase, mainly through image-level reconstruction using a diffusion model, enabling the target domain image to align with the source domain distribution during the input phase. The innovative improvement of this embodiment in the classification phase lies not in adjusting the classifier itself, but in reconstructing the style structure of the input image, thus achieving improved classification accuracy without modifying the source model.

[0143] Optionally, this embodiment, based on the gradient-guided data and combined with the denoised intermediate image, performs gradient fusion processing and iterates the backdiffusion process at each time step to obtain the final target reconstructed image. This may include: according to... The denoised intermediate image and gradient-guided data are fused together; λ is a parameter that controls the gradient.

[0144] In practical implementation, directly introducing all the acquired gradient guidance data into the denoising intermediate image may lead to guidance failure or gradient oscillation, resulting in reduced overall stability of the diffusion process. To address this issue, this embodiment introduces a gradient control parameter λ before fusing each gradient into the denoising intermediate image. This allows for reasonable adjustment of the fusion of each gradient with the image, resulting in a reconstructed image that closely resembles the style of the source domain image and has a complete structure.

[0145] Step 270: Input the reconstructed target image into the frozen classifier for prediction processing to obtain the fundus image classification result.

[0146] Therefore, the test-time adaptive scheme proposed in this embodiment transforms the "domain adaptation" problem into an "input image transformation" problem. It does not rely on target domain labels, does not require access to source data, and does not require retraining the model; it achieves accurate, efficient, and stable cross-domain image classification solely through input image guidance and reconstruction.

[0147] Unlike existing technologies, the technological innovation of this invention mainly focuses on the guiding mechanism introduced in the backdiffusion stage. These modules effectively utilize the generative potential of the diffusion model, ensuring that the image closely resembles the source domain style while retaining key medical structures during the adaptation process, thereby improving the cross-domain generalization ability of the classification model.

[0148] Furthermore, in this embodiment, before inputting the reconstructed image into the classifier for classification, the classifier and diffusion model can be trained. Referring to Figures 4 and 5, in actual implementation, combined with the classifier training process shown in Figure 4, this embodiment mainly uses the source domain labeled dataset as the training set. The source domain labeled dataset can use multiple publicly available fundus image datasets (such as REFUGE, RIM-ONE-r3, APTOS2019) as source domain data. The images are uniformly cropped (focusing on the optic disc region), scaled (e.g., uniformly 512×512 pixels), and normalized. The resulting dataset is used to train the initial classification model and diffusion model.

[0149] The classifier (the source domain model shown in Figure 4) is trained under supervision so that it can be used to perform image classification for glaucoma or diabetic retinopathy. Once trained, the model is fixed and its parameters are not adjusted during testing.

[0150] When training the diffusion model, source domain images from the training set are used to train the model, enabling it to progressively add noise and reconstruct images through reverse denoising. The diffusion model is trained solely on source domain images and requires no labels.

[0151] Referring to Figure 5, after training the diffusion model, it can be tested. During testing, image data from different hospitals, imaging devices, or quality conditions (such as ORIGA, Messidor2, etc.) are selected as the target domain image input for continuous testing. This target domain image is then input into the diffusion model to test its ability to add noise forward and denoise backward during continuous testing. This target domain image also contains no label information and is only used for adaptive processing during the testing phase. Image-guided optimization is performed during the testing phase, utilizing mechanisms such as structural information preservation, consistency prediction, and style alignment to achieve unsupervised correction of the image distribution, making it closer to the source domain distribution and improving the classification model's generalization ability to continuous multi-domain data. This approach is suitable for continuously changing image acquisition conditions in clinical practice, is highly adaptable, requires no target domain labeled data, and possesses excellent scalability and engineering feasibility.

[0152] In summary, this embodiment introduces a diffusion model to perform image-guided optimization of the target domain image during the testing phase. In this optimization process, firstly, the fundus image is forward-denoised using the diffusion model to obtain a noisy image. Then, the noisy image is reverse-denoised using the diffusion model. During this reverse denoising process, it is determined whether gradient guidance should be performed. If gradient guidance is determined, various gradient guidance information is used, including content-preserving guidance, consistency guidance, and style-alignment guidance. Gradient guidance data is introduced into the denoised image for gradient fusion to reconstruct the image. This reverse denoising process is iterated until a target reconstructed image with a style close to the source domain image and a complete structure is obtained. Finally, a classifier is used to predict the final reconstructed image. Therefore, this embodiment applies the diffusion model to adaptive fundus image classification. The diffusion model does not modify the source model parameters during the optimization and reconstruction of the input image, solving the problems existing in the prior art for medical image classification.

[0153] In practical testing, the solution presented in this embodiment can effectively improve classification accuracy in complex domain transfer scenarios. During the experimental phase, it was primarily applied to tasks including glaucoma classification and diabetic retinopathy grading. Specifically, the diffusion model and classifier were evaluated in two medical image tasks: a glaucoma binary classification task and a diabetic retinopathy grading task. Multiple source and target domain datasets were used for combined testing to simulate scenarios with constantly changing distributions in real clinical settings.

[0154] Comparison Method: Its performance was compared with current mainstream test-time adaptive methods (such as TENT, EATA, CoTTA, SAR, and DDA). Results Analysis: In the glaucoma classification task, the proposed solution improved by approximately 2.6 percentage points compared to the best baseline method; it also achieved the highest average accuracy in the diabetic retinopathy task; simultaneously, visual results show that the generated images retain structure while more closely resembling the style of the training domain, and the classifier's predictions for these images are more stable and reliable.

[0155] In summary, this implementation scheme demonstrates that the DiffCTA framework, through image-level adaptive optimization strategies, significantly improves the model's adaptability to distribution changes without altering the original model structure and parameters. This scheme overcomes the application bottlenecks of traditional test-time adaptive techniques in medical imaging, possessing a sound theoretical foundation and demonstrating its practicality, effectiveness, and adaptability in medical image classification tasks through systematic experiments. It outperforms existing test-time adaptive methods on multiple public datasets. This embodiment provides a novel, efficient, stable, and model parameter-update-free solution for fundus image classification under continuously changing distributions. It is suitable for widespread application in clinical scenarios such as remote diagnosis and continuous patient screening, exhibiting significant practical value and industrial transformation potential, and demonstrating strong cross-domain promotion capabilities and clinical applicability.

[0156] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should know that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps may be performed in other orders or simultaneously.

[0157] As shown in Figure 6, this application embodiment also provides a continuously adaptive fundus image classification system 600 based on a diffusion model, comprising:

[0158] Image acquisition module 610 is used to acquire fundus images to be classified;

[0159] The forward diffusion module 620 is used to perform forward diffusion processing on the fundus image to be classified at time steps using a preset diffusion model to obtain a noisy image, and to analyze each time step of the forward diffusion process. The diffusion model is an unconditional diffusion model.

[0160] The back diffusion module 630 is used to process the noisy image through the diffusion model at each time step in the forward diffusion process during the back diffusion process to obtain the intermediate image of the current time step.

[0161] The gradient guidance module 640 is used to perform guided recognition based on the intermediate image, and perform gradient guidance analysis processing based on the intermediate image based on the guided recognition result to obtain a denoised intermediate image and gradient guidance data. The gradient guidance data includes content preservation gradient information, consistency gradient information and style guidance gradient information.

[0162] The fusion and reconstruction module 650 is used to perform gradient fusion processing based on the gradient-guided data and the denoised intermediate image, and to iterate the back diffusion process at each time step to obtain the final target reconstructed image.

[0163] The prediction module 660 is used to input the reconstructed image of the target into the frozen classifier for prediction processing to obtain the fundus image classification result.

[0164] It should be noted that the continuous test-time adaptive fundus image classification system based on the diffusion model provided in the embodiments of this application can execute the continuous test-time adaptive fundus image classification method based on the diffusion model provided in any embodiment of this application, and has the corresponding functions and beneficial effects of the execution method.

[0165] In practical implementation, the aforementioned system can be integrated into a device, enabling the device to function as an electronic device. This device can construct an optimized target image by forward denoising and backward denoising of fundus images, and then input this image into a classifier for prediction. This achieves optimization of the fundus image using a diffusion model, while the source model parameters do not need to be modified during classification, ensuring system stability and deployability. This electronic device can consist of two or more physical entities, or it can consist of a single physical entity. For example, the electronic device can be a personal computer (PC), a computer, a server, etc. This application embodiment does not impose specific limitations in this regard.

[0166] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the diffusion-based adaptive fundus image classification method for continuous testing as provided in any of the foregoing method embodiments.

[0167] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0168] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A continuously adaptive fundus image classification method based on a diffusion model, characterized in that, include: Obtain fundus images to be classified; The fundus image to be classified is subjected to forward diffusion processing at time steps using a preset diffusion model to obtain a noisy image, and each time step of the forward diffusion process is analyzed. The diffusion model is an unconditional diffusion model. During the back diffusion process, for each time step in the forward diffusion process, the noisy image is processed by the diffusion model to obtain the intermediate image of the current time step. Based on the intermediate image, guided recognition is performed, and based on the guided recognition result, gradient guided analysis is performed on the intermediate image to obtain a denoised intermediate image and gradient guided data. The gradient guided data includes content-preserving gradient information, consistency gradient information, and style-guided gradient information. Based on the gradient-guided data, gradient fusion processing is performed using the denoised intermediate image, and the backdiffusion process is iterated at each time step to obtain the final target reconstructed image. The reconstructed image of the target is input into a frozen classifier for prediction processing to obtain the fundus image classification result.

2. The method according to claim 1, characterized in that, The fundus image to be classified is subjected to forward diffusion processing at a preset diffusion model to obtain a noisy image, including: The fundus image to be classified is input into the diffusion model, and according to... Perform a forward diffusion process to generate a noisy image. in, This indicates that the added noise follows a standard normal distribution. The cumulative retention ratio is the product of all retention coefficients from the first time step to the t-th time step.

3. The method according to claim 1, characterized in that, In the back-diffusion process, for each time step in the forward diffusion process, the noisy image is processed using the diffusion model to obtain the intermediate image at the current time step, including: During the back diffusion process, based on each time step in the forward diffusion process, the noisy image is analyzed through the diffusion model to determine the randomness introduction information, which includes the randomness introduction intensity. Based on the information introduced by randomness, the noisy image is processed by the diffusion model to obtain an intermediate image.

4. The method according to claim 3, characterized in that, Based on the information introduced by randomness, the noisy image is processed by the diffusion model to obtain an intermediate image, including: For each time step, based on the information introduced by the randomness, according to Use a diffusion model to generate an intermediate image for the current time step; in, The neural network model representing the predicted noise, i.e., the diffusion model, σ t The noise standard deviation at the current time step t is used to control whether to inject additional noise into the currently generated image. The intensity of the randomness introduction is based on σ. t Decision, and σ t The value of σ is determined based on the stage at time step t during scheduling. t When σ = 0, it represents a deterministic denoising process, which does not require the introduction of randomness. t When the value is greater than 0, it indicates a denoising step with random perturbation, which introduces noise perturbation on the basis of model prediction.

5. The method according to claim 1, characterized in that, Based on the intermediate image, guided recognition is performed, and based on the guided recognition result, gradient guided analysis is performed on the intermediate image to obtain a denoised intermediate image and gradient guided data, including: Based on the intermediate image, intermediate state estimation is performed, and based on the denoised intermediate image obtained from the intermediate state estimation, combined with the solid color image corresponding to the fundus image to be classified, image stability is determined through image distance analysis to obtain the guidance application result; When the guidance application result is determined to apply guidance, the content preservation gradient is calculated based on the denoised intermediate image to obtain the content preservation gradient information. Enhancement processing is performed on the denoised intermediate image to obtain an enhanced image, and gradient consistency prediction is performed on the denoised intermediate image and the enhanced image to obtain consistency gradient information. The similarity between the denoised intermediate image and the semantic text of the multimodal prediction model is analyzed by a preset multimodal prediction model, and the denoised intermediate image is guided to approach the target style corresponding to the semantic text during the backdiffusion reconstruction process to obtain style guidance gradient information.

6. The method according to claim 5, characterized in that, Based on the intermediate image, intermediate state estimation is performed. Then, based on the denoised intermediate image obtained from the intermediate estimation, and combined with the solid color image corresponding to the fundus image to be classified, image stability is determined through image distance analysis to obtain the guidance application result, including: according to Perform intermediate state estimation to obtain the denoised intermediate image. according to Analyze the first distance information between the denoised intermediate image and the pure black and pure white images in the solid color image, and, based on... The pixel differences between the denoised intermediate image and the fundus image to be classified are analyzed to obtain the second distance information; The first distance information and the second distance information are compared, and... When this happens, the application of guidance will be determined as the result of guidance application; Where, x blk For a pure black image, x whi For a pure white image, when the first distance information is greater than or equal to the second distance information, it is determined that the denoised intermediate image forms a stable structure, and guidance is applied.

7. The method according to claim 5, characterized in that, Enhancement processing is performed on the denoised intermediate image to obtain an enhanced image. Gradient consistency prediction is then performed on the denoised intermediate image and the enhanced image to obtain consistency gradient information, including: Image enhancement processing is performed on the denoised intermediate image to obtain the enhanced image; Using the denoised intermediate image and the enhanced image as input, according to Predict the probability distribution; according to Calculate the prediction consistency gradient; Where H() represents the entropy that measures the uncertainty of prediction, and A k () represents a series of image enhancements. This represents the probability distribution of a single prediction made by the classifier for the denoised intermediate image.

8. The method according to claim 5, characterized in that, The similarity between the denoised intermediate image and the semantic text of the multimodal prediction model is analyzed using a pre-defined multimodal prediction model. This guides the denoised intermediate image to approximate the target style corresponding to the semantic text during backdiffusion reconstruction, yielding style-guided gradient information, including: The denoised intermediate image is input into the multimodal model, according to... The semantic text of the multimodal model is introduced into the denoised intermediate image, and the similarity between the denoised intermediate image and the semantic text is calculated. according to Calculate style-guided gradient information; Where r is the text input to the model, E i () is the image encoder of the model, E t () represents the text encoder of the model, and η is the parameter that controls the style-guided gradient.

9. The method according to claim 5, characterized in that, Based on the denoised intermediate image, the content-preserving gradient is calculated to obtain content-preserving gradient information, including: Using the denoised intermediate image as input, according to The calculation content preserves the gradient g content ; Specifically, based on the gradient-guided data, gradient fusion processing is performed using the denoised intermediate image, and the backdiffusion process is iterated at each time step to obtain the final target reconstruction image, including: according to The denoised intermediate image and gradient-guided data are fused together; λ is a parameter that controls the gradient.

10. A continuously test-time adaptive fundus image classification system based on a diffusion model, characterized in that, include: Image acquisition module, used to acquire fundus images to be classified; The forward diffusion module is used to perform forward diffusion processing on the fundus image to be classified at time steps using a preset diffusion model to obtain a noisy image, and to analyze each time step of the forward diffusion process. The diffusion model is an unconditional diffusion model. The back diffusion module is used to process the noisy image through the diffusion model at each time step in the forward diffusion process during the back diffusion process to obtain the intermediate image at the current time step. The gradient guidance module is used to perform guided recognition based on the intermediate image, and perform gradient guidance analysis processing based on the intermediate image based on the guided recognition result to obtain a denoised intermediate image and gradient guidance data. The gradient guidance data includes content preservation gradient information, consistency gradient information and style guidance gradient information. The fusion and reconstruction module is used to perform gradient fusion processing based on the gradient-guided data and the denoised intermediate image, and to iterate the backdiffusion process at each time step to obtain the final target reconstructed image. The prediction module is used to input the reconstructed image of the target into the frozen classifier for prediction processing to obtain the fundus image classification result.

Images