Disease prediction model training method and device, equipment, storage medium and program product

Abstract

The application relates to a disease prediction model training method and device, equipment, a storage medium and a program product. The method comprises the following steps: training an initial base model according to a first medical image set to obtain a trained base model; training an initial prediction model based on the trained base model and a second medical image set to obtain a prediction model; the prediction model is used for predicting diseases; the second medical image set comprises a second medical image collected at a first time, a third medical image collected at a second time, and a time interval between the first time and the second time for the same object; the trained base model is used for extracting medical features of medical images; and the first medical image set comprises a plurality of first medical images. The method can improve the training efficiency of a medical prediction model and the accuracy of disease prediction by the model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a disease prediction model training method and device, equipment, storage medium and program product. BACKGROUND

[0002] Longitudinal prediction refers to a technology for predicting the development trend of a disease and the survival time of a patient by analyzing longitudinal health data (such as medical records, examination results, treatment processes, etc.) of the patient.

[0003] In recent years, the development of artificial intelligence has brought hope for coping with these complex challenges in the global healthcare field. Combining artificial intelligence training models enables the models to have disease longitudinal prediction functions, which has become an important research direction in the medical field. Typical models include the large language model (Generative Pre-trained Transformer, GPT) based on the Transformer architecture developed by OpenAI, the pre-trained language model (Bidirectional Encoder Representations from Transformers, BERT) developed by Google, and the pre-trained language model (Text-to-Text Transfer Transformer, T5).

[0004] However, the current disease prediction model still has limitations in predicting disease progression and has the problem of inaccurate prediction. SUMMARY

[0005] Therefore, it is necessary to provide a disease prediction model training method, device, equipment, storage medium and program product capable of improving disease prediction accuracy in view of the above technical problems.

[0006] In a first aspect, the present application provides a prediction model training method, comprising:

[0007] training the initial base model based on the first medical image set to obtain a trained base model; the trained base model is used to extract medical features of medical images; the first medical image set includes a plurality of first medical images;

[0008] training the initial prediction model based on the trained base model and the second medical image set to obtain a prediction model; the prediction model is used to predict a disease; the second medical image set includes: a second medical image collected at a first time, a third medical image collected at a second time, and a time interval between the first time and the second time for the same object.

[0009] In one of the embodiments, the training of the initial base model according to the first medical image set to obtain the trained base model comprises:

[0010] The first medical image in the first medical image set is subjected to first mask processing to obtain a masked first medical image.

[0011] The initial base model is trained according to the first medical image and the masked first medical image to obtain the trained base model.

[0012] In one of the embodiments, the initial base model comprises a first initial encoder and a first initial decoder, and the training of the initial base model according to the first medical image and the masked first medical image to obtain the trained base model comprises:

[0013] The masked first medical image is input into the first initial encoder for encoding processing to obtain first encoding features.

[0014] The first encoding features are input into the first initial decoder for decoding processing to obtain first decoding features.

[0015] The first initial encoder and the first initial decoder are trained according to a first difference value between the image features of the first medical image and the first decoding features to obtain the trained base model.

[0016] In one of the embodiments, the training of the initial prediction model based on the trained base model and the second medical image set to obtain the prediction model, wherein the prediction model is used for predicting the disease, comprises:

[0017] The third medical image in the second medical image set is input into the trained base model for feature extraction to obtain features of the third medical image.

[0018] The initial prediction model is trained according to the features of the third medical image and the second medical image to obtain the prediction model.

[0019] In one of the embodiments, the training of the initial prediction model according to the features of the third medical image and the second medical image to obtain the prediction model comprises:

[0020] The second medical image is subjected to second mask processing to obtain a masked second medical image.

[0021] The initial prediction model is trained according to the masked second medical image, the features of the third medical image, and a time interval between the first time and the second time to obtain the prediction model.

[0022] In one of the embodiments, the initial prediction model comprises a second initial encoder and a second initial decoder, and the initial prediction model is trained according to the masked second medical image, the features of the third medical image, and the time interval between the first time and the second time to obtain the prediction model, which comprises:

[0023] The masked second medical image and the time interval are input into the second initial encoder for encoding processing to obtain second encoding features;

[0024] The second encoding features are input into the second initial decoder for decoding processing to obtain second decoding features;

[0025] The second initial encoder and the second initial decoder are trained according to the second difference between the features of the third image and the second decoding features to obtain the prediction model.

[0026] In one of the embodiments, the initial base model is a masked autoencoder (MAE) model, and the initial prediction model is a RETFound model.

[0027] In a second aspect, the present application further provides a training device of a prediction model, which comprises:

[0028] A first training module is configured to train an initial base model according to a first medical image set to obtain a trained base model, wherein the trained base model is used to extract medical features of medical images, and the first medical image set comprises a plurality of first medical images.

[0029] A second training module is configured to train an initial prediction model based on the trained base model and a second medical image set to obtain a prediction model, wherein the prediction model is used to predict diseases, and the second medical image set comprises a second medical image collected at a first time, a third medical image collected at a second time, and a time interval between the first time and the second time for the same object.

[0030] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0031] The initial base model is trained according to a first medical image set to obtain a trained base model, wherein the trained base model is used to extract medical features of medical images, and the first medical image set comprises a plurality of first medical images.

[0032] The initial prediction model is trained based on the trained base model and the second medical image set to obtain a prediction model; the prediction model is used for predicting the disease; the second medical image set comprises: a second medical image collected at a first time, a third medical image collected at a second time, and a time interval between the first time and the second time for the same object.

[0033] In a fourth aspect, the present application further provides a computer readable storage medium, having a computer program stored thereon, the computer program being executed by a processor to implement the following steps:

[0034] The initial base model is trained according to the first medical image set to obtain a trained base model; the trained base model is used for extracting medical features of a medical image; the first medical image set comprises a plurality of first medical images;

[0035] The initial prediction model is trained based on the trained base model and the second medical image set to obtain a prediction model; the prediction model is used for predicting the disease; the second medical image set comprises: a second medical image collected at a first time, a third medical image collected at a second time, and a time interval between the first time and the second time for the same object.

[0036] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, the computer program being executed by a processor to implement the following steps:

[0037] The initial base model is trained according to the first medical image set to obtain a trained base model; the trained base model is used for extracting medical features of a medical image; the first medical image set comprises a plurality of first medical images;

[0038] The initial prediction model is trained based on the trained base model and the second medical image set to obtain a prediction model; the prediction model is used for predicting the disease; the second medical image set comprises: a second medical image collected at a first time, a third medical image collected at a second time, and a time interval between the first time and the second time for the same object.

[0039] The training method, device, equipment, storage medium and program product of the disease prediction model are based on the first set of medical images to train the initial base model to obtain the base model with the medical characteristics of simply identifying medical images, and then based on the trained base model and the second set of medical images carrying time information, the initial prediction model is trained to obtain the prediction model with the disease prediction function. Compared with the existing training method of the disease prediction model based on image labels, the training process of the whole model does not need the medical images to carry the disease type label, reduces the workload of labeling labels, and greatly improves the training efficiency of the model. In addition, the model is trained twice, which greatly improves the accuracy of the model in predicting diseases. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other related drawings without creative labor on the basis of these drawings.

[0041] Figure 1 The application environment diagram of the training method of the prediction model in one embodiment;

[0042] Figure 2 The flowchart of the training method of the prediction model in one embodiment;

[0043] Figure 3 The flowchart of the training method of the prediction model in another embodiment;

[0044] Figure 4 The flowchart of the training method of the prediction model in another embodiment;

[0045] Figure 5 The flowchart of the training method of the prediction model in another embodiment;

[0046] Figure 6 The flowchart of the training method of the prediction model in another embodiment;

[0047] Figure 7 The flowchart of the training method of the prediction model in another embodiment;

[0048] Figure 8 The flowchart of the training method of the prediction model in another embodiment;

[0049] Figure 9 The structural block diagram of the training device of the prediction model in one embodiment. DETAILED DESCRIPTION

[0050] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0051] Longitudinal prediction refers to a technology for predicting the development trend of a disease and the survival time of a patient by analyzing longitudinal health data (such as medical records, examination results, treatment processes, etc.) of the patient.

[0052] In recent years, the development of artificial intelligence has brought hope for coping with these complex challenges in the global healthcare field. Combining artificial intelligence training models enables the models to have disease longitudinal prediction functions, which has become an important research direction in the medical field. Typical models include the large language model (Generative Pre-trained Transformer, GPT) based on the Transformer architecture developed by OpenAI, the pre-trained language model (Bidirectional Encoder Representations from Transformers, BERT) and the pre-trained language model (Text-to-Text Transfer Transformer, T5) developed by Google, etc.

[0053] However, the current disease prediction model still has limitations in predicting disease progression and has the problem of inaccurate prediction. The present application aims to solve this problem.

[0054] After the above background technology of the training method of the prediction model provided by the embodiments of the present application is introduced, the implementation environment related to the training method of the prediction model provided by the embodiments of the present application will be briefly described below. The training method of the prediction model provided by the embodiments of the present application can be applied to, for example, Figure 2The computer device shown includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to perform wired or wireless communication with external terminals. The wireless communication can be achieved through WIFI, mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program is executed by the processor to implement a training method of a prediction model. The display unit of the computer device is configured to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or can be a key, a trackball, or a touchpad arranged on the shell of the computer device, or can be an external keyboard, a touchpad, or a mouse, etc.

[0055] Those skilled in the art can understand that, Figure 2 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. A specific terminal can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0056] After the above describes the application scenario of the training method of the prediction model provided by the embodiments of the present application, the training method of the prediction model described in the present application is mainly introduced.

[0057] In one embodiment, as Figure 3 shown, a training method of a prediction model is provided. The method is applied to a computer device in Figure 2 for example, and includes the following steps:

[0058] S201, training an initial base model according to a first set of medical images to obtain a trained base model.

[0059] The first medical image set includes a plurality of first medical images, and the first medical image refers to a medical image that does not carry image acquisition time information and a disease type corresponding to the image. The initial base model can be a Masked AutoEncoder (MAE) model, and the MAE model is a visual model based on Self-Supervised Learning (SSL). The trained base model can extract medical features such as edge detection, color contrast, and texture of the medical image.

[0060] In the embodiment, when it is necessary to train the prediction model to have a disease prediction function, a base model can be trained first to have the ability to extract medical features of a medical image. Before training the base model, a first medical image set including a plurality of first medical images can be obtained, and the initial base model is trained based on the first medical image set to obtain the trained base model.

[0061] Optionally, the initial base model can be trained based on the first medical image set by using self-training, co-training, image-based semi-supervised learning, generative adversarial network, variational autoencoder, clustering analysis, autoencoder, principal component analysis, and mask processing to obtain the trained base model.

[0062] In the embodiment, before processing the first medical image in the first medical image set, the first medical image set also needs to be obtained, for example, the first medical image set can be a million-level fundus image data.

[0063] It should be noted that before training the initial base model based on the first medical image set, the initial base model (i.e., the MAE model) can be pre-trained on the ImageNet dataset first, so that the MAE model learns general image structures and features to obtain the pre-trained initial base model, and then the pre-trained initial base model is trained based on the first medical image set to obtain the trained base model.

[0064] S202, training the initial prediction model based on the trained base model and the second medical image set to obtain a prediction model.

[0065] The second medical image set includes: a second medical image collected at a first time, a third medical image collected at a second time, and a time interval between the first time and the second time for the same object. It should be noted that the second medical image set can also include: a fourth medical image collected at a third time, a time interval between the first time and the third time, and a time interval between the second time and the third time for the first object.

[0066] The prediction model is used to predict diseases. It should be noted that when the prediction model is trained, if the second medical image and the third medical image in the second medical image set are used, the trained prediction model has the function of predicting diseases after the time interval between the first time and the second time, for example, if the currently collected medical image is input into the prediction model for disease prediction, the disease type and the probability of occurrence of the disease type after the time interval between the first time and the second time are obtained.

[0067] It should be noted that when the prediction model is trained, if the second medical image and the fourth medical image in the second medical image set are used, the trained prediction model has the function of predicting diseases after the time interval between the first time and the third time, for example, if the currently collected medical image is input into the prediction model for disease prediction, the disease type and the probability of occurrence of the disease type after the time interval between the first time and the third time are obtained.

[0068] In this embodiment, after obtaining the trained base model, the second medical image and the third medical image in the second medical image set can be input into the initial prediction model to train the initial prediction model, and obtain the trained prediction model.

[0069] Optionally, the initial prediction model can be trained based on the second medical image set by using self-training, co-training, image-based semi-supervised learning, generative adversarial network, variational autoencoder, clustering analysis, autoencoder, principal component analysis, and mask processing to obtain the trained prediction model.

[0070] In this embodiment, the initial base model is trained based on the first medical image set to obtain a base model having the function of simply identifying medical features of medical images, and then the initial prediction model is trained based on the trained base model and the second medical image set carrying time information to obtain a prediction model having the function of predicting diseases. Compared with the existing method of training a disease prediction model based on image labels, the present scheme does not require medical images to carry disease type labels in the whole training process of the model, reduces the workload of labeling labels, and greatly improves the training efficiency of the model. In addition, the model is trained twice, which greatly improves the accuracy of the model in predicting diseases.

[0071] In one embodiment, in the process of training the base model, the base model is trained based on the first medical image set and the second medical image set, and the trained base model has the function of identifying medical features of medical images. Figure 2 Based on the embodiment shown in FIG. 8, the process of training the base model can be described as follows: Figure 3 The above S201 "training the initial base model based on the first medical image set to obtain a trained base model" includes:

[0072] S301, perform first mask processing on the first medical image in the first medical image set to obtain a first medical image after mask processing.

[0073] The mask processing refers to a processing manner of shielding a part of a region in an image.

[0074] In this embodiment, the initial base model is trained based on the self-training manner and the first medical image in the first medical image set. Before training, the first medical image needs to be subjected to first mask processing to obtain a first medical image after mask processing.

[0075] Optionally, the first medical image in the first medical image set can be subjected to segmentation processing to obtain a plurality of segmentation images corresponding to the first medical image, and then a part of the segmentation images in the plurality of segmentation images are randomly subjected to first mask processing to obtain a first medical image after mask processing. For example, 75% of the segmentation images in the plurality of segmentation images are randomly shielded to obtain a first medical image retaining 25% of image information.

[0076] S302, training the initial base model based on the first medical image and the first medical image after mask processing to obtain a trained base model.

[0077] In this embodiment, after obtaining the first medical image after mask processing, the initial base model can be trained based on the first medical image and the first medical image after mask processing to obtain a trained base model.

[0078] Optionally, referring to Figure 4 The initial base model includes a first initial encoder and a first initial decoder, and provides a method of training the initial base model based on the first medical image and the first medical image after mask processing, that is, the above S302 "training the initial base model based on the first medical image and the first medical image after mask processing to obtain a trained base model", which includes:

[0079] S3021, inputting the first medical image after mask processing into the first initial encoder for encoding processing to obtain a first encoding feature.

[0080] The encoding processing refers to a manner of encoding an image, and different images correspond to different encoding features. In addition, the first medical image after mask processing is different, and the corresponding first encoding feature is also different.

[0081] The first encoding feature refers to a feature of a non-masked part in the first medical image after mask processing.

[0082] In this embodiment, after obtaining the first medical image after the mask, the first medical image after the mask can be input into the first initial encoder in the initial base model for encoding processing to obtain the first encoding feature.

[0083] S3022, input the first encoding feature into the first initial decoder for decoding processing to obtain the first decoding feature.

[0084] The first decoding feature refers to all features of the first medical image inferred from the first encoding feature.

[0085] In this embodiment, after obtaining the first encoding feature, the first encoding feature can be input into the first initial decoder for decoding processing to infer all features of the first medical image, i.e. the first decoding feature.

[0086] S3023, according to the first difference between the image features of the first medical image and the first decoding feature, training the first initial encoder and the first initial decoder to obtain the trained base model.

[0087] The image features of the first medical image refer to the image features after feature extraction of the first medical image.

[0088] The first difference can be a pixel-level reconstruction loss.

[0089] In this embodiment, after obtaining the first decoding feature, the first medical image can be feature extracted to obtain the image features of the first medical image, and the first difference between the image features of the first medical image and the first decoding feature is determined, and it is determined whether the first difference satisfies the first preset difference threshold. If the first difference does not satisfy the first preset difference threshold, the model parameters of the first initial encoder and the first initial decoder are adjusted to obtain a new first initial encoder and a new first initial decoder, and the step of inputting the first image after the mask into the first initial encoder for encoding processing to obtain the first encoding feature is returned to be executed until the first difference satisfies the first preset difference threshold, and the trained base model is obtained.

[0090] Optionally, the feature extraction manner of the first medical image can include: inputting the first medical image into a pre-trained image feature extraction network for feature extraction to obtain the image features of the first medical image.

[0091] In this embodiment, by training the first initial encoder and the first initial decoder in the initial base model, the base model after training can more efficiently learn the global and local features of the image while reducing the computational burden. In addition, the training method of the base model does not need to pre-label the image, only needs to perform partial mask processing on the image, and then inputs the masked image into the first initial encoder and the first initial decoder in turn, so as to train the base model, greatly reducing the workload of image labeling and improving the training efficiency of the base model.

[0092] In one embodiment, based on the embodiment shown in Figures 2-4 Based on the embodiment shown in Figure 5 As shown in the above S202, "training the initial prediction model based on the trained base model and the second medical image set to obtain the prediction model", comprising:

[0093] S401, input the third medical image in the second medical image set into the trained base model for feature extraction to obtain the feature of the third medical image.

[0094] In the embodiments of the present application, in order to optimize the input data quality, the images in the second medical image set need to go through a series of preprocessing before entering the model, including background removal processing (remove black background and improve the attention of the lesion area), down-sampling processing (uniformly adjust the image size to 256x256 to improve the calculation efficiency), and a series of data enhancement techniques, such as random cropping, size adjustment, horizontal flipping (to improve the robustness of the model), color jittering (to enhance the adaptability of the model to images taken by different devices), and Gaussian blur (to simulate the low-quality image situation in reality). These preprocessing steps effectively improve the generalization ability of the model, making it better adapt to image data from different medical institutions and improve the stability and accuracy of prediction.

[0095] In this embodiment, after obtaining the trained base model, the third medical image in the second medical image set can be input into the trained base model for feature extraction to obtain the feature of the third medical image.

[0096] In this embodiment, before processing the second medical image set, the second medical image set also needs to be obtained. For example, the second medical image set can be a longitudinal image set containing 1304292 fundus images, which covers 304345 patients and the image collection time span is 9 years. Further, after obtaining the second medical image set, each medical image in the second medical image set can be grouped according to the patient identifier, and the collection time of each medical image is labeled. Further, an automated quality control (Automated Quality Control, QC) process can also be used to remove low-quality images (for example, blurred images, overexposed images, images with too many artifacts, etc.), and unify the size of all medical images in the second medical image set (for example, adjust the size of all medical images in the second medical image set to 256x256 pixels).

[0097] S402, training the initial prediction model according to the features of the third medical image and the second medical image to obtain a prediction model.

[0098] Wherein, the third medical image and the second medical image are fundus images collected at different time points for the same object.

[0099] In this embodiment, after obtaining the features of the third medical image, the second medical image can be further obtained, and the features of the second medical image are extracted, and the initial prediction model is trained according to the difference between the features of the second medical image and the features of the third medical image to obtain a prediction model.

[0100] In this embodiment, based on the medical images of the same object at different time points, the initial prediction model is trained, so that the trained prediction model has the function of predicting the future trend of the disease; in addition, the features of the third medical image are extracted based on the above trained base model, so that the features of the obtained third medical image are more accurate, and the prediction model trained subsequently is more accurate.

[0101] In one embodiment, in the process of training the prediction model, the features of the third medical image are extracted based on the base model, and the initial prediction model is trained according to the features of the third medical image and the second medical image to obtain a prediction model. Figure 5 Based on the embodiments shown in the above, the training process of the prediction model can be further described, such as Figure 6 The above S402 "training the initial prediction model according to the features of the third medical image and the second medical image to obtain a prediction model" includes:

[0102] S501, performing second mask processing on the second medical image to obtain a masked second medical image.

[0103] In this embodiment, the initial prediction model is trained based on the self-training manner and the second medical images in the second medical image set. Before training, the second medical images need to be subjected to second mask processing to obtain the second medical images after mask processing.

[0104] Optionally, the second medical images in the second medical image set can be subjected to segmentation processing to obtain a plurality of segmentation images corresponding to the second medical images, and then part of the segmentation images in the plurality of segmentation images are randomly subjected to second mask processing to obtain the second medical images after mask processing. For example, 75% of the segmentation images in the plurality of segmentation images are randomly occluded to obtain the second medical images retaining 25% of the image information.

[0105] S502, training the initial prediction model according to the features of the second medical images after mask processing, the features of the third medical images, and the time interval between the first time and the second time, to obtain a prediction model.

[0106] In this embodiment, after the features of the second medical images after mask processing and the third medical images are obtained, the initial prediction model can be trained based on the features of the second medical images after mask processing, the features of the third medical images, and the time interval between the first time and the second time to obtain a prediction model.

[0107] Optionally, referring to Figure 7 The above initial prediction model includes a second initial encoder and a second initial decoder, and provides a method for training the initial prediction model according to the features of the second medical images after mask processing, the features of the third medical images, and the time interval between the first time and the second time, i.e., the above S502 “training the initial prediction model according to the features of the second medical images after mask processing, the features of the third medical images, and the time interval between the first time and the second time, to obtain a prediction model”, which includes:

[0108] S5021, inputting the second medical images after mask processing and the time interval into the second initial encoder for encoding processing to obtain second encoding features.

[0109] The second medical images after mask processing are different, and the corresponding second encoding features are also different.

[0110] The second encoding features refer to the features of the unmasked part of the second medical images after mask processing carrying the time interval.

[0111] In this embodiment, after the second medical images after mask processing are obtained, the second medical images after mask processing and the time interval can be input into the second initial encoder in the initial prediction model for encoding processing to obtain second encoding features.

[0112] Optionally, in order to enable the prediction model to learn the characteristics of the disease changing over time, a sinusoidal timestep embedding method is introduced, which can generate time information encoding corresponding to the time interval based on the sine (sin) and cosine (cos) functions, and map the time information encoding to a high-dimensional space, and then input the masked second medical image and the time information encoding into the second initial encoder in the initial prediction model for encoding processing to obtain the second encoding feature.

[0113] Optionally, in order to enhance the understanding of the training model to the time information encoding, in the training process of the prediction model, RETFound Plus adopts sinusoidal timestep embedding to map the time difference into a high-dimensional representation, and transforms it through a multi-layer perception (MLP) to align it with the hidden dimension (HiddenDimension) of the prediction model. This part of the time information encoding is encoded into the category label (CLS), so that the prediction model can effectively capture the characteristics of the fundus image changing over time, and improve the accuracy of disease progression prediction.

[0114] S5022, input the second encoding feature into the second initial decoder for decoding processing to obtain the second decoding feature.

[0115] Among them, the second decoding feature refers to all the features of the second medical image inferred from the second encoding feature.

[0116] In this embodiment, after obtaining the second encoding feature, the second encoding feature can be input into the second initial decoder for decoding processing to infer all the features of the second medical image, i.e. the second decoding feature.

[0117] S5023, according to the second difference between the features of the third image and the second decoding feature, training the second initial encoder and the second initial decoder to obtain the prediction model.

[0118] In this embodiment, after obtaining the features of the third image and the second decoding feature, the second difference between the features of the third image and the second decoding feature is determined, and whether the second difference satisfies the second preset difference threshold is determined. If the second difference does not satisfy the second preset difference threshold, the model parameters of the second initial encoder and the second initial decoder are adjusted to obtain a new second initial encoder and a new second initial decoder, and the step of inputting the masked second image and the time information difference into the second initial encoder for encoding processing to obtain the second encoding feature is returned. Perform until the second difference satisfies the second preset difference threshold to obtain the prediction model.

[0119] Optionally, the second difference value can be obtained by comparing the cosine similarity between the features of the third image and the second decoding features.

[0120] In this embodiment, the second initial encoder and the second initial decoder in the initial prediction model are trained, so that the prediction model after training can have a disease prediction function; in addition, the training method of the prediction model does not need to pre-label the images, only needs to perform partial mask processing on the images, and then inputs the masked images into the second initial encoder and the second initial decoder in sequence, so that the prediction model can be trained, which greatly reduces the workload of image labeling and improves the training efficiency of the prediction model.

[0121] In one embodiment, referring to Figure 8 A training method of a prediction model is also provided, comprising:

[0122] S10, performing first mask processing on the first medical images in the first medical image set to obtain the masked first medical images;

[0123] S11, inputting the masked first medical images into the first initial encoder for encoding processing to obtain first encoding features;

[0124] S12, inputting the first encoding features into the first initial decoder for decoding processing to obtain first decoding features;

[0125] S13, training the first initial encoder and the first initial decoder according to a first difference value between the image features of the first medical images and the first decoding features to obtain a trained base model;

[0126] S14, inputting a third medical image in the second medical image set into the trained base model for feature extraction to obtain features of the third medical image;

[0127] S15, performing second mask processing on the second medical image to obtain a masked second medical image; wherein the third medical image and the second medical image are medical images of the same object collected at different time points;

[0128] S16, inputting the masked second medical image, and a time interval between the second medical image and the third medical image into the second initial encoder for encoding processing to obtain second encoding features;

[0129] S17, inputting the second encoding features into the second initial decoder for decoding processing to obtain second decoding features;

[0130] S18, training the second initial encoder and the second initial decoder according to a second difference value between the features of the third image and the second decoding features to obtain a prediction model.

[0131] It should be noted that the description in S1-T18 above can refer to the relevant description in the above embodiments, and the effect is similar, and this embodiment will not be repeated here.

[0132] In one embodiment, to accommodate the downstream tasks of disease risk and progression prediction, after the training of the prediction model (i.e., RETFound Plus) is completed, the prediction model also needs to be fine-tuned by a pseudo-survival model. Each data point is represented as a triple (x, t, e), where x is the baseline retinal image, t is the event occurrence time or censoring time, and e is the event indicator (1 if the event occurs before censoring, otherwise 0). The model estimates the survival probability function S(t), which represents the probability that the event time exceeds t. This survival function is modeled using a log-logistic distribution, which contains patient-specific location parameter μ and scale parameter σ. These parameters are obtained by processing the high-dimensional representation output by the model encoder through a fully connected layer.

[0133] It should be noted that the fine-tuning process is optimized by a negative log-likelihood loss function, which is specifically designed to handle censored data. The expression of the survival function S(t) is in the form of a log-logistic distribution, and its corresponding log-likelihood function is used to estimate the parameters μ and σ. The loss function combines the likelihood of uncensored events and the survival probability of censored observations, thereby taking into account both censored and uncensored data.

[0134] Finally, the performance of the model is evaluated by two main indicators: Harrell’s concordance index (c-index) and integrated Brier score (IBS). The c-index measures the model’s ability to rank patients by their event risk, while the IBS assesses the calibration and discrimination ability of the predicted survival probability. Both indicators are adjusted for censoring by inverse probability weighting and calculated over a range of up to 10 years of follow-up time, consistent with the longest event time of the dataset. Finally, RETFound Plus is tested on multiple external datasets to evaluate its generalization ability in different populations and disease cohorts. These datasets include multi-ethnic cohorts from different geographic regions, ensuring the robustness of the model in diverse clinical environments. The predictive performance of the model is compared with existing baseline models (such as MAE and the original RETFound model), further verifying its superiority.

[0135] Optionally, the prediction model can be fine-tuned on the SDPP cohort. The SDPP cohort is a community-based longitudinal cohort comprising 79,284 participants who underwent physical examinations at the East China Sanatorium and the Sixth People's Hospital of Shanghai between December 2015 and November 2022. Baseline data included demographic information, anthropometric indices, biochemical measurements, and retinal images. After the baseline survey, 25,231 participants completed annual follow-ups for at least 4 years. DR grading and diagnosis of DME were based on macular and disc-centered fundus images of each eye at baseline and during follow-up.

[0136] In addition, external validation includes multi-country datasets, such as:

[0137] (1) CUHK-STDR cohort: A prospective observational study comprising 337 diabetic patients, participants were recruited from the Hong Kong Chinese University Eye Centre between July 2015 and November 2016, and followed up for at least 5 years consecutively;

[0138] (2) SEED cohort: A multi-ethnic longitudinal population study including Singaporean Malay, Indian, and Chinese adults, a total of 1,699 diabetic patients completed a 5-year follow-up;

[0139] (3) OUH cohort: A retrospective hospital cohort comprising 535 patients at the University Hospital of Odense, Denmark, between 2015 and 2022;

[0140] (4) AREDS cohort: A randomized, placebo-controlled clinical trial based on 11 retinal specialist clinics in the United States, recruiting 4,757 participants between November 1992 and November 2005;

[0141] (5) MEH-AlzEye cohort: A retrospective cohort study linking ophthalmic data of 353,157 patients who visited the Moorfields Eye Hospital between 2008 and 2018 with hospitalization health data across England.

[0142] It should be understood that although each step in the flowchart involved in each embodiment as described above is shown in sequence according to the direction of the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless explicitly stated otherwise herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowchart involved in each embodiment as described above can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least some of the other steps or the steps or stages in other steps.

[0143] Based on the same inventive concept, the embodiments of the present application also provide a training device for the prediction model of the training method of the prediction model. The implementation scheme of the device for solving the problem is similar to the implementation scheme described in the above method, so the specific limitations in one or more prediction model training device embodiments provided below can refer to the limitations of the prediction model training method in the above, which will not be described here.

[0144] In one exemplary embodiment, as shown in Figure 9 A training device for a prediction model is provided, comprising: a first training module 10 and a second training module 11, wherein:

[0145] The first training module 10 is configured to train an initial base model according to a first medical image set to obtain a trained base model, and the trained base model is configured to extract medical features of medical images. The first medical image set includes a plurality of first medical images.

[0146] The second training module 11 is configured to train an initial prediction model based on the trained base model and a second medical image set to obtain a prediction model, and the prediction model is configured to predict diseases. The second medical image set includes a second medical image collected at a first time and a third medical image collected at a second time for the same object, and a time interval between the first time and the second time.

[0147] In one exemplary embodiment, the first training module 10 described above comprises a processing unit and a first training unit, wherein:

[0148] The processing unit is specifically configured to perform first mask processing on the first medical images in the first medical image set to obtain masked first medical images.

[0149] The first training unit is specifically configured to train the initial base model according to the first medical images and the masked first medical images to obtain the trained base model.

[0150] In one exemplary embodiment, the initial base model includes a first initial encoder and a first initial decoder, and the first training unit is specifically configured to input the masked first medical images into the first initial encoder for encoding processing to obtain first encoding features, input the first encoding features into the first initial decoder for decoding processing to obtain first decoding features, and train the first initial encoder and the first initial decoder according to a first difference between the image features of the first medical images and the first decoding features to obtain the trained base model.

[0151] In an example embodiment, the second training module 11 comprises an extraction unit and a second training unit, wherein:

[0152] The extraction unit is specifically configured to input a third medical image in the second medical image set into the trained base model to perform feature extraction, to obtain a feature of the third medical image;

[0153] The second training unit is specifically configured to train the initial prediction model according to the feature of the third medical image and the second medical image, to obtain the prediction model.

[0154] In an example embodiment, the second training unit is specifically further configured to perform second mask processing on the second medical image to obtain a masked second medical image; and train the initial prediction model according to the masked second medical image, the feature of the third medical image, and a time interval between the first time and the second time, to obtain the prediction model.

[0155] In an example embodiment, the initial prediction model comprises a second initial encoder and a second initial decoder, and the second training unit is specifically further configured to input the masked second medical image and the time interval into the second initial encoder to perform encoding processing, to obtain a second encoding feature; input the second encoding feature into the second initial decoder to perform decoding processing, to obtain a second decoding feature; and train the second initial encoder and the second initial decoder according to a second difference between the feature of the third image and the second decoding feature, to obtain the prediction model.

[0156] In an example embodiment, the initial base model is a mask autoencoder (MAE) model, and the initial prediction model is a RETFound model.

[0157] The modules in the training device of the prediction model can be realized by software, hardware, and combinations thereof, in whole or in part. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in a computer device in software form, to be called and executed by a processor to perform operations corresponding to the modules.

[0158] In an example embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the following steps:

[0159] training an initial base model according to a first medical image set to obtain a trained base model; the trained base model is used to extract medical features of medical images; the first medical image set comprises a plurality of first medical images;

[0160] The initial prediction model is trained based on the trained base model and the second medical image set to obtain a prediction model; the prediction model is used for predicting the disease; the second medical image set comprises: a second medical image collected at a first time, a third medical image collected at a second time, and a time interval between the first time and the second time for the same object.

[0161] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0162] The first medical image in the first medical image set is subjected to first mask processing to obtain a masked first medical image.

[0163] The initial base model is trained according to the first medical image and the masked first medical image to obtain a trained base model.

[0164] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0165] The masked first medical image is input into the first initial encoder for encoding processing to obtain first encoding features.

[0166] The first encoding features are input into the first initial decoder for decoding processing to obtain first decoding features.

[0167] The first initial encoder and the first initial decoder are trained according to the first difference between the image features of the first medical image and the first decoding features to obtain the trained base model.

[0168] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0169] The third medical image in the second medical image set is input into the trained base model for feature extraction to obtain features of the third medical image.

[0170] The initial prediction model is trained according to the features of the third medical image and the second medical image to obtain a prediction model.

[0171] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0172] The second medical image is subjected to second mask processing to obtain a masked second medical image.

[0173] The initial prediction model is trained according to the masked second medical image, the features of the third medical image, and the time interval between the first time and the second time to obtain a prediction model.

[0174] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0175] The second medical image after the mask and the time interval are input into a second initial encoder for encoding processing to obtain second encoding features;

[0176] The second encoding features are input into a second initial decoder for decoding processing to obtain second decoding features;

[0177] The second initial encoder and the second initial decoder are trained according to a second difference between the features of the third image and the second decoding features to obtain a prediction model.

[0178] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0179] The initial base model is a mask autoencoder (MAE) model, and the initial prediction model is a RETFound model.

[0180] In one embodiment, a computer-readable storage medium is provided, and the computer-readable storage medium stores a computer program. The computer program, when executed by a processor, implements the following steps:

[0181] The initial base model is trained according to the first medical image set to obtain a trained base model. The trained base model is used to extract medical features of medical images. The first medical image set includes a plurality of first medical images.

[0182] The initial prediction model is trained based on the trained base model and the second medical image set to obtain a prediction model. The prediction model is used to predict diseases. The second medical image set includes: a second medical image collected at a first time, a third medical image collected at a second time, and a time interval between the first time and the second time for the same object.

[0183] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0184] The first medical image in the first medical image set is subjected to first mask processing to obtain a first medical image after the mask.

[0185] The initial base model is trained according to the first medical image and the first medical image after the mask to obtain a trained base model.

[0186] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0187] The first medical image after the mask is input into a first initial encoder for encoding processing to obtain first encoding features.

[0188] inputting the first encoded feature into a first initial decoder for decoding processing to obtain a first decoded feature;

[0189] training the first initial encoder and the first initial decoder according to a first difference value between the image feature of the first medical image and the first decoded feature to obtain a trained base model.

[0190] In an embodiment, the computer program, when executed by the processor, further implements the following steps:

[0191] inputting a third medical image in the second medical image set into the trained base model for feature extraction to obtain a feature of the third medical image;

[0192] training the initial prediction model according to the feature of the third medical image and the second medical image to obtain a prediction model.

[0193] In an embodiment, the computer program, when executed by the processor, further implements the following steps:

[0194] performing second mask processing on the second medical image to obtain a masked second medical image;

[0195] training the initial prediction model according to the masked second medical image, the feature of the third medical image, and a time interval between the first time and the second time to obtain the prediction model.

[0196] In an embodiment, the computer program, when executed by the processor, further implements the following steps:

[0197] inputting the masked second medical image and the time interval into a second initial encoder for encoding processing to obtain a second encoded feature;

[0198] inputting the second encoded feature into a second initial decoder for decoding processing to obtain a second decoded feature;

[0199] training the second initial encoder and the second initial decoder according to a second difference value between the feature of the third image and the second decoded feature to obtain the prediction model.

[0200] In an embodiment, the computer program, when executed by the processor, further implements the following steps:

[0201] The initial base model is a mask autoencoder (MAE) model, and the initial prediction model is a RETFound model.

[0202] In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the following steps:

[0203] According to the first medical image set, the initial base model is trained to obtain a trained base model; the trained base model is used to extract medical features of medical images; the first medical image set includes a plurality of first medical images;

[0204] Based on the trained base model and the second medical image set, the initial prediction model is trained to obtain a prediction model; the prediction model is used to predict diseases; the second medical image set includes: a second medical image collected at a first time, a third medical image collected at a second time, and a time interval between the first time and the second time for the same object.

[0205] In one embodiment, the computer program is executed by the processor to further implement the following steps:

[0206] The first medical image in the first medical image set is subjected to first mask processing to obtain a masked first medical image;

[0207] According to the first medical image and the masked first medical image, the initial base model is trained to obtain a trained base model.

[0208] In one embodiment, the computer program is executed by the processor to further implement the following steps:

[0209] The masked first medical image is input into a first initial encoder for encoding processing to obtain first encoding features;

[0210] The first encoding features are input into a first initial decoder for decoding processing to obtain first decoding features;

[0211] According to a first difference between the image features of the first medical image and the first decoding features, the first initial encoder and the first initial decoder are trained to obtain a trained base model.

[0212] In one embodiment, the computer program is executed by the processor to further implement the following steps:

[0213] The third medical image in the second medical image set is input into the trained base model for feature extraction to obtain features of the third medical image;

[0214] According to the features of the third medical image and the second medical image, the initial prediction model is trained to obtain a prediction model.

[0215] In one embodiment, the computer program is executed by the processor to further implement the following steps:

[0216] The second medical image is subjected to second mask processing to obtain a masked second medical image;

[0217] The initial prediction model is trained according to the features of the second medical image and the third medical image after the mask, and the time interval between the first time and the second time, to obtain a prediction model.

[0218] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0219] The second medical image after the mask and the time interval are input into a second initial encoder for encoding processing to obtain second encoding features.

[0220] The second encoding features are input into a second initial decoder for decoding processing to obtain second decoding features.

[0221] The second initial encoder and the second initial decoder are trained according to a second difference value between the features of the third image and the second decoding features, to obtain the prediction model.

[0222] In one embodiment, the computer program, when executed by the processor, further implements the following steps:

[0223] The initial base model is a mask autoencoder (MAE) model, and the initial prediction model is a RETFound model.

[0224] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0225] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist, they should be considered as the scope of the present application.

[0226] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A method for training a prediction model, characterized in that, The method includes: Based on a first set of medical images, an initial pedestal model is trained to obtain a trained pedestal model; the trained pedestal model is used to extract medical features from the medical images; the first set of medical images includes multiple first medical images. The third medical image in the second medical image set is input into the trained base model for feature extraction to obtain the features of the third medical image; the second medical image set includes: a second medical image acquired at a first time moment for the same object, a third medical image acquired at a second time moment, and the time interval between the first time moment and the second time moment; The second medical image is subjected to a second masking process to obtain a masked second medical image; Based on the features of the masked second medical image and the third medical image, as well as the time interval between the first time and the second time, the initial prediction model is trained to obtain a prediction model; the prediction model is used to predict diseases.

2. The method according to claim 1, characterized in that, The step of training the initial pedestal model based on the first medical image set to obtain the trained pedestal model includes: The first medical image in the first medical image set is subjected to a first masking process to obtain the masked first medical image. The initial pedestal model is trained based on the first medical image and the masked first medical image to obtain the trained pedestal model.

3. The method according to claim 2, characterized in that, The initial base model includes a first initial encoder and a first initial decoder. The step of training the initial base model based on the first medical image and the masked first medical image to obtain the trained base model includes: The masked first medical image is input into the first initial encoder for encoding processing to obtain the first encoded feature; The first encoded feature is input into the first initial decoder for decoding to obtain the first decoded feature; Based on the first difference between the image features of the first medical image and the first decoding features, the first initial encoder and the first initial decoder are trained to obtain the trained pedestal model.

4. The method according to claim 1, characterized in that, The initial prediction model includes a second initial encoder and a second initial decoder. The initial prediction model is trained based on the features of the masked second medical image and the third medical image, as well as the time interval between the first and second time points, to obtain the prediction model. This includes: The masked second medical image and the time interval are input into the second initial encoder for encoding processing to obtain the second encoded feature; The second encoded feature is input into the second initial decoder for decoding processing to obtain the second decoded feature; The second initial encoder and the second initial decoder are trained based on the second difference between the features of the third medical image and the second decoded features to obtain the prediction model.

5. The method according to any one of claims 1-3, characterized in that, The initial base model is a masked autoencoder (MAE) model, and the initial prediction model is a RETFound model.

6. A training device for a prediction model, characterized in that, The device includes: The first training module is used to train the initial base model based on the first medical image set to obtain the trained base model; the trained base model is used to extract medical features from the medical images; the first medical image set includes multiple first medical images. The second training module is used to input a third medical image from the second medical image set into the trained base model for feature extraction to obtain the features of the third medical image. The second medical image set includes: a second medical image acquired at a first time moment for the same object, a third medical image acquired at a second time moment, and the time interval between the first time moment and the second time moment. The second medical image is subjected to a second masking process to obtain a masked second medical image. Based on the features of the masked second medical image, the third medical image, and the time interval between the first time moment and the second time moment, an initial prediction model is trained to obtain a prediction model. The prediction model is used to predict diseases.

7. The apparatus according to claim 6, characterized in that, The first training module includes: The processing unit is specifically used to perform a first masking process on the first medical image in the first medical image set to obtain the masked first medical image. The first training unit is specifically used to train the initial pedestal model based on the first medical image and the masked first medical image to obtain the trained pedestal model.

8. The apparatus according to claim 7, characterized in that, The initial pedestal model includes a first initial encoder and a first initial decoder. The first training unit is specifically used to input the masked first medical image into the first initial encoder for encoding processing to obtain a first encoded feature; input the first encoded feature into the first initial decoder for decoding processing to obtain a first decoded feature; and train the first initial encoder and the first initial decoder according to the first difference between the image features of the first medical image and the first decoded feature to obtain the trained pedestal model.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images