Fundus image analysis method and apparatus, computer device, readable storage medium, and program product

Abstract

A fundus image analysis method and apparatus, a computer device, a computer-readable storage medium, and a computer program product. The method comprises: acquiring a fundus image of a subject, and, on the basis of whether a preset event occurred in the brain of the subject within a historical time period, determining historical brain information of the subject; when the historical brain information indicates that the subject did not experience the preset event within the historical time period, determining a first fundus image analysis result on the basis of the fundus image and a target fundus image analysis model pre-trained on the basis of a semi-supervised learning algorithm and a knowledge transfer algorithm. The first fundus image analysis result is used for representing whether the subject has a hidden abnormal brain region, and comprises first prediction information for representing whether the subject will experience the preset event within a preset time period.

Description

Fundus image analysis methods and apparatus, computer equipment, readable storage media and program products

[0001] Related applications

[0002] This application claims priority to Chinese patent application filed on December 27, 2024, with application number 202411958605.1, entitled "Method, Apparatus, Computer Equipment, Readable Storage Medium and Program Product for Fundus Image Analysis", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of data processing technology, and in particular to a method and apparatus for fundus image analysis, a computer device, a computer-readable storage medium, and a computer program product. Background Technology

[0004] With the aging of the global population and changes in lifestyle, the incidence of brain abnormalities is rising year by year, becoming a significant public health problem leading to death and disability. Therefore, early identification of high-risk groups for effective intervention is particularly important.

[0005] In existing technologies, most brain abnormalities are identified and predicted by imaging the user's brain using MRI or CT scans.

[0006] However, this method of identifying and predicting brain abnormalities is costly, making it impossible to identify and predict brain abnormalities in large populations. Summary of the Invention

[0007] Therefore, it is necessary to provide a low-cost fundus image analysis method and apparatus, computer equipment, computer-readable storage medium and computer program product that can identify and predict brain abnormalities in a large population, in order to address the above-mentioned technical problems.

[0008] In a first aspect, this application provides a method for analyzing fundus images, including:

[0009] Acquire fundus images of the user and determine the user's historical brain information based on whether a preset event has occurred in the user's brain during a historical time period;

[0010] If the historical brain information indicates that the user has not experienced the preset event within the historical time period, the first fundus image analysis result is determined based on the fundus image and the target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm; wherein, the first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain region, and includes first predictive information characterizing whether the user will experience the preset event within the preset time period.

[0011] In one embodiment, the method further includes: when the historical brain information indicates that the user has experienced the preset event within a historical time period, determining a second fundus image analysis result based on the fundus image and the target fundus image analysis model; wherein the second fundus image analysis result includes second prediction information, which is used to characterize whether the user will experience the preset event again within the preset time period.

[0012] In one embodiment, the target fundus image analysis model includes a first model and a second model. The determination of the first fundus image analysis result based on the fundus image and the target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm includes: inputting the fundus image into the first model to obtain the first fundus image analysis result output by the first model.

[0013] In one embodiment, the target fundus image analysis model includes a first model and a second model. Determining the second fundus image analysis result based on the fundus image and the target fundus image analysis model includes: inputting the fundus image into the second model to obtain the second fundus image analysis result output by the second model.

[0014] In one embodiment, acquiring the user's fundus image includes: acquiring a first fundus image of the user; if the retinal region in the first fundus image meets preset conditions, determining the first fundus image as a second fundus image; and performing image enhancement processing on the second fundus image to obtain the fundus image.

[0015] In one embodiment, the training method for the target fundus image analysis model includes: acquiring training data; the training data includes fundus images of users with brain abnormalities that have the concealment, fundus images of users without brain abnormalities that have not had the concealment, fundus images of users who have not experienced the preset event in a historical time period, fundus images of users who have experienced the preset event in a historical time period, and fundus images of random users; training an initial fundus image analysis model based on the training data, the semi-supervised learning algorithm, and the knowledge transfer algorithm to obtain the first model; adjusting the first model to obtain the second model, and determining the target fundus image analysis model based on the first model and the second model.

[0016] In one embodiment, before training the initial fundus image analysis model based on the training data, the semi-supervised learning algorithm, and the knowledge transfer algorithm, the method further includes: pre-training and initializing the initial fundus image analysis model.

[0017] In one embodiment, the historical brain information includes first information and second information, wherein the first information is used to characterize that the user's brain did not experience the preset event during the historical time period, and the second information is used to characterize that the user's brain experienced the preset event during the historical time period.

[0018] In one embodiment, the second prediction information is further used to characterize the predicted probability that the user will re-occur on the preset event within the preset time period.

[0019] In one embodiment, image enhancement processing is performed on the second fundus image, including: converting the second fundus image from the RGB color space to the LAB color space; segmenting the second fundus image into image blocks of fixed size; enhancing the luminance channel of each image block using contrast-limited adaptive histogram equalization; and converting the enhanced second fundus image back to RGB format.

[0020] In one embodiment, the image enhancement processing of the second fundus image is as follows:

[0021] P = α·P + β·Gauss(P,s) + δ

[0022] Where P refers to the second fundus image, Gauss(P,s) means applying a Gaussian filter with a standard deviation of s to the second fundus image P, and α, β, δ are adjustable parameters.

[0023] Secondly, this application also provides a fundus image analysis device, including: an acquisition module and an execution module.

[0024] The acquisition module is used to acquire fundus images of the user and determine the user's historical brain information based on whether a preset event has occurred in the user's brain during a historical time period.

[0025] The execution module is used to determine the first fundus image analysis result based on the fundus image and the target fundus image analysis model pre-trained based on the semi-supervised learning algorithm and knowledge transfer algorithm, when the historical brain information indicates that the user has not experienced the preset event within the historical time period.

[0026] The results of the first fundus image analysis are used to characterize whether the user has a hidden abnormal brain region, and include first predictive information indicating whether the user will experience the preset event within a preset time period.

[0027] Thirdly, this application also provides a computer device, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the method described in any of the embodiments of the first aspect above.

[0028] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the embodiments of the first aspect above.

[0029] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any of the embodiments of the first aspect above.

[0030] The aforementioned fundus image analysis method, apparatus, computer equipment, computer-readable storage medium, and computer program product first acquire a user's fundus image and determine the user's historical brain information based on whether a preset event has occurred in the user's brain within a historical time period. If the historical brain information indicates that the preset event has not occurred in the user's historical time period, a first fundus image analysis result is determined based on the fundus image and a target fundus image analysis model pre-trained using a semi-supervised learning algorithm and a knowledge transfer algorithm. This first fundus image analysis result is used to characterize whether the user has a hidden brain abnormality region and includes first predictive information indicating whether the preset event will occur in the user within a preset time period. The fundus image analysis method provided in this application can determine whether a user has a hidden brain abnormality region and provide first predictive information indicating whether the preset event will occur in the user within a preset time period using the user's fundus image and historical brain information. Because the cost of acquiring fundus images is relatively low, it can achieve the identification and prediction of brain abnormalities in a large population. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 is a schematic flowchart of a fundus image analysis method in one embodiment of this application;

[0033] Figure 2 is a flowchart illustrating a method for acquiring a user's fundus image in one embodiment of this application;

[0034] Figure 3 is a flowchart illustrating the training method of the target fundus image analysis model in one embodiment of this application;

[0035] Figure 4 is a flowchart illustrating a fundus image analysis method in another embodiment of this application;

[0036] Figure 5 is a structural block diagram of a fundus image analysis device in one embodiment of this application;

[0037] Figure 6 is an internal structural diagram of a computer device according to one embodiment of this application;

[0038] Figure 7 is an internal structural diagram of a computer device according to another embodiment of this application;

[0039] Figure 8 is a schematic diagram of the training process of the target fundus image analysis model in one embodiment of this application. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0041] With the aging of the global population and changes in lifestyle, the incidence of brain abnormalities is rising year by year, becoming a significant public health problem leading to death and disability. Therefore, early identification of high-risk groups for effective intervention is particularly important.

[0042] In existing technologies, most brain abnormalities are identified and predicted by imaging the user's brain using magnetic resonance imaging (MRI) or computed tomography (CT).

[0043] However, this method of identifying and predicting brain abnormalities is costly, making it impossible to identify and predict brain abnormalities in large populations.

[0044] In view of this, this application provides a fundus image analysis method. Since fundus images contain rich brain information and the cost of acquiring fundus images is low, it is possible to identify and predict brain abnormalities in a large population.

[0045] The fundus image analysis method provided in this application can be executed by a computer device, which can be a terminal or a server.

[0046] In an exemplary embodiment, as shown in FIG1, a fundus image analysis method is provided, which includes steps 101 and 102.

[0047] Step 101: Obtain the user's fundus image and determine the user's historical brain information based on whether a preset event has occurred in the user's brain during the historical time period.

[0048] Optionally, the fundus image refers to an image that can be used to characterize the structural and functional information of the retina, optic nerve, and blood vessels.

[0049] For example, non-contact imaging techniques can be used to acquire images of the user's fundus. Specifically, images of the user's fundus can be acquired using a fundus camera or optical coherence tomography (OCT).

[0050] This historical period can be pre-set by technical personnel according to actual needs. For example, the historical period can be, but is not limited to, the past ten years or the past five years.

[0051] The preset event can be a brain abnormality event. This brain abnormality event can include brain structural abnormalities, brain neurological dysfunction, brain electrophysiological abnormalities, brain metabolic abnormalities, or brain blood circulation abnormalities. Brain structural abnormalities further include brain tumor events, cerebral infarction events, cerebral hemorrhage events, or brain atrophy events. In this embodiment, the preset event is a cerebral infarction event, also known as a stroke event, within the context of brain structural abnormalities.

[0052] For example, the historical brain information can be used to characterize whether a preset event occurred within a historical time period. This historical brain information may include first information and second information, whereby the first information characterizes that no preset event occurred in the user's brain within the historical time period, and the second information characterizes that a preset event occurred in the user's brain within the historical time period.

[0053] Furthermore, if it is determined that no preset event has occurred in the user's brain during the historical time period, then the historical brain information can be determined as the first information; if it is determined that a preset event has occurred in the user's brain during the historical time period, then the historical brain information can be determined as the second information.

[0054] Step 102: If the historical brain information indicates that the user has not experienced the preset event within the historical time period, determine the first fundus image analysis result based on the fundus image and the target fundus image analysis model trained in advance based on semi-supervised learning algorithm and knowledge transfer algorithm.

[0055] The results of the first fundus image analysis are used to characterize whether the user has a hidden abnormal brain region, and include first predictive information indicating whether the user will experience the preset event within a preset time period.

[0056] Optionally, the target fundus image analysis model can be the DeepRETStroke model.

[0057] In the embodiments of this application, the region of concealed brain abnormality refers to the region to which a concealed cerebral infarction belongs.

[0058] The first prediction information can be a prediction result that characterizes whether the user will experience the preset event within a preset time period, or it can be a prediction probability that the user will experience the preset event within a preset time period.

[0059] In some exemplary embodiments, as described above, the historical brain information may include first information and second information. When the historical brain information is the first information, that is, when no preset event has occurred in the user's brain during the historical time period, the fundus image can be directly input into the target fundus image analysis model to obtain the user's first fundus image analysis result output by the target fundus image analysis model.

[0060] In some other exemplary embodiments, the target fundus image analysis model may also include a first model and a second model. When the historical brain information is the first information, that is, when no preset event has occurred in the user's brain during the historical time period, the fundus image can be directly input into the first model to obtain the first fundus image analysis result of the user output by the first model.

[0061] The aforementioned fundus image analysis method first acquires the user's fundus image and determines the user's historical brain information based on whether a preset event has occurred in the user's brain within a historical time period. If the historical brain information indicates that the preset event has not occurred in the user's historical time period, a first fundus image analysis result is determined based on the fundus image and a target fundus image analysis model pre-trained using a semi-supervised learning algorithm and a knowledge transfer algorithm. This first fundus image analysis result is used to characterize whether the user has a hidden brain abnormality region and includes first predictive information indicating whether the preset event will occur in the user within a preset time period. The fundus image analysis method provided in this application can determine whether a user has a hidden brain abnormality region and provide first predictive information indicating whether the preset event will occur in the user within a preset time period using the user's fundus image and historical brain information. Since the acquisition cost of fundus images is low, it can achieve the identification and prediction of brain abnormalities in a large-scale population.

[0062] Furthermore, the target fundus image analysis model used in this application is trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm, which makes the accuracy of its output fundus image analysis results higher.

[0063] Moreover, compared to most existing technologies where fundus image analysis results only include first predictive information characterizing whether a user will experience a preset event within a preset time period, the fundus image analysis results of this application also include analysis results on whether the user has hidden abnormal brain regions.

[0064] In an exemplary embodiment, the method further includes: determining a second fundus image analysis result based on the fundus image and the target fundus image analysis model when the historical brain information is second information, that is, the historical brain information indicates that the user has experienced the preset event within a historical time period.

[0065] The second fundus image analysis result includes second prediction information, which is used to characterize whether the user will experience the preset event again within a preset time period.

[0066] Optionally, the second prediction information can also be used to characterize the predicted probability that the user will experience the preset event again within a preset time period.

[0067] In some exemplary embodiments, as described above, the historical brain information may include first information and second information. When the historical brain information is the second information, that is, when a preset event has occurred in the user's brain during a historical time period, the fundus image can be directly input into the target fundus image analysis model to obtain the second fundus image analysis result of the user output by the target fundus image analysis model.

[0068] In some other exemplary embodiments, the target fundus image analysis model may also include a first model and a second model. When the historical brain information is the second information, that is, when the user's brain has experienced a preset event during the historical time period, the fundus image can be directly input into the second model to obtain the second fundus image analysis result of the user output by the second model.

[0069] In an exemplary embodiment, the target fundus image analysis model includes a first model and a second model. Step 102, when the historical brain information indicates that the user has not experienced the preset event within the historical time period, determines the first fundus image analysis result based on the fundus image and the target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm. This includes: when the historical brain information indicates that the user has not experienced the preset event within the historical time period, inputting the fundus image into the first model to obtain the first fundus image analysis result output by the first model.

[0070] In some exemplary embodiments, if it is determined from the user's historical brain information that the user has not experienced the preset event within the historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

[0071] In an exemplary embodiment, when the historical brain information indicates that the user has experienced the preset event within a historical time period, determining the second fundus image analysis result based on the fundus image and the target fundus image analysis model includes: when the historical brain information indicates that the user has experienced the preset event within a historical time period, inputting the fundus image into the second model to obtain the second fundus image analysis result output by the second model.

[0072] In some exemplary embodiments, if it is determined from the user's historical brain information that the user has experienced the preset event within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

[0073] In an exemplary embodiment, as shown in FIG2, the acquisition of the user's fundus image includes the following steps 201 and 202.

[0074] Step 201: Obtain the user's first fundus image. If the retinal region in the first fundus image meets the preset conditions, determine the first fundus image as the second fundus image.

[0075] Optionally, the preset condition can be pre-set by a technician according to actual needs. For example, the preset condition could be that at least 75% of the retinal area in the first fundus image is clearly observable. The preset condition could also be that there are no artifacts in the retinal area of ​​the first fundus image.

[0076] In some exemplary embodiments, after acquiring a user's first fundus image using non-contact imaging technology, it can be determined whether the first fundus image meets preset conditions.

[0077] Furthermore, if the first fundus image is determined to meet the preset conditions, the first fundus image is determined as the second fundus image.

[0078] Specifically, if at least 75% of the retinal area is clearly observable in the first fundus image, and there are no artifacts in the retinal area of ​​the first fundus image, then the first fundus image is identified as the second fundus image.

[0079] Step 202: Perform image enhancement processing on the second fundus image to obtain the fundus image.

[0080] In some exemplary embodiments, Contrast Limited Adaptive Histogram Equalization (CLAHE) can be used to enhance the contrast of the second fundus image while suppressing noise. Specifically, the second fundus image can be converted from the RGB color space to the LAB color space and divided into fixed-size image blocks. CLAHE is used to enhance the luminance channel of each image block, and the enhanced second fundus image is then converted back to RGB format.

[0081] Furthermore, color normalization is used to reduce the impact of shooting equipment and shooting environment on the second fundus image.

[0082] Specifically, in one embodiment of this application, the processing of the second fundus image P is as follows: P = α·P + β·Gauss(P,s) + δ, where P refers to the second fundus image, Gauss(P,s) refers to applying a Gaussian filter with a standard deviation of s to the second fundus image P, and α, β, and δ are adjustable parameters. For example, α = 4, β = -4, δ = 128, and s = 5, and the resolution of each fundus image is adjusted to 512×512.

[0083] In an exemplary embodiment, as shown in FIG3, the training method of the target fundus image analysis model includes the following steps 301 to 303.

[0084] Step 301: Obtain training data.

[0085] The training data includes fundus images of users with concealed brain abnormalities, fundus images of users without concealed brain abnormalities, fundus images of users who have not experienced the preset event in the historical time period, fundus images of users who have experienced the preset event in the historical time period, and fundus images of random users.

[0086] The random user is a user whose brain abnormality is uncertain, whether it is hidden, and whether the preset event has occurred in the historical time period.

[0087] Step 302: Based on the training data, the semi-supervised learning algorithm, and the knowledge transfer algorithm, train the initial fundus image analysis model to obtain the first model.

[0088] The first model can be used to predict whether a user will experience a preset event within a preset time period, and it can also be used to predict the probability of a user experiencing a preset event within a preset time period.

[0089] Step 303: Adjust the first model to obtain the second model, and determine the target fundus image analysis model based on the first model and the second model.

[0090] The second model can be used to predict whether a user will experience the preset event again within a preset time period, and it can also be used to predict the probability that a user will experience the preset event again within a preset time period.

[0091] In an optional embodiment of this application, as shown in FIG8, FIG8 is a schematic diagram of the training process of the target fundus image analysis model in an embodiment of this application. In step 302, the initial fundus image analysis model is trained based on the training data and the semi-supervised learning algorithm to obtain the first model. Before training the initial fundus image analysis model, the model pre-training and model initialization of the initial fundus image analysis model are performed, that is, the initial fundus image analysis model is pre-trained and initialized.

[0092] Specifically, the initial fundus image analysis model can adopt the architecture shown in Figure 8. The encoder of the initial fundus image analysis model can employ a large visual Transformer network architecture, which may include 24 Transformer modules with an embedding vector size of 1024; the decoder of the initial fundus image analysis model can employ a small visual Transformer network architecture, which may include 8 Transformer modules with an embedding vector size of 512.

[0093] In the process of pre-training the initial fundus image analysis model using the MAE algorithm, "fundus images of random users" were used as training data. Each image was divided into 16×16 patches, 75% of which were masked, and the remaining 25% were used as input to the model for training. The loss function in this process is...

[0094] Where N refers to the total number of training data, N p This refers to the total number of pixels that are not masked in each training data set, and C refers to the total number of image channels. s,k,q This refers to the predicted value of the model in the q-th channel of the k-th pixel in the training data numbered s, x s,k,q This refers to the true value of the q-th channel of the k-th pixel in the training data with pixel number s, where s = 1, 2, 3, ..., N; k = 1, 2, 3, ..., N p ;i=1,2,3,…,C.

[0095] Furthermore, the diagnostic head of the initial fundus image analysis model can employ two identical logistic regression classifiers. The learning head and prediction head can respectively employ multilayer perceptrons with 2D and 5D outputs. Using fundus images of users who have not experienced the preset event within a historical time period as training data, the model performs a learning task to predict the probability of the user experiencing the preset event within a preset time period, thereby updating the encoder and prediction head components. The loss function in this process is...

[0096] Where N refers to the total number of training data, t represents the year, and T i T represents the total number of years of follow-up for user number i. For users who experienced a preset event during the follow-up period or whose follow-up time reached the preset time interval but did not experience a preset event, T represents the total number of years of follow-up. i =5; If no preset event occurs during the follow-up period but the follow-up time is less than the preset time period, T i <5), x i This represents the fundus image of user i (i = 1, 2, 3, ..., N), p θ,t (x i y represents the probability that a predetermined event will occur before time point t. i,t This represents the user state of user i at time point t (users whose preset events occurred before time point t, y). i,t =1; For users who did not experience the preset event, y i,t =0).

[0097] Furthermore, the process of training a model based on semi-supervised learning and knowledge transfer algorithms can be viewed as a loop, with each loop cycle including two steps: "semi-supervised learning" and "knowledge transfer". Simultaneously, the "semi-supervised learning" in each loop cycle can be considered a sub-loop, with each sub-loop cycle including two steps: "labeled database update" and "diagnostic header update".

[0098] For example, in the "semi-supervised learning" step, the training data is reorganized into two new databases: a "labeled database" and an "unlabeled database." Before this sub-loop begins, the "labeled database" includes fundus images of all users with "concealed brain abnormalities" and "fundus images of users without concealed brain abnormalities," while the "unlabeled database" includes fundus images of all users who have not experienced the preset event within the historical time period. After the sub-loop begins, for each training round of a sub-loop, the "labeled database" is used first to update the model's diagnostic head by learning from the "concealed brain abnormality identification" task.

[0099] After the update, the model is used again to perform the "identification of hidden brain abnormalities" task on the "validation database" of "fundus images of users with hidden brain abnormalities" and "fundus images of users without hidden brain abnormalities," and the probability of a hidden brain abnormality corresponding to the fundus image of each user with a hidden brain abnormality is determined. Then, the minimum probability of the preset event occurring that maintains the prediction accuracy (the proportion of truly positive samples among those predicted as positive) of positive samples above 0.75 is selected as the high prediction confidence standard pos_thr for positive samples, and the maximum probability of the preset event occurring that maintains the prediction accuracy (the proportion of truly negative samples among those predicted as negative) of negative samples above 0.75 is selected as the high prediction confidence standard neg_thr for negative samples.

[0100] The model is then used to perform a task of "identifying concealed brain abnormalities" on an "unlabeled database." Training data with a predicted probability greater than pos_thr are labeled with positive "pseudo-labels," and training data with a predicted probability less than neg_thr are labeled with negative "pseudo-labels." These training data are then removed from the "unlabeled database" and added to the "labeled database." In the next sub-loop training round, this expanded "labeled database" is used to update the model's diagnostic head. In this way, samples from the "unlabeled database" are continuously added to the "labeled database" as the sub-loop progresses, until the "unlabeled database" has no samples or no samples with high predictive confidence that can be assigned "pseudo-labels." This sub-loop (the step of "training the initial fundus image analysis model based on training data and semi-supervised learning algorithms") ends.

[0101] For example, this model is used to perform a task on "concealed brain abnormalities" on "fundus images of users who have not experienced the preset event within a historical time period," and each training data point is assigned a "prediction probability" as a "soft label" to obtain a database with added "soft labels." Further, using this database with added "soft labels," the encoder, learning head, and prediction head in the model are updated through the model's learning of the tasks of "predicting the probability of a user experiencing the preset event within a preset time period" and "soft label prediction." The loss function in this process is...

[0102] The loss function consists of two parts. The first part is the same as the loss function in the "model initialization" phase, representing the prediction loss for the task of "predicting the probability of a user experiencing a preset event within a preset time period." The second part represents the prediction loss for the "soft tag prediction" task, where x... iThis represents the fundus image of user number i, where i = 1, 2, 3, ..., N. This represents the true probability distribution of the "soft label" in category j. This represents the predicted probability distribution output by the model for category j. α is a weighting parameter controlling the tasks of "soft label prediction" and "predicting the probability of a user experiencing a preset event within a preset time period," and α can be 0.3. Through this method, the model can simultaneously learn a user's brain state at baseline and during follow-up, thereby improving the accuracy of predicting the probability of a user experiencing a preset event within a preset time period.

[0103] After the update is completed, the model is used to perform a prediction task on the "validation database" of "fundus images of users who have not experienced the preset event within the historical time period" to "predict the probability of a user experiencing the preset event within the preset time period". When the model performance reaches the preset standard, it means that the first model has been built and the entire loop ends; otherwise, the next large loop update begins.

[0104] As mentioned above, after training the initial fundus image analysis model based on the training data and the semi-supervised learning algorithm to obtain the first model, the first model needs to be adjusted to obtain the second model, and the target fundus image analysis model is determined based on the first model and the second model.

[0105] The first model (especially the encoder part) can serve as a pre-trained model with strong prior knowledge, thus providing a certain advantage in training other brain abnormality prediction tasks. Therefore, this application uses the encoder of the first model as a pre-trained model, and on the basis of the first model, a second model is developed through fine-tuning using fundus images of users who have experienced the preset event within a historical time period. The loss function for this process is...

[0106] It can be observed that this loss function has the same form as the loss function in the previous "model initialization" step, the difference being that the observed event changes from the occurrence of the preset event to the recurrence of the preset event.

[0107] In an exemplary embodiment, as shown in FIG4, another fundus image analysis method is provided, which includes the following steps 401 to 403.

[0108] Step 401: Obtain the user's first fundus image. If the retinal region in the first fundus image meets the preset conditions, determine the first fundus image as the second fundus image. Perform image enhancement processing on the second fundus image to obtain the fundus image. Determine the user's historical brain information based on whether the user's brain has experienced a preset event during the historical time period.

[0109] Step 402: If the historical brain information indicates that the user has not experienced the preset event within the historical time period, the fundus image is input into a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm to obtain the first fundus image analysis result output by the first model; wherein, the first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain region, and includes first prediction information characterizing whether the user will experience the preset event within the preset time period.

[0110] Step 403: If the historical brain information indicates that the user has experienced the preset event within the historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model; wherein, the second fundus image analysis result includes second prediction information, which is used to characterize whether the user will experience the preset event again within the preset time period.

[0111] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0112] Based on the same inventive concept, this application also provides a fundus image analysis device for implementing the fundus image analysis method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more fundus image analysis device embodiments provided below can be found in the limitations of the fundus image analysis method described above, and will not be repeated here.

[0113] In an exemplary embodiment, as shown in FIG5, a fundus image analysis device 500 is provided, including: an acquisition module 501 and an execution module 502.

[0114] The acquisition module 501 is used to acquire the user's fundus image and determine the user's historical brain information based on whether a preset event has occurred in the user's brain during a historical time period.

[0115] The execution module 502 is used to determine the first fundus image analysis result based on the fundus image and the target fundus image analysis model pre-trained based on the semi-supervised learning algorithm and the knowledge transfer algorithm when the historical brain information indicates that the user has not experienced the preset event within the historical time period.

[0116] The results of the first fundus image analysis are used to characterize whether the user has a hidden abnormal brain region, and include first predictive information indicating whether the user will experience the preset event within a preset time period.

[0117] In one embodiment, the execution module 502 is further configured to determine a second fundus image analysis result based on the fundus image and the target fundus image analysis model when the historical brain information indicates that the user has experienced the preset event within a historical time period; wherein the second fundus image analysis result includes second prediction information, which is used to characterize whether the user will experience the preset event again within the preset time period.

[0118] In one embodiment, the target fundus image analysis model includes a first model and a second model. The execution module 502 is specifically used to input the fundus image into the first model when the historical brain information indicates that the user has not experienced the preset event within the historical time period, so as to obtain the first fundus image analysis result output by the first model.

[0119] In one embodiment, the execution module 502 is specifically used to input the fundus image into the second model when the historical brain information indicates that the user has experienced the preset event within a historical time period, so as to obtain the second fundus image analysis result output by the second model.

[0120] In one embodiment, the acquisition module 501 is specifically used to acquire the user's first fundus image, and if the retinal region in the first fundus image meets preset conditions, determine the first fundus image as a second fundus image; and perform image enhancement processing on the second fundus image to obtain the fundus image.

[0121] In one embodiment, the execution module 502 is further configured to acquire training data; the training data includes fundus images of users with brain abnormalities possessing the concealment, fundus images of users without brain abnormalities possessing the concealment, fundus images of users who have not experienced the preset event in the historical time period, fundus images of users who have experienced the preset event in the historical time period, and fundus images of random users; the initial fundus image analysis model is trained based on the training data, the semi-supervised learning algorithm, and the knowledge transfer algorithm to obtain the first model; the first model is adjusted to obtain the second model, and the target fundus image analysis model is determined based on the first model and the second model.

[0122] Each module in the aforementioned fundus image analysis device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0123] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram is shown in Figure 6. The computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is connected to the system bus via the I / O interfaces. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store data. The I / O interfaces of the computer device are used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a fundus image analysis method.

[0124] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram is shown in Figure 7. The computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a fundus image analysis method. The display unit of the computer device is used to form a visually visible image and may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0125] Those skilled in the art will understand that the structures shown in Figure 6 or Figure 7 are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements.

[0126] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0127] Acquire fundus images of the user and determine the user's historical brain information based on whether a preset event has occurred in the user's brain during a historical time period;

[0128] If the historical brain information indicates that the user has not experienced the preset event within the historical time period, the first fundus image analysis result is determined based on the fundus image and the target fundus image analysis model pre-trained based on semi-supervised learning algorithm and knowledge transfer algorithm.

[0129] The results of the first fundus image analysis are used to characterize whether the user has a hidden abnormal brain region, and include first predictive information indicating whether the user will experience the preset event within a preset time period.

[0130] In one embodiment, when the processor executes the computer program, it further performs the following steps: when the historical brain information indicates that the user has experienced the preset event within a historical time period, it determines a second fundus image analysis result based on the fundus image and the target fundus image analysis model; wherein the second fundus image analysis result includes second prediction information, which is used to characterize whether the user will experience the preset event again within the preset time period.

[0131] In one embodiment, when the processor executes the computer program, it further performs the following steps: when the historical brain information indicates that the user has not experienced the preset event within the historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

[0132] In one embodiment, when the processor executes the computer program, it further performs the following steps: when the historical brain information indicates that the user has experienced the preset event within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

[0133] In one embodiment, when the processor executes the computer program, it further performs the following steps: acquiring a first fundus image of the user, and if the retinal region in the first fundus image meets preset conditions, determining the first fundus image as a second fundus image; performing image enhancement processing on the second fundus image to obtain the fundus image.

[0134] In one embodiment, when the processor executes the computer program, it further performs the following steps: acquiring training data; the training data includes fundus images of users with brain abnormalities possessing the concealment, fundus images of users without brain abnormalities possessing the concealment, fundus images of users who have not experienced the preset event in a historical time period, fundus images of users who have experienced the preset event in a historical time period, and fundus images of random users; training an initial fundus image analysis model based on the training data, the semi-supervised learning algorithm, and the knowledge transfer algorithm to obtain the first model; adjusting the first model to obtain the second model, and determining the target fundus image analysis model based on the first model and the second model.

[0135] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0136] Acquire fundus images of the user and determine the user's historical brain information based on whether a preset event has occurred in the user's brain during a historical time period;

[0137] If the historical brain information indicates that the user has not experienced the preset event within the historical time period, the first fundus image analysis result is determined based on the fundus image and the target fundus image analysis model pre-trained based on semi-supervised learning algorithm and knowledge transfer algorithm.

[0138] The results of the first fundus image analysis are used to characterize whether the user has a hidden abnormal brain region, and include first predictive information indicating whether the user will experience the preset event within a preset time period.

[0139] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: when the historical brain information indicates that the user has experienced the preset event within a historical time period, determining a second fundus image analysis result based on the fundus image and the target fundus image analysis model; wherein the second fundus image analysis result includes second prediction information, which is used to characterize whether the user will experience the preset event again within the preset time period.

[0140] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: when the historical brain information indicates that the user has not experienced the preset event within the historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

[0141] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: when the historical brain information indicates that the user has experienced the preset event within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

[0142] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring a first fundus image of the user, and if the retinal region in the first fundus image meets preset conditions, determining the first fundus image as a second fundus image; performing image enhancement processing on the second fundus image to obtain the fundus image.

[0143] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring training data; the training data includes fundus images of users with brain abnormalities possessing the concealment, fundus images of users without brain abnormalities possessing the concealment, fundus images of users who have not experienced the preset event in a historical time period, fundus images of users who have experienced the preset event in a historical time period, and fundus images of random users; training an initial fundus image analysis model based on the training data, the semi-supervised learning algorithm, and the knowledge transfer algorithm to obtain the first model; adjusting the first model to obtain the second model, and determining the target fundus image analysis model based on the first model and the second model.

[0144] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0145] Acquire fundus images of the user and determine the user's historical brain information based on whether a preset event has occurred in the user's brain during a historical time period;

[0146] If the historical brain information indicates that the user has not experienced the preset event within the historical time period, the first fundus image analysis result is determined based on the fundus image and the target fundus image analysis model pre-trained based on semi-supervised learning algorithm and knowledge transfer algorithm.

[0147] The results of the first fundus image analysis are used to characterize whether the user has a hidden abnormal brain region, and include first predictive information indicating whether the user will experience the preset event within a preset time period.

[0148] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: when the historical brain information indicates that the user has experienced the preset event within a historical time period, determining a second fundus image analysis result based on the fundus image and the target fundus image analysis model; wherein the second fundus image analysis result includes second prediction information, which is used to characterize whether the user will experience the preset event again within the preset time period.

[0149] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: when the historical brain information indicates that the user has not experienced the preset event within the historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

[0150] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: when the historical brain information indicates that the user has experienced the preset event within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

[0151] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring a first fundus image of the user, and if the retinal region in the first fundus image meets preset conditions, determining the first fundus image as a second fundus image; performing image enhancement processing on the second fundus image to obtain the fundus image.

[0152] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: acquiring training data; the training data includes fundus images of users with brain abnormalities possessing the concealment, fundus images of users without brain abnormalities possessing the concealment, fundus images of users who have not experienced the preset event in a historical time period, fundus images of users who have experienced the preset event in a historical time period, and fundus images of random users; training an initial fundus image analysis model based on the training data, the semi-supervised learning algorithm, and the knowledge transfer algorithm to obtain the first model; adjusting the first model to obtain the second model, and determining the target fundus image analysis model based on the first model and the second model.

[0153] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0154] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0155] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0156] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for analyzing fundus images, characterized in that, The method includes: Acquire fundus images of the user and determine the user's historical brain information based on whether a preset event has occurred in the user's brain during a historical time period; When the historical brain information indicates that the user has not experienced the preset event within the historical time period, the first fundus image analysis result is determined based on the fundus image and the target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm. The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain region, and includes first predictive information characterizing whether the user will experience the preset event within a preset time period.

2. The method according to claim 1, characterized in that, The method further includes: When the historical brain information indicates that the user has experienced the preset event within a historical time period, the second fundus image analysis result is determined based on the fundus image and the target fundus image analysis model; The second fundus image analysis result includes second prediction information, which is used to characterize whether the user will experience the preset event again within a preset time period.

3. The method according to claim 1, characterized in that, The target fundus image analysis model includes a first model and a second model. Determining the first fundus image analysis result based on the fundus image and the target fundus image analysis model pre-trained using a semi-supervised learning algorithm and a knowledge transfer algorithm includes: The fundus image is input into the first model to obtain the analysis result of the first fundus image output by the first model.

4. The method according to claim 2, characterized in that, The target fundus image analysis model includes a first model and a second model. Determining the second fundus image analysis result based on the fundus image and the target fundus image analysis model includes: The fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

5. The method according to any one of claims 1-4, characterized in that, The acquisition of the user's fundus image includes: A first fundus image of the user is obtained, and if the retinal region in the first fundus image meets a preset condition, the first fundus image is determined as a second fundus image. The second fundus image is subjected to image enhancement processing to obtain the fundus image.

6. The method according to any one of claims 1 to 5, characterized in that, The training method for the target fundus image analysis model includes: Acquire training data; the training data includes fundus images of users with brain abnormal regions that have the concealment, fundus images of users without brain abnormal regions that have not the concealment, fundus images of users who have not experienced the preset event in the historical time period, fundus images of users who have experienced the preset event in the historical time period, and fundus images of random users. The initial fundus image analysis model is trained based on the training data, the semi-supervised learning algorithm, and the knowledge transfer algorithm to obtain the first model; The first model is adjusted to obtain the second model, and the target fundus image analysis model is determined based on the first model and the second model.

7. The method according to claim 6, characterized in that, Before training the initial fundus image analysis model based on the training data, the semi-supervised learning algorithm, and the knowledge transfer algorithm, the method further includes: pre-training and initializing the initial fundus image analysis model.

8. The method according to any one of claims 1 to 7, characterized in that, The historical brain information includes first information and second information. The first information is used to indicate that the user's brain did not experience the preset event during the historical time period, and the second information is used to indicate that the user's brain experienced the preset event during the historical time period.

9. The method according to claim 2 or 4, characterized in that, The second prediction information is also used to characterize the predicted probability that the user will experience the preset event again within the preset time period.

10. The method according to claim 5, characterized in that, Image enhancement processing is performed on the second fundus image, including: Convert the second fundus image from the RGB color space to the LAB color space; The second fundus image is segmented into image blocks of fixed size; Enhance the luminance channel of each image patch using contrast-limited adaptive histogram equalization; Convert the enhanced second fundus image back to RGB format.

11. The method according to claim 5, characterized in that, Image enhancement processing of the second fundus image is as follows: P = α·P + β·Gauss(P,s) + δ Where P refers to the second fundus image, Gauss(P,s) means applying a Gaussian filter with a standard deviation of s to the second fundus image P, and α, β, δ are adjustable parameters.

12. A fundus image analysis device, characterized in that, The device includes: The acquisition module is used to acquire fundus images of the user and determine the user's historical brain information based on whether a preset event has occurred in the user's brain during a historical time period. The execution module is used to determine the first fundus image analysis result based on the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm when the historical brain information indicates that the user has not experienced the preset event within the historical time period. The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain region, and includes first predictive information characterizing whether the user will experience the preset event within a preset time period.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images