Eye fundus image analysis method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to an eye image analysis method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a fundus image of a user, and determining historical brain information of the user according to whether a preset event occurs in the brain of the user in a historical time period; under the condition that the historical brain information indicates that the user does not have the preset event in the historical time period, determining a first fundus image analysis result according to the fundus image and a target fundus image analysis model obtained by training in advance based on a semi-supervised learning algorithm and a knowledge migration algorithm; wherein the first fundus image analysis result is used for representing whether the user has a hidden brain abnormal area or not and representing first prediction information of the preset event occurring in the preset time period of the user. The method can effectively reduce the cost, and achieves the recognition and prediction of brain abnormalities for large-scale people.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a fundus image analysis method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the aging of the global population and changes in lifestyle, the incidence of brain abnormalities has increased year by year, becoming an important public health issue leading to death and disability. Therefore, it is particularly important to identify high-risk groups early for effective intervention.

[0003] In the prior art, most of the methods are to image the user's brain through magnetic resonance imaging or computer tomography to identify and predict the user's brain abnormalities.

[0004] However, this method of identifying and predicting brain abnormalities is costly, which makes it impossible to identify and predict brain abnormalities in large populations. Summary of the invention

[0005] Based on this, it is necessary to provide a fundus image analysis method, device, computer equipment, computer-readable storage medium and computer program product with low cost to address the above technical problems and thus be able to identify and predict brain abnormalities in a large population.

[0006] In a first aspect, the present application provides a fundus image analysis method, comprising:

[0007] Obtaining a fundus image of the user, and determining historical brain information of the user based on whether a preset event has occurred in the user's brain within a historical time period;

[0008] When the historical brain information indicates that the preset event has not occurred to the user within the historical time period, a first fundus image analysis result is determined based on the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm; wherein the first fundus image analysis result is used to characterize whether the user has a hidden brain abnormal area, and the first prediction information of the occurrence of the preset event to the user within a preset time period.

[0009] In one of the embodiments, the method also includes: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, determining a second fundus image analysis result based on the fundus image and the target fundus image analysis model; wherein the second fundus image analysis result is used to represent second prediction information that the preset event will occur again to the user within a preset time period.

[0010] In one embodiment, the target fundus image analysis model includes a first model and a second model, and the first fundus image analysis result is determined based on the fundus image and the target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm, including: when the historical brain information indicates that the preset event has not occurred to the user within a historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

[0011] In one embodiment, the second fundus image analysis result is determined based on the fundus image and the target fundus image analysis model, including: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

[0012] In one embodiment, the method of obtaining a fundus image of a user includes: obtaining a first fundus image of the user, and if a retinal area in the first fundus image satisfies a preset condition, determining the first fundus image as a second fundus image; and performing image enhancement processing on the second fundus image to obtain the fundus image.

[0013] In one embodiment, the training method of the target fundus image analysis model includes: obtaining training data; the training data includes fundus images of users with the hidden brain abnormality area, fundus images of users without the hidden brain abnormality area, fundus images of users who have not experienced the preset event within a historical time period, fundus images of users who have experienced the preset event within a historical time period, and fundus images of random users; based on the training data, the semi-supervised learning algorithm and the knowledge transfer algorithm, the initial fundus image analysis model is trained to obtain the first model; the first model is adjusted to obtain the second model, and the target fundus image analysis model is determined based on the first model and the second model.

[0014] In a second aspect, the present application also provides a fundus image analysis device, comprising:

[0015] An acquisition module, used to acquire a fundus image of a user and determine the historical brain information of the user according to whether a preset event has occurred in the user's brain within a historical time period;

[0016] An execution module, configured to determine a first fundus image analysis result according to the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm, when the historical brain information indicates that the preset event has not occurred to the user within a historical time period;

[0017] The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, as well as first prediction information of the occurrence of the preset event within a preset time period for the user.

[0018] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the method described in any embodiment of the first aspect when executing the computer program.

[0019] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any one of the embodiments of the first aspect above.

[0020] In a fifth aspect, the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the method described in any embodiment of the first aspect above.

[0021] The above-mentioned fundus image analysis method, device, computer equipment, computer-readable storage medium and computer program product first obtain the user's fundus image, and determine the user's historical brain information based on whether the preset event has occurred in the user's brain within the historical time period; when the historical brain information indicates that the preset event has not occurred in the user within the historical time period, determine the first fundus image analysis result based on the fundus image and the target fundus image analysis model pre-trained based on the semi-supervised learning algorithm and the knowledge transfer algorithm; wherein the first fundus image analysis result is used to characterize whether the user has a hidden brain abnormal area, and the first prediction information of the user's occurrence of the preset event within the preset time period. The fundus image analysis method provided in the present application can determine whether the user has a hidden brain abnormal area and the first prediction information of the user's occurrence of the preset event within the preset time period through the user's fundus image and the user's historical brain information. Since the acquisition cost of the fundus image is low, it can realize the identification and prediction of brain abnormalities for a large population. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 is a schematic diagram of a flow chart of a fundus image analysis method in one embodiment;

[0024] Figure 2 A schematic diagram of a flow chart of a method for acquiring a fundus image of a user in one embodiment;

[0025] Figure 3 is a flow chart of a method for training a target fundus image analysis model in one embodiment;

[0026] Figure 4 is a schematic flow chart of a fundus image analysis method in another embodiment;

[0027] Figure 5 is a structural block diagram of a fundus image analysis device in one embodiment;

[0028] Figure 6 is an internal structure diagram of a computer device in one embodiment;

[0029] Figure 7 is an internal structure diagram of a computer device in another embodiment;

[0030] Figure 8 Schematic diagram of the target fundus image analysis model training process in one embodiment. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0032] With the aging of the global population and changes in lifestyle, the incidence of brain abnormalities has increased year by year, becoming an important public health issue leading to death and disability. Therefore, it is particularly important to identify high-risk groups early for effective intervention.

[0033] In the prior art, most of the methods are to image the user's brain through magnetic resonance imaging or computer tomography (CT) to identify and predict the user's brain abnormalities.

[0034] However, this method of identifying and predicting brain abnormalities is costly, which makes it impossible to identify and predict brain abnormalities in large populations.

[0035] In view of this, the present application provides a fundus image analysis method. Since fundus images contain rich brain information and the cost of acquiring fundus images is low, it can realize the identification and prediction of brain abnormalities for a large population.

[0036] The fundus image analysis method provided in the present application may be executed by a computer device, which may be a terminal or a server.

[0037] In an exemplary embodiment, Figure 1 As shown, a fundus image analysis method is provided, the method comprising the following steps:

[0038] Step 101: Acquire a fundus image of a user, and determine historical brain information of the user based on whether a preset event has occurred in the user's brain within a historical time period.

[0039] Optionally, the fundus image refers to an image that can be used to characterize structural and functional information of the retina, optic nerve, and blood vessels.

[0040] Exemplarily, a non-contact photographing technology may be used to obtain the user's fundus image. Specifically, the user's fundus image may be obtained based on a fundus camera, or based on optical coherence tomography.

[0041] The historical time period may be pre-set by a technician according to actual needs, for example, the historical time period may be the past ten years or the past five years.

[0042] The preset event may be a brain abnormality event. The brain abnormality event includes brain structural abnormality event, brain neurological abnormality event, brain electrophysiological abnormality event, brain metabolic abnormality event, and brain blood circulation abnormality event. The brain structural abnormality event further includes brain tumor event, cerebral infarction event, cerebral hemorrhage event, and brain atrophy event. In the embodiment of the present application, the preset event is a cerebral infarction event among brain structural abnormality events, which may also be referred to as a stroke event.

[0043] Exemplarily, the historical brain information can be used to indicate whether a preset event has occurred in a historical time period. The historical brain information may include first information and second information, wherein the first information is used to indicate whether a preset event has not occurred in the user's brain in the historical time period, and the second information is used to indicate whether a preset event has occurred in the user's brain in the historical time period.

[0044] Furthermore, that is, if it is determined that no preset event has occurred in the user's brain within the historical time period, the historical brain information can be determined as the first information; if it is determined that a preset event has occurred in the user's brain within the historical time period, the historical brain information can be determined as the second information.

[0045] Step 102: When the historical brain information indicates that the preset event has not occurred to the user within the historical time period, determine the first fundus image analysis result based on the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm.

[0046] The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, as well as first prediction information of the occurrence of the preset event within a preset time period for the user.

[0047] Optionally, the target fundus image analysis model may be a DeepRETStroke model.

[0048] In the embodiment of the present application, the area with latent brain abnormality refers to the area where latent cerebral infarction belongs.

[0049] The first prediction information may be a prediction result used to characterize whether the preset event will occur to the user within a preset time period, or may be a prediction probability used to characterize the preset event occurring to the user within a preset time period.

[0050] In some exemplary embodiments, as described above, the historical brain information may include first information and second information. When the historical brain information is the first information, that is, no preset event has occurred in the user's brain within the historical time period, the fundus image can be directly input into the target fundus image analysis model to obtain the first fundus image analysis result of the user output by the target fundus image analysis model.

[0051] In other exemplary embodiments, the target fundus image analysis model may also include a first model and a second model. When the historical brain information is the first information, that is, when no preset events have occurred in the user's brain within the historical time period, the fundus image can be directly input into the first model to obtain the first fundus image analysis result of the user output by the first model.

[0052] The above-mentioned fundus image analysis method first obtains the user's fundus image, and determines the user's historical brain information based on whether the preset event has occurred in the user's brain within the historical time period; when the historical brain information indicates that the preset event has not occurred in the user within the historical time period, the first fundus image analysis result is determined based on the fundus image and the target fundus image analysis model pre-trained based on the semi-supervised learning algorithm and the knowledge transfer algorithm; wherein the first fundus image analysis result is used to characterize whether the user has a hidden brain abnormal area, and the first prediction information of the user's occurrence of the preset event within the preset time period. The fundus image analysis method provided in the present application can determine whether the user has a hidden brain abnormal area and the first prediction information of the user's occurrence of the preset event within the preset time period through the user's fundus image and the user's historical brain information. Since the acquisition cost of the fundus image is low, it can realize the identification and prediction of brain abnormalities for a large population.

[0053] Furthermore, the target fundus image analysis model used in the present application is trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm, so that the output fundus image analysis results are more accurate.

[0054] Moreover, compared to most existing technologies in which the fundus image analysis results only include the first prediction information of the user's occurrence of a preset event within a preset time period, the fundus image analysis results of the present application also include analysis results of whether the user has hidden brain abnormalities.

[0055] In an exemplary embodiment, when the historical brain information indicates that the preset event occurred to the user within a historical time period, the method further includes: determining a second fundus image analysis result based on the fundus image and the target fundus image analysis model.

[0056] The second fundus image analysis result is used to represent second prediction information of the user's recurrence of the preset event within a preset time period.

[0057] Optionally, the second prediction information may be a prediction result representing whether the preset event will occur again for the user within a preset time period, or may be a prediction probability representing the preset event will occur again for the user within a preset time period.

[0058] In some exemplary embodiments, as described above, the historical brain information may include first information and second information. When the historical brain information is the second information, that is, when a preset event has occurred in the user's brain within a historical time period, the fundus image can be directly input into the target fundus image analysis model to obtain the second fundus image analysis result of the user output by the target fundus image analysis model.

[0059] In other exemplary embodiments, the target fundus image analysis model may also include a first model and a second model. When the historical brain information is the second information, that is, when a preset event has occurred in the user's brain within a historical time period, the fundus image can be directly input into the second model to obtain the second fundus image analysis result of the user output by the second model.

[0060] In an exemplary embodiment, the target fundus image analysis model includes a first model and a second model, and the first fundus image analysis result is determined based on the fundus image and the target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm, including: when the historical brain information indicates that the preset event has not occurred to the user within a historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

[0061] In some exemplary embodiments, when it is determined based on the user's historical brain information that the preset event has not occurred to the user within a historical time period, the fundus image is input into the first model to obtain a first fundus image analysis result output by the first model.

[0062] In an exemplary embodiment, the second fundus image analysis result is determined based on the fundus image and the target fundus image analysis model, including: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

[0063] In some exemplary embodiments, when it is determined that the preset event has occurred to the user within a historical time period based on the user's historical brain information, the fundus image is input into the second model to obtain a second fundus image analysis result output by the second model.

[0064] In an exemplary embodiment, Figure 2 As shown, the method of obtaining the fundus image of the user includes the following steps:

[0065] Step 201: Acquire a first fundus image of the user, and when a retinal area in the first fundus image meets a preset condition, determine the first fundus image as a second fundus image.

[0066] Optionally, the preset condition may be pre-set by a technician according to actual needs. Exemplarily, the preset condition may be that at least 75% of the retinal area in the first fundus image is clearly observable. The preset condition may also be that there are no artifacts in the retinal area in the first fundus image.

[0067] In some exemplary embodiments, after a first fundus image of a user is acquired using a non-contact photographing technique, it may be determined whether the first fundus image meets a preset condition.

[0068] Further, when it is determined that the first fundus image meets a preset condition, the first fundus image is determined as the second fundus image.

[0069] Specifically, when at least 75% of the retinal area in the first fundus image is clearly observable and no artifacts exist in the retinal area in the first fundus image, the first fundus image is determined as the second fundus image.

[0070] Step 202: Perform image enhancement processing on the second fundus image to obtain the fundus image.

[0071] In some exemplary embodiments, contrast limited adaptive histogram equalization (CLAHE) can be used to enhance the contrast of the second fundus image while suppressing noise. Specifically, the second fundus image can be converted from the RGB color space to the LAB color space, and the second fundus image can be divided into small blocks of fixed size. CLAHE is applied to the brightness channel of each small block, and the processed second fundus image is converted back to RGB format.

[0072] Furthermore, color normalization is used to reduce the influence of the shooting equipment and the shooting environment on the second fundus image.

[0073] Specifically, , where P refers to the second fundus image, and Gauss (P, s) refers to applying a Gaussian filter with a standard deviation equal to s to the second fundus image P. Refers to adjustable parameters. For example, The resolution of each fundus image was adjusted to 512×512.

[0074] In an exemplary embodiment, Figure 3 As shown, the training method of the target fundus image analysis model includes the following steps:

[0075] Step 301: Obtain training data.

[0076] Among them, the training data includes fundus images of users with hidden brain abnormalities, fundus images of users without hidden brain abnormalities, fundus images of users who have not experienced the preset event within the historical time period, fundus images of users who have experienced the preset event within the historical time period, and fundus images of random users.

[0077] The random user is a user for whom it is uncertain whether or not there is a hidden abnormal brain area, and it is uncertain whether or not the preset event has occurred in a historical time period.

[0078] Step 302: Perform model training on the initial fundus image analysis model based on the training data, the semi-supervised learning algorithm and the knowledge transfer algorithm to obtain a first model.

[0079] The first model can be used to predict whether the preset event will occur to the user within a preset time period, and can also be used to predict the probability of the preset event occurring to the user within the preset time period.

[0080] Step 303: adjust the first model to obtain a second model, and determine a target fundus image analysis model according to the first model and the second model.

[0081] The second model can be used to predict whether the preset event will occur again to the user within a preset time period, and can also be used to predict the probability of the preset event occurring again to the user within the preset time period.

[0082] In an optional embodiment of the present application, Figure 8 As shown, Figure 8 The schematic diagram of the training process of the target fundus image analysis model is that the initial fundus image analysis model is trained based on the training data and the semi-supervised learning algorithm to obtain a first model. Before the initial fundus image analysis model is trained, model pre-training and model initialization of the initial fundus image analysis model are performed.

[0083] Specifically, the initial fundus image analysis model can be used Figure 8 The architecture shown in FIG. 1 is a schematic diagram of an encoder of the initial fundus image analysis model. The encoder of the initial fundus image analysis model may adopt a large visual Transformer network architecture, which may include 24 Transformer modules and an embedding vector size of 1024; the decoder of the initial fundus image analysis model may adopt a small visual Transformer network architecture, which may include 8 Transformer modules and an embedding vector size of 512.

[0084] In the process of pre-training the initial fundus image analysis model using the MAE algorithm, the training data will be divided into 16×16 patches, 75% of the patches will be masked, and the remaining 25% of the patches will be used as the input of the model to participate in the model update. The loss function in this process is , where N refers to the total number of training data, N p refers to the total number of pixels that are not masked in each training data, and C refers to the total number of image channels. s , k, q refers to the model's predicted value of the qth channel of the kth pixel in the training data numbered s, x s , k, q refers to the true value of the qth channel of the kth pixel in the training data numbered s, .

[0085] Furthermore, the diagnostic head of the initial fundus image analysis model can use two logistic regression classifiers with the same structure. The learning head and the prediction head can use multi-layer perceptrons with output results of 2 dimensions and 5 dimensions respectively. Based on the first training data in the training data, the learning of the task of "predicting the probability of the user's occurrence of the preset event within the preset time period" is performed to update the encoder and prediction head parts in the model and complete the model initialization. The first training data is the fundus image of the user who has not experienced the preset event within the historical time period. The loss function in this process is , where N represents the total number of first training data, t represents the year, and T i represents the total number of follow-up years for the user numbered i in the first training data. (If a preset event occurs during the follow-up period or the follow-up time reaches the preset time period but no preset event occurs, T i =5; if no preset event occurs during the follow-up period but the follow-up time is less than the preset time period, T i <5), x i represents the fundus image of the user numbered i (i=1, 2, 3, ..., N), p θ，t (x i ) represents the probability of a preset event occurring before time point t, y i，t represents the user status at time point t (users who have a preset event before time point t, y i，t =1; users who have not experienced the preset event, y i，t =0).

[0086] Furthermore, the process of training the model based on the semi-supervised learning algorithm and the knowledge transfer algorithm can be regarded as a large cycle, and each cycle includes the two steps of "semi-supervised learning" and "knowledge transfer". At the same time, each large cycle of "semi-supervised learning" can be regarded as a small cycle, and each small cycle includes the two steps of "labeled database update" and "diagnostic head update".

[0087] Exemplarily, the training data can be defined as two new databases, namely, a "labeled database" and an "unlabeled database". Before the start of this small cycle, the "labeled database" includes the fundus images of all users with hidden brain abnormalities and the fundus images of users without hidden brain abnormalities, and the "unlabeled database" includes the fundus images of all users who have not experienced the preset event in the historical time period. After the small cycle starts, for one small cycle training round, the "labeled database" is first used to update the diagnostic head part of the model by learning the task of "identifying hidden brain abnormalities".

[0088] After the update is completed, the model is used to perform the "hidden brain abnormality area identification" task on the "validation database" of "fundus images of users with hidden brain abnormality areas and fundus images of users without hidden brain abnormality areas", and the probability of hidden brain abnormality areas corresponding to each fundus image of a user with hidden brain abnormality areas is determined. Then, the minimum probability of the occurrence of a preset event that can maintain the prediction accuracy of positive samples (the proportion of true positive samples in samples predicted as positive) above 0.75 is selected as the high prediction confidence standard pos_thr for positive samples, and the maximum probability of the occurrence of a preset event that maintains the prediction accuracy of negative samples (the proportion of true negative samples in samples predicted as negative) above 0.75 is selected as the high prediction confidence standard neg_thr for negative samples.

[0089] Then use this model to perform the task of "identifying hidden brain abnormal areas" on the "unlabeled database", and label the training data with a predicted probability greater than pos_thr with a positive "pseudo-label", and label the training data with a predicted probability less than neg_thr with a negative "pseudo-label". These training data are deleted from the "unlabeled database" and added to the "labeled database". And in the next small cycle training round, use this expanded "labeled database" to re-update the model's diagnostic head. In this way, samples from the "unlabeled database" will be continuously added to the "labeled database" with the small cycle until there are no samples in the "unlabeled database" or no samples with high prediction confidence that can be assigned "pseudo-labels" can be found from it, and this small cycle (the step of "training the initial fundus image analysis model based on training data and semi-supervised learning algorithm") ends.

[0090] Exemplarily, the first model is used to predict the task of "hidden brain abnormality areas" on "fundus images of users who have not experienced the preset event within the historical time period", and each training data is assigned a "prediction probability" as a "soft label" to obtain a database with "soft labels" added.

[0091] Through the learning of the first model for the tasks of "predicting the probability of a user experiencing a preset event within a preset time period" and "soft label prediction", the encoder, learning head, and prediction head in the model are updated. The loss function in this process is

[0092] ;

[0093] The loss function consists of two parts. The first part is the same as the loss function in the "model initialization" stage, which represents the prediction loss of the task of "predicting the probability of a user experiencing a preset event within a preset time period". The second part represents the prediction loss of the "soft label prediction" task, where represents the fundus image of user number i, , represents the true probability distribution of the “soft label” on category j, represents the predicted probability distribution output by the first model on category j, α is the weight parameter for controlling the tasks of "soft label prediction" and "predicting the probability of a user having the preset event within a preset time period", and α can be 0.3. In this way, the first model can simultaneously learn the brain condition of a user during the baseline and follow-up periods, thereby improving the accuracy of the prediction of the probability of a user having the preset event within a preset time period.

[0094] After the update is completed, the first model is used to predict the probability of the user experiencing the preset event within the preset time period on the "verification database" of "fundus images of users who have not experienced the preset event within the historical time period". When the model performance reaches the pre-set standard, it means that the first model has been built, and the entire macro cycle ends; otherwise, the next macro cycle update begins.

[0095] As described above, after the initial fundus image analysis model is trained based on the training data and the semi-supervised learning algorithm to obtain a first model, the first model needs to be adjusted to obtain a second model, and the target fundus image analysis model is determined based on the first model and the second model.

[0096] The first model (especially the encoder part) can be used as a pre-training model with strong prior knowledge, so as to play a certain role in the training of other brain abnormality prediction tasks. Therefore, this application uses the encoder of the first model as a pre-training model, and develops a second model after fine-tuning based on the first model. The loss function of this process is , it can be found that the loss function is expressed in the same form as the loss function of the "model initialization" step in the previous stage. The difference is that the observed event changes from the occurrence of the preset event to the recurrence of the preset event.

[0097] In an exemplary embodiment, Figure 4 As shown, another fundus image analysis method is provided, the method comprising the following steps:

[0098] Step 401: acquiring a first fundus image of the user, and if a retinal area in the first fundus image satisfies a preset condition, determining the first fundus image as a second fundus image, performing image enhancement processing on the second fundus image to obtain the fundus image, and determining historical brain information of the user according to whether a preset event has occurred in the brain of the user within a historical time period;

[0099] Step 402: when the historical brain information indicates that the preset event has not occurred to the user within the historical time period, the fundus image is input into a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm to obtain the first fundus image analysis result output by the first model; wherein the first fundus image analysis result is used to characterize whether the user has a hidden brain abnormal area, and first prediction information of the user's occurrence of the preset event within the preset time period;

[0100] Step 403: When the historical brain information indicates that the preset event has occurred to the user within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model; wherein the second fundus image analysis result is used to characterize the second prediction information that the preset event will occur again to the user within a preset time period.

[0101] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0102] Based on the same inventive concept, the embodiment of the present application also provides a fundus image analysis device for implementing the fundus image analysis method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more fundus image analysis device embodiments provided below can refer to the limitations of the fundus image analysis method above, and will not be repeated here.

[0103] In an exemplary embodiment, Figure 5 As shown, a fundus image analysis device 500 is provided, comprising: an acquisition module 501 and an execution module 502, wherein:

[0104] The acquisition module 501 is used to acquire the fundus image of the user and determine the historical brain information of the user according to whether a preset event has occurred in the brain of the user within a historical time period;

[0105] An execution module 502 is used to determine a first fundus image analysis result according to the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm when the historical brain information indicates that the preset event has not occurred to the user within a historical time period;

[0106] The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, as well as first prediction information of the occurrence of the preset event within a preset time period for the user.

[0107] In one embodiment, the execution module 502 is also used to determine a second fundus image analysis result based on the fundus image and the target fundus image analysis model when the historical brain information indicates that the preset event has occurred to the user within a historical time period; wherein the second fundus image analysis result is used to represent second prediction information that the preset event will occur again to the user within a preset time period.

[0108] In one embodiment, the target fundus image analysis model includes a first model and a second model. The execution module 502 is specifically used to input the fundus image into the first model when the historical brain information indicates that the preset event has not occurred to the user within a historical time period, so as to obtain the first fundus image analysis result output by the first model.

[0109] In one embodiment, the execution module 502 is specifically used to input the fundus image into the second model when the historical brain information indicates that the preset event has occurred to the user within a historical time period, so as to obtain the second fundus image analysis result output by the second model.

[0110] In one embodiment, the acquisition module 501 is specifically used to acquire a first fundus image of the user, and when the retinal area in the first fundus image meets a preset condition, the first fundus image is determined as a second fundus image; and image enhancement processing is performed on the second fundus image to obtain the fundus image.

[0111] In one embodiment, the execution module 502 is also used to obtain training data; the training data includes fundus images of users with the hidden brain abnormality area, fundus images of users without the hidden brain abnormality area, fundus images of users who have not experienced the preset event within the historical time period, fundus images of users who have experienced the preset event within the historical time period, and fundus images of random users; based on the training data, the semi-supervised learning algorithm and the knowledge transfer algorithm, the initial fundus image analysis model is trained to obtain the first model; the first model is adjusted to obtain the second model, and the target fundus image analysis model is determined based on the first model and the second model.

[0112] Each module in the fundus image analysis device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0113] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a fundus image analysis method is implemented.

[0114] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 7As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a fundus image analysis method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0115] Those skilled in the art will understand that Figure 6 or Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0116] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0117] Obtaining a fundus image of the user, and determining historical brain information of the user based on whether a preset event has occurred in the user's brain within a historical time period;

[0118] When the historical brain information indicates that the preset event has not occurred to the user within the historical time period, determining a first fundus image analysis result according to the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm;

[0119] The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, as well as first prediction information of the occurrence of the preset event within a preset time period for the user.

[0120] In one embodiment, the processor also implements the following steps when executing the computer program: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, a second fundus image analysis result is determined based on the fundus image and the target fundus image analysis model; wherein the second fundus image analysis result is used to represent second prediction information that the preset event will occur again to the user within a preset time period.

[0121] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the historical brain information indicates that the preset event has not occurred to the user within the historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

[0122] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

[0123] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining a first fundus image of the user, and when the retinal area in the first fundus image meets a preset condition, determining the first fundus image as a second fundus image; performing image enhancement processing on the second fundus image to obtain the fundus image.

[0124] In one embodiment, the processor also implements the following steps when executing the computer program: acquiring training data; the training data includes fundus images of users having the hidden brain abnormality area, fundus images of users not having the hidden brain abnormality area, fundus images of users who have not experienced the preset event within the historical time period, fundus images of users who have experienced the preset event within the historical time period, and fundus images of random users; performing model training on the initial fundus image analysis model based on the training data, the semi-supervised learning algorithm, and the knowledge transfer algorithm to obtain the first model; adjusting the first model to obtain the second model, and determining the target fundus image analysis model based on the first model and the second model.

[0125] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0126] Obtaining a fundus image of the user, and determining historical brain information of the user based on whether a preset event has occurred in the user's brain within a historical time period;

[0127] When the historical brain information indicates that the preset event has not occurred to the user within the historical time period, determining a first fundus image analysis result according to the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm;

[0128] The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, as well as first prediction information of the occurrence of the preset event within a preset time period for the user.

[0129] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, a second fundus image analysis result is determined based on the fundus image and the target fundus image analysis model; wherein the second fundus image analysis result is used to represent second prediction information that the preset event will occur again to the user within a preset time period.

[0130] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the historical brain information indicates that the preset event has not occurred to the user within the historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

[0131] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

[0132] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: obtaining a first fundus image of the user, and when the retinal area in the first fundus image meets a preset condition, determining the first fundus image as a second fundus image; performing image enhancement processing on the second fundus image to obtain the fundus image.

[0133] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining training data; the training data includes fundus images of users having the hidden brain abnormality area, fundus images of users not having the hidden brain abnormality area, fundus images of users who have not experienced the preset event within the historical time period, fundus images of users who have experienced the preset event within the historical time period, and fundus images of random users; based on the training data, the semi-supervised learning algorithm and the knowledge transfer algorithm, the initial fundus image analysis model is trained to obtain the first model; the first model is adjusted to obtain the second model, and the target fundus image analysis model is determined based on the first model and the second model.

[0134] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0135] Obtaining a fundus image of the user, and determining historical brain information of the user based on whether a preset event has occurred in the user's brain within a historical time period;

[0136] When the historical brain information indicates that the preset event has not occurred to the user within the historical time period, determining a first fundus image analysis result according to the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm;

[0137] The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, as well as first prediction information of the occurrence of the preset event within a preset time period for the user.

[0138] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, a second fundus image analysis result is determined based on the fundus image and the target fundus image analysis model; wherein the second fundus image analysis result is used to represent second prediction information that the preset event will occur again to the user within a preset time period.

[0139] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the historical brain information indicates that the preset event has not occurred to the user within the historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

[0140] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

[0141] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: obtaining a first fundus image of the user, and when the retinal area in the first fundus image meets a preset condition, determining the first fundus image as a second fundus image; performing image enhancement processing on the second fundus image to obtain the fundus image.

[0142] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining training data; the training data includes fundus images of users having the hidden brain abnormality area, fundus images of users not having the hidden brain abnormality area, fundus images of users who have not experienced the preset event within the historical time period, fundus images of users who have experienced the preset event within the historical time period, and fundus images of random users; based on the training data, the semi-supervised learning algorithm and the knowledge transfer algorithm, the initial fundus image analysis model is trained to obtain the first model; the first model is adjusted to obtain the second model, and the target fundus image analysis model is determined based on the first model and the second model.

[0143] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0144] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0145] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0146] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A fundus image analysis method, characterized in that: The method comprises: Acquire a fundus image of a user, and determine historical brain information of the user according to whether a preset event has occurred in the brain of the user within a historical time period; When the historical brain information indicates that the preset event has not occurred to the user within the historical time period, determining a first fundus image analysis result according to the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm; The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, as well as first prediction information of the user's occurrence of the preset event within a preset time period.

2. The method according to claim 1, characterized in that The method further comprises: In a case where the historical brain information indicates that the preset event has occurred to the user within a historical time period, determining a second fundus image analysis result according to the fundus image and the target fundus image analysis model; The second fundus image analysis result is used to represent second prediction information of the user's recurrence of the preset event within a preset time period.

3. The method according to claim 1, characterized in that The target fundus image analysis model includes a first model and a second model, and determining a first fundus image analysis result according to the fundus image and the target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm includes: When the historical brain information indicates that the preset event has not occurred to the user within a historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

4. The method according to claim 3, characterized in that The determining a second fundus image analysis result according to the fundus image and the target fundus image analysis model comprises: When the historical brain information indicates that the preset event has occurred to the user within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

5. The method according to claim 1, characterized in that The obtaining of the user's fundus image comprises: Acquire a first fundus image of the user, and if a retinal area in the first fundus image meets a preset condition, determine the first fundus image as a second fundus image; Perform image enhancement processing on the second fundus image to obtain the fundus image.

6. The method according to any one of claims 1 to 5, characterized in that: The training method of the target fundus image analysis model includes: Acquire training data; the training data includes fundus images of users with the hidden brain abnormality area, fundus images of users without the hidden brain abnormality area, fundus images of users who have not experienced the preset event in a historical time period, fundus images of users who have experienced the preset event in a historical time period, and fundus images of random users; Performing model training on an initial fundus image analysis model based on the training data, the semi-supervised learning algorithm, and the knowledge transfer algorithm to obtain the first model; The first model is adjusted to obtain the second model, and the target fundus image analysis model is determined according to the first model and the second model.

7. A fundus image analysis device, characterized in that: The device comprises: An acquisition module, used to acquire a fundus image of a user, and determine historical brain information of the user according to whether a preset event has occurred in the brain of the user within a historical time period; An execution module, configured to determine a first fundus image analysis result according to the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm, when the historical brain information indicates that the preset event has not occurred to the user within a historical time period; The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, as well as first prediction information of the user's occurrence of the preset event within a preset time period.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images