Fundus image analysis method, device, computer equipment, readable storage medium and program product

Abstract

The present application relates to an eye image analysis method, apparatus, computer device, computer-readable storage medium, and computer program product. The method comprises: obtaining a fundus image of a user, and determining the user's historical brain information based on whether a preset event has occurred in the user's brain within a historical time period; if the historical brain information indicates that the preset event has not occurred in the user within the historical time period, determining a first fundus image analysis result based on the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm; wherein the first fundus image analysis result is used to characterize whether the user has a hidden brain abnormality area, as well as first prediction information of the occurrence of the preset event in the user within a preset time period. The use of this method can effectively reduce costs, thereby realizing the identification and prediction of brain abnormalities for a large population.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a fundus image analysis method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the aging of the global population and changes in lifestyle, the incidence of brain abnormalities is increasing year by year, becoming a major public health issue leading to death and disability. Therefore, early identification of high-risk groups for effective intervention is particularly important.

[0003] In the prior art, most methods use magnetic resonance imaging or computed tomography to image the user's brain in order to identify and predict abnormalities in the user's brain.

[0004] However, this method of identifying and predicting brain abnormalities is costly, which makes it impossible to identify and predict brain abnormalities in large populations. Summary of the Invention

[0005] Based on this, it is necessary to provide a fundus image analysis method, device, computer equipment, computer-readable storage medium and computer program product with low cost to address the above technical problems, and thus can be used to identify and predict brain abnormalities in a large population.

[0006] In a first aspect, the present application provides a fundus image analysis method, comprising:

[0007] Obtaining a fundus image of the user and determining the user's historical brain information based on whether a preset event has occurred in the user's brain within a historical time period;

[0008] When the historical brain information indicates that the preset event has not occurred to the user within the historical time period, a first fundus image analysis result is determined based on the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm; wherein the first fundus image analysis result is used to characterize whether the user has a hidden brain abnormal area, and the first prediction information of the occurrence of the preset event to the user within the preset time period.

[0009] In one embodiment, the method further includes: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, determining a second fundus image analysis result based on the fundus image and the target fundus image analysis model; wherein the second fundus image analysis result is used to represent second prediction information that the preset event will occur again to the user within the preset time period.

[0010] In one embodiment, the target fundus image analysis model includes a first model and a second model, and the first fundus image analysis result is determined based on the fundus image and the target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm, including: when the historical brain information indicates that the preset event has not occurred to the user within a historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

[0011] In one embodiment, the second fundus image analysis result is determined based on the fundus image and the target fundus image analysis model, including: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

[0012] In one embodiment, obtaining a fundus image of a user includes: obtaining a first fundus image of the user, and if a retinal area in the first fundus image meets a preset condition, determining the first fundus image as a second fundus image; and performing image enhancement processing on the second fundus image to obtain the fundus image.

[0013] In one embodiment, the training method of the target fundus image analysis model includes: obtaining training data; the training data includes fundus images of users with the hidden brain abnormality area, fundus images of users without the hidden brain abnormality area, fundus images of users who have not experienced the preset event within the historical time period, fundus images of users who have experienced the preset event within the historical time period, and fundus images of random users; based on the training data, the semi-supervised learning algorithm and the knowledge transfer algorithm, the initial fundus image analysis model is trained to obtain the first model; the first model is adjusted to obtain the second model, and the target fundus image analysis model is determined based on the first model and the second model.

[0014] In a second aspect, the present application further provides a fundus image analysis device, comprising:

[0015] An acquisition module is used to acquire a fundus image of a user and determine the user's historical brain information based on whether a preset event has occurred in the user's brain within a historical time period;

[0016] an execution module, configured to determine a first fundus image analysis result based on the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm, when the historical brain information indicates that the preset event has not occurred to the user within a historical time period;

[0017] The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, as well as first prediction information of the user's occurrence of the preset event within a preset time period.

[0018] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the method described in any one of the embodiments of the first aspect when executing the computer program.

[0019] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any one of the embodiments of the first aspect above.

[0020] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in any one of the embodiments of the first aspect above.

[0021] The above-mentioned fundus image analysis method, apparatus, computer device, computer-readable storage medium, and computer program product first obtain a fundus image of a user and determine the user's historical brain information based on whether a preset event has occurred in the user's brain within a historical time period; if the historical brain information indicates that the preset event has not occurred in the user within the historical time period, a first fundus image analysis result is determined based on the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm; wherein the first fundus image analysis result is used to characterize whether the user has a hidden brain abnormality area and first prediction information of the user's occurrence of the preset event within a preset time period. The fundus image analysis method provided in the present application can determine whether the user has a hidden brain abnormality area and first prediction information of the user's occurrence of the preset event within a preset time period through the user's fundus image and the user's historical brain information. Since the acquisition cost of the fundus image is relatively low, it can realize the identification and prediction of brain abnormalities for a large population. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 1 is a flow chart of a fundus image analysis method according to an embodiment;

[0024] Figure 2 1 is a flow chart of a method for obtaining a fundus image of a user in one embodiment;

[0025] Figure 3 1 is a flow chart of a method for training a target fundus image analysis model in one embodiment;

[0026] Figure 4 is a schematic flow chart of a fundus image analysis method according to another embodiment;

[0027] Figure 5 is a structural block diagram of a fundus image analysis device in one embodiment;

[0028] Figure 6 is a diagram of the internal structure of a computer device in one embodiment;

[0029] Figure 7 is a diagram showing the internal structure of a computer device in another embodiment;

[0030] Figure 8 Schematic diagram of the target fundus image analysis model training process in one embodiment. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0032] With the aging of the global population and changes in lifestyle, the incidence of brain abnormalities is increasing year by year, becoming a major public health issue leading to death and disability. Therefore, early identification of high-risk groups for effective intervention is particularly important.

[0033] In the existing technology, most of the methods are to image the user's brain through magnetic resonance imaging or computed tomography (CT) to identify and predict the user's brain abnormalities.

[0034] However, this method of identifying and predicting brain abnormalities is costly, which makes it impossible to identify and predict brain abnormalities in large populations.

[0035] In view of this, the present application provides a fundus image analysis method. Since fundus images contain rich brain information and the cost of acquiring fundus images is low, it can realize the identification and prediction of brain abnormalities for a large population.

[0036] The fundus image analysis method provided in the present application can be executed by a computer device, which can be a terminal or a server.

[0037] In an exemplary embodiment, Figure 1 As shown, a fundus image analysis method is provided, which includes the following steps:

[0038] Step 101: Acquire a fundus image of a user, and determine historical brain information of the user based on whether a preset event has occurred in the user's brain within a historical time period.

[0039] Optionally, the fundus image refers to an image that can be used to characterize structural and functional information of the retina, optic nerve, and blood vessels.

[0040] For example, a non-contact photographing technique may be used to obtain the user's fundus image. Specifically, the user's fundus image may be obtained based on a fundus camera or optical coherence tomography.

[0041] The historical time period may be pre-set by a technician based on actual needs, for example, the historical time period may be the past ten years or the past five years.

[0042] The preset event may be a brain abnormality event. Such brain abnormality events include brain structural abnormality events, brain neurological dysfunction events, brain electrophysiological abnormality events, brain metabolic abnormality events, and brain blood circulation abnormality events. Such brain structural abnormality events further include brain tumor events, cerebral infarction events, cerebral hemorrhage events, and brain atrophy events. In the embodiment of the present application, the preset event is a cerebral infarction event among brain structural abnormality events, which may also be referred to as a stroke event.

[0043] For example, the historical brain information can be used to indicate whether a preset event has occurred in a historical time period. The historical brain information may include first information and second information, wherein the first information is used to indicate whether the preset event has not occurred in the user's brain in the historical time period, and the second information is used to indicate whether the preset event has occurred in the user's brain in the historical time period.

[0044] Furthermore, if it is determined that no preset event has occurred in the user's brain within the historical time period, the historical brain information can be determined as the first information; if it is determined that a preset event has occurred in the user's brain within the historical time period, the historical brain information can be determined as the second information.

[0045] Step 102: When the historical brain information indicates that the preset event has not occurred to the user within the historical time period, determine the first fundus image analysis result based on the fundus image and the target fundus image analysis model pre-trained based on the semi-supervised learning algorithm and the knowledge transfer algorithm.

[0046] The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, as well as first prediction information of the user's occurrence of the preset event within a preset time period.

[0047] Optionally, the target fundus image analysis model may be a DeepRETStroke model.

[0048] In the embodiment of the present application, the region with latent brain abnormality refers to the region where latent cerebral infarction occurs.

[0049] The first prediction information may be a prediction result for representing whether the preset event will occur to the user within a preset time period, or may be a prediction probability for representing the preset event occurring to the user within the preset time period.

[0050] In some exemplary embodiments, as described above, the historical brain information may include first information and second information. When the historical brain information is the first information, that is, when no preset event has occurred in the user's brain within the historical time period, the fundus image can be directly input into the target fundus image analysis model to obtain the first fundus image analysis result of the user output by the target fundus image analysis model.

[0051] In other exemplary embodiments, the target fundus image analysis model may also include a first model and a second model. When the historical brain information is the first information, that is, when no preset events have occurred in the user's brain within the historical time period, the fundus image can be directly input into the first model to obtain the first fundus image analysis result of the user output by the first model.

[0052] The above-mentioned fundus image analysis method first obtains the user's fundus image, and determines the user's historical brain information based on whether a preset event has occurred in the user's brain within a historical time period; when the historical brain information indicates that the user has not experienced the preset event within the historical time period, a first fundus image analysis result is determined based on the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm; wherein the first fundus image analysis result is used to characterize whether the user has a hidden brain abnormality area, as well as first prediction information of the user's occurrence of the preset event within a preset time period. The fundus image analysis method provided in the present application can determine whether the user has a hidden brain abnormality area and first prediction information of the user's occurrence of the preset event within a preset time period through the user's fundus image and the user's historical brain information. Since the acquisition cost of the fundus image is relatively low, it can realize the identification and prediction of brain abnormalities for a large population.

[0053] Furthermore, the target fundus image analysis model used in this application is trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm, so that the fundus image analysis results it outputs are more accurate.

[0054] Moreover, compared to most existing technologies in which the fundus image analysis results only include the first prediction information of whether the user will experience a preset event within a preset time period, the fundus image analysis results of the present application also include analysis results of whether the user has hidden brain abnormalities.

[0055] In an exemplary embodiment, when the historical brain information indicates that the preset event occurred to the user within a historical time period, the method further includes: determining a second fundus image analysis result based on the fundus image and the target fundus image analysis model.

[0056] The second fundus image analysis result is used to represent second prediction information of the user's recurrence of the preset event within a preset time period.

[0057] Optionally, the second prediction information may be a prediction result indicating whether the preset event will occur again to the user within a preset time period, or may be a prediction probability indicating whether the preset event will occur again to the user within a preset time period.

[0058] In some exemplary embodiments, as described above, the historical brain information may include first information and second information. When the historical brain information is the second information, that is, when a preset event has occurred in the user's brain within a historical time period, the fundus image can be directly input into the target fundus image analysis model to obtain the second fundus image analysis result of the user output by the target fundus image analysis model.

[0059] In other exemplary embodiments, the target fundus image analysis model may also include a first model and a second model. When the historical brain information is the second information, that is, when a preset event has occurred in the user's brain within a historical time period, the fundus image can be directly input into the second model to obtain the second fundus image analysis result of the user output by the second model.

[0060] In an exemplary embodiment, the target fundus image analysis model includes a first model and a second model, which determines the first fundus image analysis result based on the fundus image and the target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm, including: when the historical brain information indicates that the preset event has not occurred to the user within a historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

[0061] In some exemplary embodiments, when it is determined based on the user's historical brain information that the preset event has not occurred to the user within the historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

[0062] In an exemplary embodiment, the second fundus image analysis result is determined based on the fundus image and the target fundus image analysis model, including: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

[0063] In some exemplary embodiments, when it is determined based on the user's historical brain information that the preset event has occurred to the user within a historical time period, the fundus image is input into the second model to obtain a second fundus image analysis result output by the second model.

[0064] In an exemplary embodiment, Figure 2 As shown, the method of obtaining the user's fundus image includes the following steps:

[0065] Step 201: Acquire a first fundus image of the user, and if a retinal area in the first fundus image meets a preset condition, determine the first fundus image as a second fundus image.

[0066] Optionally, the preset condition can be pre-set by a technician based on actual needs. For example, the preset condition can be that at least 75% of the retinal area in the first fundus image is clearly observable. The preset condition can also be that there are no artifacts in the retinal area in the first fundus image.

[0067] In some exemplary embodiments, after a first fundus image of a user is acquired using a non-contact photographing technique, it may be determined whether the first fundus image meets a preset condition.

[0068] Further, when it is determined that the first fundus image meets a preset condition, the first fundus image is determined as the second fundus image.

[0069] Specifically, when at least 75% of the retinal area in the first fundus image is clearly observable and no artifacts exist in the retinal area in the first fundus image, the first fundus image is determined as the second fundus image.

[0070] Step 202: Perform image enhancement processing on the second fundus image to obtain the fundus image.

[0071] In some exemplary embodiments, contrast-limited adaptive histogram equalization (CLAHE) can be used to enhance the contrast of the second fundus image while suppressing noise. Specifically, the second fundus image can be converted from RGB color space to LAB color space and divided into small blocks of fixed size. CLAHE is applied to the luminance channel of each small block, and the processed second fundus image is converted back to RGB format.

[0072] Furthermore, color normalization is used to reduce the influence of the shooting equipment and the shooting environment on the second fundus image.

[0073] Specifically, , where P refers to the second fundus image, and Gauss(P, s) refers to applying a Gaussian filter with a standard deviation equal to s to the second fundus image P. Refers to adjustable parameters. For example, The resolution of each fundus image was adjusted to 512×512.

[0074] In an exemplary embodiment, Figure 3 As shown, the training method of the target fundus image analysis model includes the following steps:

[0075] Step 301: Obtain training data.

[0076] Among them, the training data includes fundus images of users with hidden brain abnormality areas, fundus images of users without hidden brain abnormality areas, fundus images of users who have not experienced the preset event within the historical time period, fundus images of users who have experienced the preset event within the historical time period, and fundus images of random users.

[0077] The random user is a user for whom it is uncertain whether or not there is a hidden abnormal brain area, and it is uncertain whether or not the preset event has occurred in a historical time period.

[0078] Step 302: Perform model training on the initial fundus image analysis model based on the training data, the semi-supervised learning algorithm, and the knowledge transfer algorithm to obtain a first model.

[0079] The first model can be used to predict whether the preset event will occur to the user within a preset time period, and can also be used to predict the probability of the preset event occurring to the user within the preset time period.

[0080] Step 303: Adjust the first model to obtain a second model, and determine a target fundus image analysis model based on the first model and the second model.

[0081] The second model can be used to predict whether the preset event will occur again to the user within a preset time period, and can also be used to predict the probability of the preset event occurring again to the user within the preset time period.

[0082] In an optional embodiment of the present application, Figure 8 As shown, Figure 8 This is a schematic diagram of the training process of the target fundus image analysis model. The initial fundus image analysis model is trained based on the training data and the semi-supervised learning algorithm to obtain a first model. Before the initial fundus image analysis model is trained, model pre-training and model initialization of the initial fundus image analysis model are first performed.

[0083] Specifically, the initial fundus image analysis model can be used Figure 8 The architecture shown in the figure. The encoder of the initial fundus image analysis model can adopt a large visual Transformer network architecture, which can include 24 Transformer modules and an embedding vector size of 1024; the decoder of the initial fundus image analysis model can adopt a small visual Transformer network architecture, which can include 8 Transformer modules and an embedding vector size of 512.

[0084] In the process of pre-training the initial fundus image analysis model using the MAE algorithm, the training data will be divided into 16×16 patch blocks, 75% of the patch blocks are masked, and the remaining 25% of the patch blocks are used as the input of the model to participate in the model update. The loss function in this process is , where N refers to the total number of training data, N p Refers to the total number of pixels that are not masked on each training data, and C refers to the total number of image channels. s , k, q refers to the model's predicted value of the qth channel of the kth pixel in the training data numbered s, x s , k, q refers to the true value of the qth channel of the kth pixel in the training data numbered s, .

[0085] Furthermore, the diagnostic head of the initial fundus image analysis model can use two logistic regression classifiers with the same structure. The learning head and the prediction head can use multi-layer perceptrons with output results of 2 dimensions and 5 dimensions respectively. Based on the first training data in the training data, the learning of the task of "predicting the probability of the user having the preset event within the preset time period" is performed to update the encoder and prediction head parts in the model and complete the model initialization. The first training data is the fundus image of the user who has not had the preset event within the historical time period. The loss function in this process is , where N represents the total number of first training data, t represents the year, and T i represents the total number of follow-up years for the user numbered i in the first training data. (If a preset event occurs during the follow-up period or the user follows up for the preset time period but no preset event occurs, T i =5; if no preset event occurs during the follow-up period but the follow-up time is less than the preset time period, T i <5), x i represents the fundus image of user numbered i (i=1, 2, 3, ..., N), p θ，t (x i ) represents the probability of a preset event occurring before time point t, y i，t Represents the user status at time point t (users who have a preset event before time point t, y i，t =1; users who have not experienced the preset event, y i，t =0).

[0086] Furthermore, the process of training a model using a semi-supervised learning algorithm and a knowledge transfer algorithm can be viewed as a large cycle, with each cycle consisting of the steps of "semi-supervised learning" and "knowledge transfer." At the same time, each large cycle of "semi-supervised learning" can be viewed as a small cycle, with each small cycle consisting of the steps of "updating the labeled database" and "updating the diagnostic head."

[0087] Exemplarily, the training data can be defined as two new databases, namely a "labeled database" and an "unlabeled database." Before the start of this small cycle, the "labeled database" includes fundus images of all users with hidden brain abnormalities and fundus images of users without hidden brain abnormalities, and the "unlabeled database" includes fundus images of all users who have not experienced the preset event within the historical time period. After the small cycle starts, for each small cycle training round, the "labeled database" is first used to update the diagnostic head of the model by learning the task of "identifying hidden brain abnormalities."

[0088] After the update is complete, the model is used to perform the "hidden brain abnormality area identification" task on the "validation database" of fundus images of users with hidden brain abnormalities and users without hidden brain abnormalities. The probability of each fundus image of a user with a hidden brain abnormality corresponding to a hidden brain abnormality area is determined. The minimum probability of the predetermined event occurring that maintains the prediction accuracy of positive samples (the proportion of true positive samples among samples predicted as positive) above 0.75 is then selected as the high prediction confidence standard pos_thr for positive samples. The maximum probability of the predetermined event occurring that maintains the prediction accuracy of negative samples (the proportion of true negative samples among samples predicted as negative) above 0.75 is then selected as the high prediction confidence standard neg_thr for negative samples.

[0089] This model is then used to perform the "identification of hidden brain abnormality regions" task on the "unlabeled database." Training data with a predicted probability greater than pos_thr is assigned a positive "pseudo-label," while training data with a predicted probability less than neg_thr is assigned a negative "pseudo-label." These training data are then removed from the "unlabeled database" and added to the "labeled database." In the next training round, this expanded "labeled database" is used to re-update the model's diagnostic header. In this way, samples from the "unlabeled database" are continuously added to the "labeled database" as the mini-cycle continues until the "unlabeled database" is empty or no samples with high prediction confidence can be assigned a "pseudo-label." This mini-cycle (the "training the initial fundus image analysis model based on the training data and semi-supervised learning algorithm" step) ends.

[0090] Exemplarily, the first model is used to predict the task of "hidden brain abnormal areas" on the "fundus images of users who have not experienced the preset event within the historical time period", and each training data is assigned a "prediction probability" as a "soft label" to obtain a database with "soft labels" added.

[0091] Through the first model's learning of the tasks of "predicting the probability of a user experiencing a preset event within a preset time period" and "soft label prediction", the encoder, learning head, and prediction head in the model are updated. The loss function in this process is

[0092] ;

[0093] The loss function consists of two parts. The first half is the same as the loss function in the "model initialization" stage, which represents the prediction loss of the task of "predicting the probability of a user experiencing a preset event within a preset time period". The second half represents the prediction loss of the "soft label prediction" task, where represents the fundus image of user number i, , represents the true probability distribution of the “soft label” on category j, represents the predicted probability distribution output by the first model for category j. α is the weight parameter controlling the tasks of "soft label prediction" and "predicting the probability of a user experiencing a preset event within a preset time period." α can be 0.3. This approach allows the first model to simultaneously learn about a user's brain status during baseline and follow-up, thereby improving the accuracy of its prediction of the probability of a user experiencing a preset event within a preset time period.

[0094] After the update is complete, the first model is used to predict the probability of a user experiencing the preset event within the preset time period using the verification database of fundus images of users who have not experienced the preset event within the historical time period. If the model performance meets the pre-set standard, the first model is complete and the entire macro cycle ends; otherwise, the next macro cycle update begins.

[0095] As described above, after the initial fundus image analysis model is trained based on the training data and the semi-supervised learning algorithm to obtain a first model, the first model needs to be adjusted to obtain a second model, and the target fundus image analysis model is determined based on the first model and the second model.

[0096] The first model (especially the encoder part) can be used as a pre-training model with strong prior knowledge, thus playing a certain role in the training of other brain abnormality prediction tasks. Therefore, this application uses the encoder of the first model as a pre-training model, and develops a second model based on the first model after fine-tuning. The loss function of this process is , it can be found that the loss function is expressed in the same form as the loss function of the "model initialization" step in the previous stage. The difference is that the observed event changes from the occurrence of the preset event to the recurrence of the preset event.

[0097] In an exemplary embodiment, Figure 4 As shown, another fundus image analysis method is provided, which includes the following steps:

[0098] Step 401: Acquire a first fundus image of the user. If a retinal area in the first fundus image satisfies a preset condition, determine the first fundus image as a second fundus image. Perform image enhancement processing on the second fundus image to obtain the fundus image. Determine historical brain information of the user based on whether a preset event has occurred in the user's brain within a historical time period.

[0099] Step 402: If the historical brain information indicates that the user has not experienced the preset event within the historical time period, input the fundus image into a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm to obtain a first fundus image analysis result output by the first model; wherein the first fundus image analysis result is used to indicate whether the user has a hidden brain abnormality area and first prediction information of the occurrence of the preset event within the preset time period.

[0100] Step 403: When the historical brain information indicates that the preset event has occurred to the user within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model; wherein the second fundus image analysis result is used to represent the second prediction information that the preset event will occur again to the user within the preset time period.

[0101] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0102] Based on the same inventive concept, embodiments of the present application also provide a fundus image analysis device for implementing the fundus image analysis method described above. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations of one or more fundus image analysis device embodiments provided below can be found in the limitations of the fundus image analysis method described above and will not be further elaborated here.

[0103] In an exemplary embodiment, Figure 5 As shown, a fundus image analysis device 500 is provided, comprising: an acquisition module 501 and an execution module 502, wherein:

[0104] The acquisition module 501 is used to acquire a fundus image of a user and determine the user's historical brain information based on whether a preset event has occurred in the user's brain within a historical time period;

[0105] An execution module 502 is configured to determine a first fundus image analysis result based on the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm, if the historical brain information indicates that the preset event has not occurred to the user within a historical time period;

[0106] The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, as well as first prediction information of the user's occurrence of the preset event within a preset time period.

[0107] In one embodiment, the execution module 502 is also used to determine a second fundus image analysis result based on the fundus image and the target fundus image analysis model when the historical brain information indicates that the preset event has occurred to the user within a historical time period; wherein the second fundus image analysis result is used to represent second prediction information that the preset event will occur again to the user within a preset time period.

[0108] In one embodiment, the target fundus image analysis model includes a first model and a second model. The execution module 502 is specifically used to input the fundus image into the first model when the historical brain information indicates that the preset event has not occurred to the user within the historical time period, so as to obtain the first fundus image analysis result output by the first model.

[0109] In one embodiment, the execution module 502 is specifically used to input the fundus image into the second model when the historical brain information indicates that the preset event has occurred to the user within a historical time period, so as to obtain the second fundus image analysis result output by the second model.

[0110] In one embodiment, the acquisition module 501 is specifically used to acquire a first fundus image of the user, and when the retinal area in the first fundus image meets a preset condition, the first fundus image is determined as a second fundus image; and image enhancement processing is performed on the second fundus image to obtain the fundus image.

[0111] In one embodiment, the execution module 502 is also used to obtain training data; the training data includes fundus images of users with the hidden brain abnormality area, fundus images of users without the hidden brain abnormality area, fundus images of users who have not experienced the preset event within the historical time period, fundus images of users who have experienced the preset event within the historical time period, and fundus images of random users; based on the training data, the semi-supervised learning algorithm and the knowledge transfer algorithm, the initial fundus image analysis model is trained to obtain the first model; the first model is adjusted to obtain the second model, and the target fundus image analysis model is determined based on the first model and the second model.

[0112] Each module in the fundus image analysis device described above can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0113] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a fundus image analysis method is implemented.

[0114] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 7As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means, and the wireless means can be implemented via Wi-Fi, a mobile cellular network, near-field communication (NFC), or other technologies. When executed by the processor, the computer program implements a fundus image analysis method. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0115] Those skilled in the art will understand that Figure 6 or Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0116] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0117] Obtaining a fundus image of the user and determining the user's historical brain information based on whether a preset event has occurred in the user's brain within a historical time period;

[0118] When the historical brain information indicates that the preset event has not occurred to the user within the historical time period, determining a first fundus image analysis result based on the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm;

[0119] The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, as well as first prediction information of the user's occurrence of the preset event within a preset time period.

[0120] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, a second fundus image analysis result is determined based on the fundus image and the target fundus image analysis model; wherein the second fundus image analysis result is used to represent second prediction information that the preset event will occur again to the user within a preset time period.

[0121] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the historical brain information indicates that the preset event has not occurred to the user within the historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

[0122] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

[0123] In one embodiment, when the processor executes the computer program, it also implements the following steps: obtaining a first fundus image of the user, and when the retinal area in the first fundus image meets a preset condition, determining the first fundus image as a second fundus image; performing image enhancement processing on the second fundus image to obtain the fundus image.

[0124] In one embodiment, when the processor executes the computer program, it also implements the following steps: obtaining training data; the training data includes fundus images of users with the hidden brain abnormality area, fundus images of users without the hidden brain abnormality area, fundus images of users who have not experienced the preset event within the historical time period, fundus images of users who have experienced the preset event within the historical time period, and fundus images of random users; based on the training data, the semi-supervised learning algorithm and the knowledge transfer algorithm, the initial fundus image analysis model is trained to obtain the first model; the first model is adjusted to obtain the second model, and the target fundus image analysis model is determined based on the first model and the second model.

[0125] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0126] Obtaining a fundus image of the user and determining the user's historical brain information based on whether a preset event has occurred in the user's brain within a historical time period;

[0127] When the historical brain information indicates that the preset event has not occurred to the user within the historical time period, determining a first fundus image analysis result based on the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm;

[0128] The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, as well as first prediction information of the user's occurrence of the preset event within a preset time period.

[0129] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, a second fundus image analysis result is determined based on the fundus image and the target fundus image analysis model; wherein the second fundus image analysis result is used to represent the second prediction information that the preset event will occur again to the user within the preset time period.

[0130] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the historical brain information indicates that the preset event has not occurred to the user within the historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

[0131] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

[0132] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: obtaining a first fundus image of the user, and when the retinal area in the first fundus image meets a preset condition, determining the first fundus image as a second fundus image; performing image enhancement processing on the second fundus image to obtain the fundus image.

[0133] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining training data; the training data includes fundus images of users with the hidden brain abnormality area, fundus images of users without the hidden brain abnormality area, fundus images of users who have not experienced the preset event within the historical time period, fundus images of users who have experienced the preset event within the historical time period, and fundus images of random users; based on the training data, the semi-supervised learning algorithm and the knowledge transfer algorithm, the initial fundus image analysis model is trained to obtain the first model; the first model is adjusted to obtain the second model, and the target fundus image analysis model is determined based on the first model and the second model.

[0134] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0135] Obtaining a fundus image of the user and determining the user's historical brain information based on whether a preset event has occurred in the user's brain within a historical time period;

[0136] When the historical brain information indicates that the preset event has not occurred to the user within the historical time period, determining a first fundus image analysis result based on the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm;

[0137] The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, as well as first prediction information of the user's occurrence of the preset event within a preset time period.

[0138] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, a second fundus image analysis result is determined based on the fundus image and the target fundus image analysis model; wherein the second fundus image analysis result is used to represent the second prediction information that the preset event will occur again to the user within the preset time period.

[0139] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the historical brain information indicates that the preset event has not occurred to the user within the historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

[0140] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the historical brain information indicates that the preset event has occurred to the user within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

[0141] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: obtaining a first fundus image of the user, and when the retinal area in the first fundus image meets a preset condition, determining the first fundus image as a second fundus image; performing image enhancement processing on the second fundus image to obtain the fundus image.

[0142] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining training data; the training data includes fundus images of users with the hidden brain abnormality area, fundus images of users without the hidden brain abnormality area, fundus images of users who have not experienced the preset event within the historical time period, fundus images of users who have experienced the preset event within the historical time period, and fundus images of random users; based on the training data, the semi-supervised learning algorithm and the knowledge transfer algorithm, the initial fundus image analysis model is trained to obtain the first model; the first model is adjusted to obtain the second model, and the target fundus image analysis model is determined based on the first model and the second model.

[0143] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0144] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0145] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0146] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A fundus image analysis method, characterized in that: The method comprises: Acquire a fundus image of the user, and determine historical brain information of the user based on whether a preset event has occurred in the user's brain within a historical time period; When the historical brain information indicates that the user has not experienced the preset event within a historical time period, determining a first fundus image analysis result based on the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm; The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, and first prediction information of the user occurring the preset event within a preset time period.

2. The method according to claim 1, characterized in that The method further comprises: When the historical brain information indicates that the preset event occurred to the user within a historical time period, determining a second fundus image analysis result according to the fundus image and the target fundus image analysis model; The second fundus image analysis result is used to represent second prediction information of the user reoccurring the preset event within a preset time period.

3. The method according to claim 1, characterized in that The target fundus image analysis model includes a first model and a second model, and determining a first fundus image analysis result according to the fundus image and the target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm includes: When the historical brain information indicates that the preset event has not occurred to the user within a historical time period, the fundus image is input into the first model to obtain the first fundus image analysis result output by the first model.

4. The method according to claim 3, characterized in that The determining a second fundus image analysis result according to the fundus image and the target fundus image analysis model includes: When the historical brain information indicates that the preset event occurred to the user within a historical time period, the fundus image is input into the second model to obtain the second fundus image analysis result output by the second model.

5. The method according to claim 1, wherein The obtaining of the user's fundus image includes: Acquiring a first fundus image of the user, and determining the first fundus image as a second fundus image if a retinal area in the first fundus image meets a preset condition; Perform image enhancement processing on the second fundus image to obtain the fundus image.

6. The method according to any one of claims 1 to 5, characterized in that: The training method of the target fundus image analysis model includes: Acquire training data; the training data includes fundus images of users with the hidden brain abnormality area, fundus images of users without the hidden brain abnormality area, fundus images of users who have not experienced the preset event within a historical time period, fundus images of users who have experienced the preset event within a historical time period, and fundus images of random users; Performing model training on an initial fundus image analysis model based on the training data, the semi-supervised learning algorithm, and the knowledge transfer algorithm to obtain the first model; The first model is adjusted to obtain the second model, and the target fundus image analysis model is determined based on the first model and the second model.

7. A fundus image analysis device, characterized in that: The device comprises: an acquisition module, configured to acquire a fundus image of a user and determine historical brain information of the user based on whether a preset event has occurred in the user's brain within a historical time period; an execution module, configured to determine a first fundus image analysis result based on the fundus image and a target fundus image analysis model pre-trained based on a semi-supervised learning algorithm and a knowledge transfer algorithm, when the historical brain information indicates that the preset event has not occurred to the user within a historical time period; The first fundus image analysis result is used to characterize whether the user has a hidden abnormal brain area, and first prediction information of the user occurring the preset event within a preset time period.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images