Image recognition method and device for eye fundus image, computer equipment and storage medium

Abstract

The invention relates to an eye fundus image recognition method and device, computer equipment and a storage medium. The method comprises the steps of inputting a fundus image of a to-be-detected patient into a feature extraction model obtained by performing federal training on an initial mask auto-encoder model based on a historical fundus image to obtain a feature image corresponding to the fundus image, and then inputting the feature image and clinical index information of the to-be-detected patient into an evaluation identification model, the damage condition of the hypertension target organ and / or the damage probability of the hypertension target organ are / is obtained. According to the embodiment of the invention, the hypertension diagnosis and prediction result of the to-be-detected patient can be determined only based on the fundus image and the clinical index information of the to-be-detected patient, the operation process of hypertension target organ injury assessment is simplified, the assessment cost is reduced, and assessment is performed based on the feature extraction model and the assessment identification model, so that the assessment efficiency is improved. And the accuracy of evaluating the hypertension target organ injury is also improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image recognition method, device, computer equipment and storage medium for fundus images. Background Art

[0002] Image recognition of fundus images refers to the structural or functional changes in the heart, arteries, kidneys, retina, and brain caused by increased blood pressure. It is a sign of preclinical or asymptomatic cardiovascular disease and has a strong early warning effect on the occurrence of subsequent cardiovascular and cerebrovascular events. Therefore, evaluating the image recognition of fundus images is of great significance for guiding the diagnosis and treatment of hypertensive patients and reducing the occurrence of complications.

[0003] Traditional assessment of target organ damage in hypertension usually involves multiple organs and requires technical support from a variety of laboratory tests and imaging examinations, which places high demands on the examiner's technical level and related medical investment.

[0004] Therefore, how to simplify the evaluation of target organ damage in hypertension has become a technical problem that needs to be solved urgently in this field. Summary of the invention

[0005] Based on this, it is necessary to provide an image recognition method, device, computer equipment and storage medium for fundus images that can simplify the evaluation operation of target organ damage caused by hypertension in order to address the above technical problems.

[0006] In a first aspect, the present application provides an image recognition method for fundus images. The method comprises:

[0007] Inputting the fundus image of the patient to be tested into the feature extraction model to obtain a feature image corresponding to the fundus image; the feature extraction model is obtained by federally training the initial mask autoencoder model based on the historical fundus images;

[0008] The characteristic image and the clinical indicator information of the patient to be tested are input into the evaluation and recognition model to obtain an evaluation result; the evaluation result includes: the damage status of the target organs of hypertension, and / or the probability of damage to the target organs of hypertension.

[0009] In one embodiment, the feature extraction model includes an embedding layer, an encoder, and a decoder; the fundus image of the patient to be detected is input into the feature extraction model to obtain a feature image corresponding to the fundus image, including:

[0010] Inputting the fundus image of the patient to be detected into the embedding layer for preliminary feature extraction to obtain a first feature vector;

[0011] Inputting the first feature vector into the encoder for feature abstraction to obtain a second feature vector of a preset length; the preset length is greater than the length of the first feature vector;

[0012] The second feature vector is input to the decoder to obtain a feature image corresponding to the fundus image.

[0013] In one embodiment, the method further comprises:

[0014] Masking a first preset number of images in the historical fundus images to obtain mask images;

[0015] Annotating a second preset number of images in the historical fundus images to obtain annotated images;

[0016] Based on the mask image, the annotated image, and the historical fundus image, the initial mask autoencoder model is federated trained to obtain the feature extraction model.

[0017] In one embodiment, the initial mask autoencoder model is federated trained based on the mask image, the annotated image, and the historical fundus image to obtain the feature extraction model, including:

[0018] Training the initial mask autoencoder model based on the mask image and the historical fundus image to obtain a first intermediate mask autoencoder model;

[0019] Training the first intermediate mask autoencoder model based on the annotated image and the historical fundus image to obtain a second intermediate mask autoencoder model;

[0020] The masked autoencoder model is determined based on the first model parameters of the second intermediate masked autoencoder model and the second model parameters of other participants in the federated training; the second model parameters are the model parameters of the second intermediate masked autoencoder model obtained by other participant devices in the federated training.

[0021] In one embodiment, the determining the masked autoencoder model based on the first model parameters of the second intermediate masked autoencoder model and the second model parameters of other participants in the federated training includes:

[0022] Acquire a shared model parameter from the second model parameter; the shared model parameter is a model parameter of a preset proportion in the second model parameter;

[0023] Aggregating the shared model parameter and the first model parameter to obtain a target model parameter;

[0024] The masked autoencoder model is determined according to the target model parameters.

[0025] In one embodiment, the characteristic image and the clinical indicator information of the patient to be detected are input into a risk prediction model to obtain a risk prediction result, including:

[0026] Inputting the characteristic image and the clinical indicator information of the patient to be tested into a risk prediction model to obtain a risk prediction score;

[0027] The risk prediction result is determined based on the risk prediction score and the survival curve; the survival curve is determined based on the historical risk prediction score.

[0028] In one embodiment, the evaluation and recognition model includes a diagnosis model and / or a risk prediction model, and the characteristic image and the clinical indicator information of the patient to be detected are input into the evaluation and recognition model to obtain an evaluation result, including:

[0029] The characteristic image and the clinical index information of the patient to be detected are input into the diagnostic model to obtain a diagnostic result; the diagnostic result is used to characterize the hypertensive target organ damage of the patient to be detected;

[0030] The characteristic image and the clinical indicator information of the patient to be tested are input into the risk prediction model to obtain a risk prediction result; the risk prediction result is used to characterize the probability of damage to the hypertensive target organs of the patient to be tested.

[0031] In a second aspect, the present application also provides an image recognition device for fundus images. The device comprises:

[0032] A first determination module is used to input the fundus image of the patient to be detected into a feature extraction model to obtain a feature image corresponding to the fundus image; the feature extraction model is obtained by federally training an initial mask autoencoder model based on historical fundus images;

[0033] The second determination module is used to input the characteristic image and the clinical indicator information of the patient to be detected into the evaluation and recognition model to obtain an evaluation result; the evaluation result includes: the damage status of the target organs of hypertension, and / or the probability of damage to the target organs of hypertension.

[0034] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any of the above methods when executing the computer program.

[0035] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above methods when executed by a processor.

[0036] The above-mentioned image recognition method, device, computer equipment and storage medium for fundus images are obtained by inputting the fundus image of the patient to be tested into a feature extraction model obtained by federally training the initial mask autoencoder model based on historical fundus images to obtain a feature image corresponding to the fundus image, and then inputting the feature image and the clinical indicator information of the patient to be tested into the evaluation and recognition model to obtain the damage of the target organs of hypertension, and / or the probability of damage to the target organs of hypertension. In the embodiment of the present application, the diagnosis and prediction results of hypertension of the patient to be tested can be determined based on the fundus image and clinical indicator information of the patient to be tested, and there is no need to examine multiple organs involved in the target organ damage of hypertension, which simplifies the operational process of the evaluation of target organ damage of hypertension, reduces the evaluation cost, and the evaluation based on the feature extraction model and the evaluation and recognition model also improves the accuracy of the evaluation of target organ damage of hypertension. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] FIG1 is an internal structure diagram of a computer device provided in an embodiment of the present application;

[0038] FIG2 is a flow chart of an image recognition method for fundus images provided in an embodiment of the present application;

[0039] FIG3 is a flow chart of a hypertension screening and prediction method provided in an embodiment of the present application;

[0040] FIG4 is a schematic diagram of a flow chart of a method for acquiring a feature image according to an embodiment of the present application;

[0041] FIG5 is a schematic diagram of a flow chart of a method for acquiring a feature extraction model provided in an embodiment of the present application;

[0042] FIG6 is a flow chart of a method for determining a feature extraction model provided in an embodiment of the present application;

[0043] FIG7 is a flow chart of another method for determining a feature extraction model provided in an embodiment of the present application;

[0044] FIG8 is a schematic diagram of a process flow of federated training provided in an embodiment of the present application;

[0045] FIG9 is a flow chart of a method for obtaining an evaluation result provided in an embodiment of the present application;

[0046] FIG10 is a schematic diagram of a flow chart of a method for obtaining risk prediction results provided in an embodiment of the present application;

[0047] FIG11 is a flow chart of a method for diagnosing and early warning of hypertension and target organ damage caused by hypertension based on fundus images according to an embodiment of the present application;

[0048] FIG. 12 is a structural block diagram of an image recognition device for fundus images provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] Image recognition of fundus images refers to the structural or functional changes in the heart, arteries, kidneys, retina, and brain caused by increased blood pressure. It is a sign of preclinical or asymptomatic cardiovascular disease and has a strong early warning effect on the occurrence of subsequent cardiovascular and cerebrovascular events. Therefore, evaluating the image recognition of fundus images is of great significance for guiding the diagnosis and treatment of hypertensive patients and reducing the occurrence of complications.

[0051] Traditional assessment of target organ damage in hypertension usually involves multiple organs and requires technical support from a variety of laboratory tests and imaging examinations, which places high demands on the examiner's technical level and related medical investment.

[0052] For computer vision models, models trained on a single center are usually prone to overfitting, and their performance on other centers, sample distributions, and tasks may drop significantly, and more training samples often lead to better model performance. In real multi-center medical collaboration, there may be problems such as data not being interoperable between institutions and no effective solutions to privacy issues, which makes model training in the data set difficult.

[0053] Therefore, how to simplify the evaluation of target organ damage in hypertension has become a technical problem that needs to be solved urgently in this field.

[0054] The image recognition method for fundus images provided in an embodiment of the present application can be applied in an application environment as shown in Figure 1. Figure 1 is an internal structure diagram of a computer device provided in an embodiment of the present application. The computer device can be a server, and its internal structure diagram can be as shown in Figure 1. The computer device includes a processor, a memory, and a network interface connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for image recognition of fundus images is implemented.

[0055] Those skilled in the art will understand that the structure shown in FIG. 1 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0056] In one embodiment, as shown in FIG. 2 , FIG. 2 is a flow chart of an image recognition method for fundus images provided in an embodiment of the present application. The method can be applied to the computer device in FIG. 1 . The method includes the following steps:

[0057] S201, inputting the fundus image of the patient to be detected into a feature extraction model to obtain a feature image corresponding to the fundus image.

[0058] In one embodiment, the feature extraction model is obtained by federally training an initial Masked Auto-Encoders (MAE) model based on historical fundus images, wherein the feature extraction model is composed of an embedding layer (Embedding), an encoder (Encoder) and a decoder (Decoder).

[0059] For example, The original fundus image of the patient to be tested is converted into 3 color channels by bicubic interpolation algorithm. resolution fundus image, and then convert the 3 color channels, The fundus image with high resolution is input to the embedding layer according to the convolution kernel , and perform convolution operation with a step length of 16 to obtain a length of The first feature vector is then input into the encoder for feature abstraction, and a second feature vector of length 1024 is extracted. The second feature vector is then input into the decoder to obtain a 3-color channel, Resolution feature image.

[0060] Optionally, when the initial Masked Auto-Encoders (MAE) model is federated trained, each participant in the federated training may respectively mask a first preset number of images in the historical fundus images to obtain mask images, and then train the initial masked auto-encoder model based on the mask images to obtain a first intermediate masked auto-encoder model, so that the image generated by the model based on the mask image is as close to the original image as possible. In addition, each participant in the federated training may also annotate a second preset number of images in the historical fundus images to obtain annotated images, and then perform semi-supervised training on the first intermediate masked auto-encoder model based on the annotated images to obtain a second intermediate masked auto-encoder model. Then, the target model parameters may be obtained based on the first model parameters of the second intermediate masked auto-encoder model and the second model parameters of other participants in the federated training, and the target model parameters may be used as the model parameters of the feature extraction model.

[0061] S202, inputting the characteristic image and the clinical indicator information of the patient to be detected into the evaluation and recognition model to obtain an evaluation result.

[0062] The evaluation results include: the status of hypertensive target organ damage, and / or the probability of hypertensive target organ damage.

[0063] Exemplarily, the clinical indicator information of the patient to be tested may include, for example, the age, gender, body mass index (BMI), smoking status, diabetes, triglycerides (TG), low-density lipoprotein cholesterol (LDL-C), and high-density lipoprotein cholesterol (HDL-C) of the patient to be tested.

[0064] In one embodiment, the evaluation and identification model may include a diagnostic model and / or a risk prediction model. The characteristic image and the clinical indicator information of the patient to be detected may be input into the diagnostic model together to obtain a diagnostic result output by the diagnostic model, and the diagnostic result is used to characterize the hypertensive target organ damage of the patient to be detected. The characteristic image and the clinical indicator information of the patient to be detected may also be input into the risk prediction model together to obtain a risk prediction result, and the risk prediction result is used to characterize the probability of damage to the hypertensive target organ of the patient to be detected.

[0065] Optionally, the above 8 clinical indicator information can be combined into an indicator vector Indicates. Among them, Represents the scalar value of each indicator. In other words, the indicator vector and the feature image vector can be used together as the input of the evaluation recognition model to obtain the evaluation result.

[0066] Exemplarily, refer to FIG. 3, which is a flowchart of hypertension screening and prediction provided by an embodiment of the present application. Opportunistic screening can be performed on community populations through blood pressure measurement. If the screening result is positive, the resident is determined to be a patient to be tested (i.e., a hypertensive patient). The characteristic image and clinical indicator information of the fundus image of the patient to be tested are then used as inputs to the diagnostic model to perform cross-sectional auxiliary diagnosis of target organ damage of hypertension and cardiovascular and cerebrovascular diseases, and the characteristic image and clinical indicator information of the fundus image of the patient to be tested are used as inputs to the risk prediction model to perform longitudinal risk prediction.

[0067] In an embodiment of the present application, the fundus image of the patient to be detected is input into a feature extraction model obtained by federally training the initial mask autoencoder model based on the historical fundus image to obtain a feature image corresponding to the fundus image, and then the feature image and the clinical indicator information of the patient to be detected are input into the evaluation and recognition model to obtain the damage of the target organ of hypertension, and / or the probability of damage to the target organ of hypertension. In the embodiment of the present application, only the fundus image and clinical indicator information of the patient to be detected can be used to determine the diagnosis and prediction results of hypertension in the patient to be detected, and there is no need to examine multiple organs involved in the target organ damage of hypertension, which simplifies the operational process of the evaluation of target organ damage of hypertension, reduces the evaluation cost, and based on the feature extraction model and the evaluation and recognition model, the accuracy of the evaluation of target organ damage of hypertension is also improved. At the same time, federal training can also break the data islands in the field of medical applications, expand the amount of training data under the premise of protecting the data privacy of all parties, and further improve the reliability and robustness of the feature extraction model.

[0068] In one embodiment, the feature extraction model includes an embedding layer, an encoder, and a decoder. Referring to FIG. 4 , FIG. 4 is a flow chart of a feature image acquisition method provided in an embodiment of the present application. This implementation involves a possible implementation method of how to input the fundus image of the patient to be detected into the feature extraction model to obtain a feature image corresponding to the fundus image. Based on the above embodiment, the above S202 includes the following steps:

[0069] S401, inputting the fundus image of the patient to be detected into the embedding layer for preliminary feature extraction to obtain a first feature vector.

[0070] For example, The original fundus image of the patient to be tested is converted into 3 color channels by bicubic interpolation algorithm. resolution fundus image, and then convert the 3 color channels, The fundus image with high resolution is input to the embedding layer according to the convolution kernel , and perform convolution operation with a step length of 16 to obtain a length of The first eigenvector of .

[0071] S402: Input the first feature vector to an encoder for feature abstraction to obtain a second feature vector of a preset length.

[0072] The preset length is greater than the length of the first eigenvector.

[0073] In the embodiment of the present application, a preset length may be set according to the length of the first feature vector, so that the preset length is greater than the length of the first feature vector, so as to perform feature abstraction on the first feature vector and reduce the complexity of the data.

[0074] Exemplarily, the first feature vector may be input into an encoder for feature abstraction, and a second feature vector with a length of 1024 may be extracted.

[0075] S403: Input the second feature vector to a decoder to obtain a feature image corresponding to the fundus image.

[0076] Exemplarily, the second feature vector can be input to a decoder to obtain a 3-color channel, The feature image of resolution, the 3 color channels, The characteristic image of the resolution is the characteristic image corresponding to the fundus image.

[0077] In an embodiment of the present application, the fundus image of the patient to be tested is input into the embedding layer for preliminary feature extraction to obtain a first feature vector, the first feature vector is input into the encoder for feature abstraction to obtain a second feature vector of a preset dimension, and the second feature vector is input into the decoder to obtain a feature image corresponding to the fundus image, so that the features in the fundus image of the patient to be tested can be extracted through the feature extraction model, which facilitates the subsequent diagnosis of target organ damage and risk prediction of hypertension based on the feature image, thereby improving the accuracy and reliability of the diagnosis and risk prediction of target organ damage of hypertension.

[0078] Referring to Figure 5, Figure 5 is a flow chart of a method for acquiring a feature extraction model provided in an embodiment of the present application. Based on the above embodiment, the method further includes the following steps:

[0079] S501 , masking a first preset number of images in historical fundus images to obtain mask images.

[0080] Optionally, historical fundus images of hypertensive patients and normal residents can be obtained from the Internet or a local database.

[0081] In one embodiment, a first preset number of images may be randomly extracted from historical fundus images, and then a mask is performed on some image blocks of the first preset number of images to obtain a mask image.

[0082] Exemplarily, a first preset number of images may be randomly selected from historical fundus images, and then 75% of the image blocks of each of the first preset number of images may be randomly masked to obtain a mask image.

[0083] It should be noted that the first preset number may be less than or equal to the number of historical fundus images.

[0084] S502: Annotate a second preset number of images in the historical fundus images to obtain annotated images.

[0085] In one embodiment, a second preset number of images may be randomly selected from historical fundus images, and then the hypertensive target organ damage areas in the second preset number of images may be annotated to obtain annotated images.

[0086] It should be noted that the second preset number may be less than or equal to the number of historical fundus images.

[0087] S503, based on the mask image, the annotated image, and the historical fundus image, the initial mask autoencoder model is federated trained to obtain a feature extraction model.

[0088] Exemplarily, assuming that 75% of the image blocks of each of the first preset number of images are randomly masked to obtain a masked image, the initial masked autoencoder model can be trained based on the masked image and the historical fundus image to obtain a first intermediate masked autoencoder model. Pixel loss is used as the loss function of the training so that the image generated by the model based on the remaining 25% of the image blocks that are not masked is as close as possible to the original historical fundus image.

[0089] In one embodiment, the first intermediate mask autoencoder model can be trained based on the annotated images and the historical fundus images to obtain the second intermediate mask autoencoder model: the annotated images can be used to fine-tune a ResNet-50 model pre-trained on ImageNet as a teacher model for semi-supervised knowledge distillation. Then the historical fundus image is input into the pre-trained ResNet-50 model for image recognition, and the image recognition result output by the ResNet-50 model is used as the historical fundus image Pseudo labels ,Right now:

[0090]

[0091] in, is the indicative function, is the Sigmoid function. Then the pseudo label As a signal, a supervised learning classification task is set to train the first intermediate mask autoencoder model to obtain a second intermediate mask autoencoder model.

[0092] Optionally, cross entropy can be used as the loss function for the classification task. In the above training process based on mask images, annotated images, and historical fundus images, the loss function is It can be expressed as:

[0093]

[0094] in, , They are pixel loss function and cross entropy function respectively.

[0095] In an embodiment of the present application, there are multiple participants in the federated training, and the above-mentioned process of training the initial mask autoencoder model based on the mask image, the annotated image, and the historical fundus image to obtain the first intermediate mask autoencoder model and the second intermediate mask autoencoder model needs to be performed separately by each participant in the federated training.

[0096] Exemplarily, assuming that there are N participants in the federated training, each participant will obtain a second intermediate masked autoencoder model based on its local data training, that is, a total of N second intermediate masked autoencoder models with different model parameters will be obtained.

[0097] In one embodiment, a participant in the federated training may aggregate the first model parameters of the local second intermediate mask autoencoder model with the second model parameters of other participants in the federated training to obtain target model parameters, and determine the target model parameters as the model parameters of the feature extraction model.

[0098] In the embodiment of the present application, a first preset number of images in the historical fundus images are masked to obtain mask images, a second preset number of images in the historical fundus images are annotated to obtain an annotated image, and based on the mask image, the annotated image, and the historical fundus image, the initial mask autoencoder model is federated trained to obtain a feature extraction model, so that the characteristics of hypertensive target organ damage can be learned based on the historical fundus images, thereby improving the accuracy of the feature extraction model. Moreover, training based on mask images and annotated images can further improve the accuracy and robustness of the feature extraction model, facilitate the subsequent diagnosis and risk prediction of hypertensive target organ damage based on feature images, and improve the accuracy and reliability of the diagnosis and risk prediction of hypertensive target organ damage.

[0099] Referring to Figure 6, Figure 6 is a flow chart of a method for determining a feature extraction model provided in an embodiment of the present application. This implementation involves how to perform federated training on an initial mask autoencoder model based on a mask image, annotated image, and historical fundus image to obtain a possible implementation of a feature extraction model. Based on the above embodiment, the above S503 includes the following steps:

[0100] S601, training an initial mask autoencoder model based on the mask image and the historical fundus image to obtain a first intermediate mask autoencoder model.

[0101] Exemplarily, assuming that 75% of the image blocks of each of the first preset number of images are randomly masked to obtain a masked image, the initial masked autoencoder model can be trained based on the masked image and the historical fundus image to obtain a first intermediate masked autoencoder model. Pixel loss is used as the loss function of the training so that the image generated by the model based on the remaining 25% of the image blocks that are not masked is as close as possible to the original historical fundus image.

[0102] S602: Train the first intermediate mask autoencoder model based on the annotated image and the historical fundus image to obtain a second intermediate mask autoencoder model.

[0103] In one embodiment, the first intermediate mask autoencoder model can be trained based on the annotated images and the historical fundus images to obtain the second intermediate mask autoencoder model: the annotated images can be used to fine-tune a ResNet-50 model pre-trained on ImageNet as a teacher model for semi-supervised knowledge distillation. Then the historical fundus image is input into the pre-trained ResNet-50 model for image recognition, and the image recognition result output by the ResNet-50 model is used as the historical fundus image Pseudo-labels ,Right now:

[0104]

[0105] in, is the indicative function, is the Sigmoid function. Then the pseudo label As a signal, a supervised learning classification task is set to train the first intermediate mask autoencoder model to obtain a second intermediate mask autoencoder model.

[0106] S603, determining a feature extraction model based on the first model parameters of the second intermediate mask autoencoder model and the second model parameters of other participants in the federated training.

[0107] The second model parameters are model parameters of the second intermediate mask autoencoder model obtained by other participant devices in the federated training.

[0108] In an embodiment of the present application, there are multiple participants in the federated training, and the above-mentioned process of training the initial mask autoencoder model based on the mask image, the annotated image, and the historical fundus image to obtain the first intermediate mask autoencoder model and the second intermediate mask autoencoder model needs to be performed separately by each participant in the federated training.

[0109] Exemplarily, assuming that there are N participants in the federated training, each participant will obtain a second intermediate masked autoencoder model based on its local data training, that is, a total of N second intermediate masked autoencoder models with different model parameters will be obtained.

[0110] In one embodiment, a participant in the federated training may aggregate the first model parameters of the local second intermediate mask autoencoder model with the second model parameters of other participants in the federated training to obtain target model parameters, and determine the target model parameters as the model parameters of the feature extraction model.

[0111] In the embodiment of the present application, the initial mask autoencoder model is trained based on the mask image and the historical fundus image to obtain the first intermediate mask autoencoder model, the first intermediate mask autoencoder model is trained based on the annotated image and the historical fundus image to obtain the second intermediate mask autoencoder model, and the feature extraction model is determined based on the first model parameters of the second intermediate mask autoencoder model and the second model parameters of other participants in the federated training, so that the characteristics of hypertensive target organ damage can be learned based on the historical fundus image, thereby improving the accuracy of the feature extraction model. In addition, training based on the mask image and the annotated image can further improve the accuracy and robustness of the feature extraction model, facilitate the subsequent diagnosis and risk prediction of hypertensive target organ damage based on the feature image, and improve the accuracy and reliability of the diagnosis and risk prediction of hypertensive target organ damage.

[0112] Referring to Figure 7, Figure 7 is a flow chart of another feature extraction model determination method provided by an embodiment of the present application. This implementation involves a possible implementation method of determining a feature extraction model based on the first model parameters of the second intermediate mask autoencoder model and the second model parameters of other participants in the federated training. Based on the above embodiment, the above S603 includes the following steps:

[0113] S701: Acquire a shared model parameter from a second model parameter.

[0114] The shared model parameters are model parameters of a preset proportion in the second model parameters.

[0115] In one embodiment, a preset ratio can be set according to demand, and then shared model parameters of the preset ratio can be obtained from other participating devices in the federated training.

[0116] Exemplarily, the preset ratio can be set to 15%, and then 15% of the model parameters can be obtained from each of the other participating devices in the federated training, and the obtained model parameters can be jointly determined as shared model parameters.

[0117] S702: Aggregate the shared model parameters and the first model parameters to obtain target model parameters.

[0118] In a possible implementation, the shared model parameters and the first model parameters may be weightedly summed to obtain the target model parameters.

[0119] Alternatively, a weighted sum may be performed on the shared model parameters and the first model parameters to obtain a sum result, and the sum result is used as the model parameter for the next round of training.

[0120] For example, referring to FIG8, FIG8 is a schematic diagram of a process flow of federated training provided in an embodiment of the present application. In each training round, each participant in the federation training Each of them optimizes and updates a round of model parameters locally in parallel, namely:

[0121] For each client in parallel do

[0122]

[0123]

[0124] in, For the Some model parameters obtained by round aggregation, For the In round Model parameters for each participant; For the The model performance of each participant is multiplied by the amount of data used to evaluate the model performance. ; It is a filter that randomly selects parameters that need to be shared and aggregated. The implementation method is: all parties share the random number seed of the same random library, and In the round, the shared parameter layer is the first layer generated by the random library with the seed. Group random non-negative integers; It is the ratio of the number of parameters shared in each round to the total number of parameters, which is set to 15% here.

[0125] S703, determining a feature extraction model according to the target model parameters.

[0126] In one embodiment, the target model parameters may be determined as model parameters of the feature extraction model, and then the model parameters of the initial mask autoencoder model may be updated based on the target model parameters to obtain the feature extraction model.

[0127] In an embodiment of the present application, shared model parameters are obtained from the second model parameters, the shared model parameters and the first model parameters are aggregated to obtain target model parameters, and a feature extraction model is determined based on the target model parameters, so that the feature extraction model can be jointly obtained based on the historical fundus data of devices of different participants, breaking the data silos in the field of medical applications. Under the premise of protecting the data privacy of all parties, the amount of training data is expanded, the reliability and robustness of the feature extraction model are improved, and the subsequent diagnosis of target organ damage and risk prediction of hypertension based on feature images are facilitated, thereby improving the accuracy and reliability of the diagnosis and risk prediction of target organ damage of hypertension.

[0128] The evaluation and recognition model includes a diagnostic model and / or a risk prediction model. Referring to FIG9 , FIG9 is a flow chart of a method for obtaining an evaluation result provided in an embodiment of the present application. This implementation involves a possible implementation method of how to input the characteristic image and the clinical indicator information of the patient to be detected into the evaluation and recognition model to obtain the evaluation result. Based on the above embodiment, the above S203 includes the following steps:

[0129] S901, input the characteristic image and the clinical index information of the patient to be detected into the diagnosis model to obtain a diagnosis result.

[0130] Among them, the diagnostic results are used to characterize the target organ damage of hypertension in the patients to be tested.

[0131] Optionally, the diagnosis model may be a multi-layer perceptron neural network model.

[0132] In the embodiment of the present application, the clinical indicator information can be expressed as an indicator vector Then, the characteristic image and clinical index information of the same patient to be tested are concatenated in the form of vectors to obtain the model input vector , and then input the model into the vector Input to the diagnostic model to obtain the diagnostic results:

[0133]

[0134] Optionally, cross entropy can be used as a loss function in the diagnostic model training process, and Stochastic Gradient Descent (SGD) can be used as an optimizer for model training, so as to realize a hypertension diagnosis model based on clinical indicator information and fundus feature images. At the same time, the model can also fit the patient's PWV value through regression analysis.

[0135] S902, inputting the characteristic image and the clinical indicator information of the patient to be tested into the risk prediction model to obtain a risk prediction result.

[0136] Among them, the risk prediction results are used to characterize the probability of damage to the target organs of hypertension in the patients to be tested.

[0137] Optionally, the diagnostic model may be a neural network model.

[0138] In the embodiment of the present application, the clinical indicator information can be expressed as an indicator vector Indicates that the characteristic image and clinical indicator information of the same patient to be detected are then concatenated in the form of a vector to obtain a model input vector, and then the model input vector is input into the risk prediction model to obtain a risk prediction result.

[0139] For example, the clinical index information of the patient to be tested may include age, gender, BMI, smoking status, diabetes, systolic blood pressure, diastolic blood pressure, TG, LDL-C and HDL-C, which together with the feature image extracted from the fundus image feature constitute 9+1024 covariates, namely , to describe a risk function:

[0140]

[0141] in, is the regression coefficient of the independent variable, is the base risk, which is mathematically defined as , that is, the input is fixed to The risk function when .

[0142] In one embodiment, the hypertension-negative individuals can be divided into a low-risk group and a high-risk group according to the historical risk prediction scores of the training set, and the division threshold can be, for example, the median of the historical risk prediction scores. Then, the survival curves of the two groups of people are drawn using the Kaplan-Meier survival analysis method, and then the risk prediction results are determined based on the risk prediction scores and the survival curves.

[0143] In an embodiment of the present application, the characteristic image and the clinical indicator information of the patient to be tested are input into a diagnostic model to obtain a diagnostic result, and the characteristic image and the clinical indicator information of the patient to be tested are input into a risk prediction model to obtain a risk prediction result. Therefore, the hypertension diagnosis and prediction results of the patient to be tested can be determined based on the fundus image and clinical indicator information of the patient to be tested, without the need to examine multiple organs involved in hypertension target organ damage, thereby simplifying the operational process of hypertension target organ damage assessment and reducing the assessment cost. The assessment based on the feature extraction model and the assessment recognition model also improves the accuracy of hypertension target organ damage assessment.

[0144] Referring to FIG. 10 , FIG. 10 is a flow chart of a method for obtaining risk prediction results provided in an embodiment of the present application. This embodiment involves a possible implementation method of how to input the characteristic image and the clinical indicator information of the patient to be detected into the risk prediction model to obtain the risk prediction result. Based on the above embodiment, the above S902 includes the following steps:

[0145] S1001, input the characteristic image and the clinical indicator information of the patient to be tested into the risk prediction model to obtain a risk prediction score.

[0146] In one embodiment, the feature image and the clinical indicator information of the patient to be tested can be concatenated in the form of a vector to obtain a model input vector, and then the model input vector is input into the risk function of the risk prediction model to obtain a risk prediction score.

[0147] S1002, determining the risk prediction result based on the risk prediction score and the survival curve.

[0148] Among them, the survival curve is determined based on the historical risk prediction score.

[0149] In one embodiment, the model input vector of the study population can be used as a training set, and then the historical risk prediction score is determined based on each model input vector in the training set, and then the median of multiple historical risk prediction scores is determined as the partition threshold. According to the partition threshold, the hypertension-negative individuals are divided into a low-risk group and a high-risk group, and the survival curves of the two groups of people are drawn using the Kaplan-Meier survival analysis method, and then the risk prediction results are determined based on the risk prediction scores and the survival curves. Among them, the survival function calculation method is:

[0150]

[0151] Optionally, a log-rank test can be used to compare the differences in the risk of disease between the low-risk group and the high-risk group, and a time-dependent ROC curve can be used to quantify the model prediction performance at different future times.

[0152] In an embodiment of the present application, the characteristic image and the clinical indicator information of the patient to be tested are input into the risk prediction model to obtain a risk prediction score. Based on the risk prediction score and the survival curve, the risk prediction result is determined, so that the individual's risk of disease in a specific time in the future can be quantified, the accuracy of the risk prediction results can be improved, and more accurate prevention advice can be obtained based on the risk prediction results, in order to reduce the incidence, disability and mortality rates of hypertension.

[0153] Referring to FIG. 11 , FIG. 11 is a flow chart of a method for diagnosing and early warning of hypertension and target organ damage caused by hypertension based on fundus images according to an embodiment of the present application. The method comprises the following steps:

[0154] S1101 , masking a first preset number of images in the historical fundus images to obtain masked images, and annotating a second preset number of images in the historical fundus images to obtain annotated images.

[0155] S1102, training an initial mask autoencoder model based on the mask image and the historical fundus image to obtain a first intermediate mask autoencoder model.

[0156] S1103: Train the first intermediate mask autoencoder model based on the annotated image and the historical fundus image to obtain a second intermediate mask autoencoder model.

[0157] S1104, obtaining shared model parameters from the second model parameters, aggregating the shared model parameters and the first model parameters to obtain target model parameters, and determining a feature extraction model according to the target model parameters.

[0158] S1105, inputting the fundus image of the patient to be detected into a feature extraction model to obtain a feature image corresponding to the fundus image.

[0159] S1106, input the characteristic image and the clinical indicator information of the patient to be detected into the diagnosis model to obtain a diagnosis result.

[0160] S1107, input the characteristic image and the clinical indicator information of the patient to be tested into the risk prediction model to obtain a risk prediction result.

[0161] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0162] Based on the same inventive concept, the embodiment of the present application also provides an image recognition device for fundus images for implementing the image recognition method for fundus images involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more fundus image image recognition devices provided below can refer to the limitations of the image recognition method for fundus images above, and will not be repeated here.

[0163] In one embodiment, as shown in FIG. 12 , FIG. 12 is a structural block diagram of an image recognition device for fundus images provided in an embodiment of the present application, and the device 1200 includes:

[0164] The first determination module 1201 is used to input the fundus image of the patient to be detected into the feature extraction model to obtain a feature image corresponding to the fundus image; the feature extraction model is obtained by federally training the initial mask autoencoder model based on historical fundus images.

[0165] The second determination module 1202 is used to input the characteristic image and the clinical indicator information of the patient to be detected into the evaluation and recognition model to obtain the evaluation result; the evaluation result includes: the damage of the target organs of hypertension, and / or the probability of damage to the target organs of hypertension.

[0166] In one embodiment, the first determining module 1201 includes:

[0167] The first determining unit is used to input the fundus image of the patient to be detected into the embedding layer for preliminary feature extraction to obtain a first feature vector.

[0168] The second determining unit is used to input the first feature vector into the encoder for feature abstraction to obtain a second feature vector of a preset length; the preset length is greater than the length of the first feature vector.

[0169] The third determining unit is used to input the second feature vector into the decoder to obtain a feature image corresponding to the fundus image.

[0170] In one embodiment, the apparatus 1200 further includes:

[0171] The third determining module is used to mask a first preset number of images in the historical fundus images to obtain a mask image.

[0172] The fourth determining module is used to annotate a second preset number of images in the historical fundus images to obtain annotated images.

[0173] The fifth determination module is used to perform federated training on the initial mask autoencoder model based on the mask image, the annotated image, and the historical fundus image to obtain a feature extraction model.

[0174] In one embodiment, the fifth determination module includes:

[0175] The fourth determining unit is used to train the initial mask autoencoder model based on the mask image and the historical fundus image to obtain a first intermediate mask autoencoder model.

[0176] The fifth determining unit is used to train the first intermediate mask autoencoder model based on the annotated image and the historical fundus image to obtain a second intermediate mask autoencoder model.

[0177] The sixth determination unit is used to determine the feature extraction model based on the first model parameters of the second intermediate mask autoencoder model and the second model parameters of other participants in the federated training; the second model parameters are the model parameters of the second intermediate mask autoencoder model obtained by the other participant devices in the federated training.

[0178] In one embodiment, the sixth determining unit includes:

[0179] The acquisition subunit is used to acquire the shared model parameters from the second model parameters; the shared model parameters are model parameters of a preset proportion in the second model parameters.

[0180] The first determining subunit is used to aggregate the shared model parameters and the first model parameters to obtain target model parameters.

[0181] The second determining subunit is used to determine the feature extraction model according to the target model parameters.

[0182] In one embodiment, the second determining module 1202 includes:

[0183] The sixth determination module is used to input the characteristic image and the clinical indicator information of the patient to be tested into the diagnosis model to obtain a diagnosis result; the diagnosis result is used to characterize the hypertensive target organ damage of the patient to be tested.

[0184] The seventh determination module is used to input the characteristic image and the clinical indicator information of the patient to be tested into the risk prediction model to obtain the risk prediction result; the risk prediction result is used to characterize the probability of damage to the hypertensive target organs of the patient to be tested.

[0185] In one embodiment, the seventh determination module includes:

[0186] The seventh determination unit is used to input the characteristic image and the clinical indicator information of the patient to be detected into the risk prediction model to obtain a risk prediction score.

[0187] The eighth determination unit is used to determine the risk prediction result based on the risk prediction score and the survival curve; the survival curve is determined based on the historical risk prediction score.

[0188] Each module in the above fundus image recognition device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0189] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0190] Inputting the fundus image of the patient to be tested into the feature extraction model to obtain a feature image corresponding to the fundus image; the feature extraction model is obtained by federally training the initial mask autoencoder model based on the historical fundus image;

[0191] The characteristic image and the clinical indicator information of the patient to be tested are input into the evaluation and recognition model to obtain the evaluation results; the evaluation results include: the damage status of the target organs of hypertension, and / or the probability of damage to the target organs of hypertension.

[0192] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0193] Inputting the fundus image of the patient to be detected into the embedding layer for preliminary feature extraction to obtain a first feature vector;

[0194] Inputting the first feature vector into an encoder for feature abstraction to obtain a second feature vector of a preset length; the preset length is greater than the length of the first feature vector;

[0195] The second feature vector is input to the decoder to obtain a feature image corresponding to the fundus image.

[0196] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0197] Masking a first preset number of images in the historical fundus images to obtain a mask image;

[0198] annotating a second preset number of images in the historical fundus images to obtain annotated images;

[0199] Based on the mask image, the annotated image, and the historical fundus image, the initial mask autoencoder model is federated trained to obtain a feature extraction model.

[0200] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0201] Training an initial mask autoencoder model based on the mask image and the historical fundus image to obtain a first intermediate mask autoencoder model;

[0202] Training the first intermediate mask autoencoder model based on the annotated image and the historical fundus image to obtain a second intermediate mask autoencoder model;

[0203] Based on the first model parameters of the second intermediate masked autoencoder model and the second model parameters of other participants in the federated training, a feature extraction model is determined; the second model parameters are the model parameters of the second intermediate masked autoencoder model obtained in the other participant devices in the federated training.

[0204] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0205] Acquire shared model parameters from the second model parameters; the shared model parameters are model parameters of a preset proportion in the second model parameters;

[0206] Aggregating the shared model parameters and the first model parameters to obtain target model parameters;

[0207] The feature extraction model is determined according to the target model parameters.

[0208] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0209] Inputting the characteristic image and the clinical index information of the patient to be tested into the diagnostic model to obtain a diagnostic result; the diagnostic result is used to characterize the hypertensive target organ damage of the patient to be tested;

[0210] The characteristic image and the clinical indicator information of the patient to be tested are input into the risk prediction model to obtain the risk prediction result; the risk prediction result is used to characterize the probability of damage to the target organs of hypertension in the patient to be tested.

[0211] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0212] Inputting the characteristic image and the clinical indicator information of the patient to be tested into the risk prediction model to obtain a risk prediction score;

[0213] The risk prediction result is determined based on the risk prediction score and the survival curve; the survival curve is determined based on the historical risk prediction score.

[0214] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0215] Inputting the fundus image of the patient to be tested into the feature extraction model to obtain a feature image corresponding to the fundus image; the feature extraction model is obtained by federally training the initial mask autoencoder model based on the historical fundus image;

[0216] The characteristic image and the clinical indicator information of the patient to be tested are input into the evaluation and recognition model to obtain the evaluation results; the evaluation results include: the damage status of the target organs of hypertension, and / or the probability of damage to the target organs of hypertension.

[0217] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0218] Inputting the fundus image of the patient to be detected into the embedding layer for preliminary feature extraction to obtain a first feature vector;

[0219] Inputting the first feature vector into an encoder for feature abstraction to obtain a second feature vector of a preset length; the preset length is greater than the length of the first feature vector;

[0220] The second feature vector is input to the decoder to obtain a feature image corresponding to the fundus image.

[0221] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0222] Masking a first preset number of images in the historical fundus images to obtain a mask image;

[0223] annotating a second preset number of images in the historical fundus images to obtain annotated images;

[0224] Based on the mask image, the annotated image, and the historical fundus image, the initial mask autoencoder model is federated trained to obtain a feature extraction model.

[0225] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0226] Training an initial mask autoencoder model based on the mask image and the historical fundus image to obtain a first intermediate mask autoencoder model;

[0227] Training the first intermediate mask autoencoder model based on the annotated image and the historical fundus image to obtain a second intermediate mask autoencoder model;

[0228] Based on the first model parameters of the second intermediate masked autoencoder model and the second model parameters of other participants in the federated training, a feature extraction model is determined; the second model parameters are the model parameters of the second intermediate masked autoencoder model obtained in the other participant devices in the federated training.

[0229] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0230] Acquire shared model parameters from the second model parameters; the shared model parameters are model parameters of a preset proportion in the second model parameters;

[0231] Aggregating the shared model parameters and the first model parameters to obtain target model parameters;

[0232] The feature extraction model is determined according to the target model parameters.

[0233] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0234] Inputting the characteristic image and the clinical index information of the patient to be tested into the diagnostic model to obtain a diagnostic result; the diagnostic result is used to characterize the hypertensive target organ damage of the patient to be tested;

[0235] The characteristic image and the clinical indicator information of the patient to be tested are input into the risk prediction model to obtain the risk prediction result; the risk prediction result is used to characterize the probability of damage to the target organs of hypertension in the patient to be tested.

[0236] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0237] Inputting the characteristic image and the clinical indicator information of the patient to be tested into the risk prediction model to obtain a risk prediction score;

[0238] The risk prediction result is determined based on the risk prediction score and the survival curve; the survival curve is determined based on the historical risk prediction score.

[0239] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0240] Inputting the fundus image of the patient to be tested into the feature extraction model to obtain a feature image corresponding to the fundus image; the feature extraction model is obtained by federally training the initial mask autoencoder model based on the historical fundus image;

[0241] The characteristic image and the clinical indicator information of the patient to be tested are input into the evaluation and recognition model to obtain the evaluation results; the evaluation results include: the damage status of the target organs of hypertension, and / or the probability of damage to the target organs of hypertension.

[0242] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0243] Inputting the fundus image of the patient to be detected into the embedding layer for preliminary feature extraction to obtain a first feature vector;

[0244] Inputting the first feature vector into an encoder for feature abstraction to obtain a second feature vector of a preset length; the preset length is greater than the length of the first feature vector;

[0245] The second feature vector is input to the decoder to obtain a feature image corresponding to the fundus image.

[0246] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0247] Masking a first preset number of images in the historical fundus images to obtain a mask image;

[0248] annotating a second preset number of images in the historical fundus images to obtain annotated images;

[0249] Based on the mask image, the annotated image, and the historical fundus image, the initial mask autoencoder model is federated trained to obtain a feature extraction model.

[0250] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0251] Training an initial mask autoencoder model based on the mask image and the historical fundus image to obtain a first intermediate mask autoencoder model;

[0252] Training the first intermediate mask autoencoder model based on the annotated image and the historical fundus image to obtain a second intermediate mask autoencoder model;

[0253] Based on the first model parameters of the second intermediate masked autoencoder model and the second model parameters of other participants in the federated training, a feature extraction model is determined; the second model parameters are the model parameters of the second intermediate masked autoencoder model obtained in the other participant devices in the federated training.

[0254] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0255] Acquire shared model parameters from the second model parameters; the shared model parameters are model parameters of a preset proportion in the second model parameters;

[0256] Aggregating the shared model parameters and the first model parameters to obtain target model parameters;

[0257] The feature extraction model is determined according to the target model parameters.

[0258] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0259] Inputting the characteristic image and the clinical index information of the patient to be tested into the diagnostic model to obtain a diagnostic result; the diagnostic result is used to characterize the hypertensive target organ damage of the patient to be tested;

[0260] The characteristic image and the clinical indicator information of the patient to be tested are input into the risk prediction model to obtain the risk prediction result; the risk prediction result is used to characterize the probability of damage to the target organs of hypertension in the patient to be tested.

[0261] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0262] Inputting the characteristic image and the clinical indicator information of the patient to be tested into the risk prediction model to obtain a risk prediction score;

[0263] The risk prediction result is determined based on the risk prediction score and the survival curve; the survival curve is determined based on the historical risk prediction score.

[0264] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0265] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0266] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image recognition method for fundus images, characterized in that: The method comprises: Inputting the fundus image of the patient to be tested into a feature extraction model to obtain a feature image corresponding to the fundus image; the feature extraction model is obtained by federally training an initial mask autoencoder model based on historical fundus images; The characteristic image and the clinical indicator information of the patient to be detected are input into the evaluation and recognition model to obtain an evaluation result; the evaluation result includes: the damage status of the target organs of hypertension, and / or the probability of damage to the target organs of hypertension.

2. The method according to claim 1, characterized in that The feature extraction model includes an embedding layer, an encoder, and a decoder; the fundus image of the patient to be detected is input into the feature extraction model to obtain a feature image corresponding to the fundus image, including: Inputting the fundus image of the patient to be detected into the embedding layer for preliminary feature extraction to obtain a first feature vector; Inputting the first feature vector into the encoder for feature abstraction to obtain a second feature vector of a preset length; the preset length is greater than the length of the first feature vector; The second feature vector is input to the decoder to obtain a feature image corresponding to the fundus image.

3. The method according to claim 2, characterized in that The method further comprises: Masking a first preset number of images in the historical fundus images to obtain mask images; Annotating a second preset number of images in the historical fundus images to obtain annotated images; Based on the mask image, the annotated image, and the historical fundus image, the initial mask autoencoder model is federated trained to obtain the feature extraction model.

4. The method according to claim 3, characterized in that The step of performing federated training on the initial mask autoencoder model based on the mask image, the annotated image, and the historical fundus image to obtain the feature extraction model includes: Training the initial mask autoencoder model based on the mask image and the historical fundus image to obtain a first intermediate mask autoencoder model; Training the first intermediate mask autoencoder model based on the annotated image and the historical fundus image to obtain a second intermediate mask autoencoder model; The feature extraction model is determined based on the first model parameters of the second intermediate mask autoencoder model and the second model parameters of other participants in the federated training; the second model parameters are the model parameters of the second intermediate mask autoencoder model obtained in the other participant devices in the federated training.

5. The method according to claim 4, characterized in that The determining the feature extraction model based on the first model parameters of the second intermediate mask autoencoder model and the second model parameters of other participants in the federated training includes: Acquire shared model parameters from the second model parameters; the shared model parameters are model parameters of a preset proportion in the second model parameters; Aggregating the shared model parameters and the first model parameters to obtain target model parameters; The feature extraction model is determined according to the target model parameters.

6. The method according to any one of claims 1 to 5, characterized in that: The evaluation and recognition model includes a diagnosis model and / or a risk prediction model. The characteristic image and the clinical indicator information of the patient to be detected are input into the evaluation and recognition model to obtain an evaluation result, including: Inputting the characteristic image and the clinical index information of the patient to be detected into the diagnostic model to obtain a diagnostic result; the diagnostic result is used to characterize the hypertensive target organ damage of the patient to be detected; The characteristic image and the clinical indicator information of the patient to be tested are input into the risk prediction model to obtain a risk prediction result; the risk prediction result is used to characterize the probability of damage to the hypertensive target organs of the patient to be tested.

7. The method according to claim 6, characterized in that The step of inputting the characteristic image and the clinical indicator information of the patient to be detected into a risk prediction model to obtain a risk prediction result includes: Inputting the characteristic image and the clinical indicator information of the patient to be detected into a risk prediction model to obtain a risk prediction score; The risk prediction result is determined based on the risk prediction score and the survival curve; the survival curve is determined based on the historical risk prediction score.

8. An image recognition device for fundus images, characterized in that: The device comprises: A first determination module is used to input the fundus image of the patient to be detected into a feature extraction model to obtain a feature image corresponding to the fundus image; the feature extraction model is obtained by federally training an initial mask autoencoder model based on historical fundus images; The second determination module is used to input the characteristic image and the clinical indicator information of the patient to be detected into the evaluation and recognition model to obtain an evaluation result; the evaluation result includes: the damage status of the target organs of hypertension, and / or the probability of damage to the target organs of hypertension.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images