Auxiliary diagnosis method and system for detecting disease symptoms through human eyes

Abstract

The present invention discloses an auxiliary diagnosis method and system for detecting symptoms through human eyes, and relates to the technical field of medical image processing. An auxiliary diagnosis system for detecting symptoms through human eyes includes: a data acquisition module, a sample optimization module, an eye feature analysis module, a comprehensive symptom analysis module and a comprehensive report module. The present invention combines eye image data with patient vital sign data. The method adopts a multimodal learning model, which can not only accurately detect eye symptoms, but also comprehensively analyze symptoms related to other systemic diseases, and can more comprehensively perform auxiliary diagnosis of diseases; by introducing an adaptive weight adjustment mechanism, it can optimize the weight distribution of the model according to the result association of the two analysis models, and improve the accuracy of symptom classification and detection, especially in the identification of complex or minor symptoms. It has a significant effect; by cyclically optimizing the analysis samples, the analysis results of the samples are continuously improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to an auxiliary diagnosis method and system for detecting symptoms of disease through human eyes. Background Art

[0002] With the rapid development of modern medical technology, medical image processing has become an important tool for diagnosing diseases. Image data analysis plays a key role in the diagnosis process of detecting symptoms. Traditional symptom detection mostly relies on the doctor's naked eye observation and subjective judgment, combined with the patient's clinical signs for diagnosis. However, this method is easily affected by the doctor's experience, fatigue and equipment performance, resulting in inaccurate diagnostic results. Especially in the detection of early symptoms or minor lesions, there is a risk of missed diagnosis or misdiagnosis. In addition, the traditional diagnostic process is time-consuming and requires a high level of technical skills from the operator, and the diagnostic efficiency needs to be improved urgently.

[0003] With the development of artificial intelligence technology, especially deep learning and multimodal data fusion technology, it has become a trend to apply computer image processing to the medical field. By comprehensively analyzing eye image data and patient's vital sign data, it is expected to improve the accuracy of disease detection and diagnostic efficiency. However, most of the current image analysis systems still have problems such as single data processing and insufficient intelligence of analysis models. They fail to make full use of multi-source data to improve the accuracy and reliability of diagnosis; they are unable to remove the influence of specific eye features on image recognition results, and they are unable to adjust data according to actual conditions during the recognition process, resulting in low recognition accuracy. Summary of the invention

[0005] The object of the present invention is to provide an auxiliary diagnosis method and system for detecting symptoms through human eyes, so as to solve the problems raised in the above-mentioned background technology.

[0006] An auxiliary diagnosis method for detecting disease symptoms through human eyes, comprising: S1. Obtain the patient's eye image data, including but not limited to fundus images, anterior segment images, and corneal images; obtain the patient's physical sign data, including but not limited to age, gender, blood pressure, blood sugar, body temperature, heart rate, medical history, patient self-report and doctor's preliminary judgment; the patient's self-report and doctor's preliminary judgment are used for corpus identification of symptoms; perform preliminary preprocessing on the patient's eye image data and physical sign data to obtain initial analysis data; S2. Perform standard image processing on the eye image data in the initial analysis data, and combine them with the vital sign data to construct an initial analysis sample, match a specific eye image processing method in the characteristic disease-symptom association table according to the eye feature analysis results, and use the specific eye image processing method to process the eye image data in the initial analysis data to obtain specific processed image data, reconstruct the vital sign data according to the eye feature analysis results, and combine the specific processed image data with the reconstructed vital sign data to construct an optimized analysis sample; S3. Send the initial analysis sample to the eye feature analysis model for calculation to obtain the eye feature analysis result, which is used to match the specific eye image processing method in step S2. The eye feature analysis model is trained using a multimodal learning method, integrating a deep convolutional neural network for processing image data and a multi-layer perceptron for processing numerical data. By integrating the attention mechanism in the model, the model adaptively changes the weights during the loop optimization process to improve the accuracy of the analysis; S4. Send the optimized analysis sample into the comprehensive disease symptom analysis model for auxiliary diagnosis of the disease symptom, obtain the comprehensive disease symptom analysis result, compare the comprehensive disease symptom analysis result with the eye feature analysis result in the characteristic disease symptom association table, and if there is an association result, use the comprehensive disease symptom analysis result and the eye feature analysis result as the final comprehensive disease symptom analysis result and the final eye feature analysis result; otherwise, match the specific eye image processing method in the characteristic disease symptom association table according to the comprehensive disease symptom analysis result, process the eye image data in the initial analysis data, reconstruct the patient's vital sign data, and update the optimized analysis sample, save the reconstruction and optimization records in the optimization log, send the optimized analysis sample into the eye feature analysis model for calculation, and obtain the optimized eye feature analysis result; Using the optimized eye feature analysis result to update the optimized analysis sample, and using the optimized analysis sample to calculate the optimized comprehensive symptom analysis result, the optimized comprehensive symptom analysis result and the optimized eye feature analysis result are compared in the feature symptom association table, and if there is an association result, the final comprehensive symptom analysis result and the final eye feature analysis result are obtained, otherwise the current step is looped; S5. Obtain an auxiliary diagnosis report of the disease based on the final comprehensive disease symptom analysis results and the final eye feature analysis results.

[0007] As a preferred technical solution of the present invention, the eye feature analysis model is composed of an image data analysis unit, a vital sign data analysis unit, an attention adaptation unit and a feature analysis unit; The image data analysis unit is used to extract the features of the image data. It is composed of a deep convolutional neural network with the output layer removed. The eye image data in the analysis sample enters the unit through the integrated input layer after being sent to the model, and finally outputs the feature map of the image data; The vital sign data analysis unit is used to extract the vital sign data, and is composed of a multi-layer perceptron with the output layer removed. After the vital sign data in the analysis sample is sent to the model, it enters the unit through the integrated input layer, and finally outputs the feature vector of the vital sign data; The attention adaptive unit is used to dynamically adjust the importance of image features and physical features, and fuse multimodal information in an adaptive manner. It consists of 1 hidden layer and 1 fully connected layer. It weights the feature map and feature vector according to the optimization log, and fuses the weighted features to obtain the fusion analysis features. The feature analysis unit is used to perform the final classification based on the fusion analysis features, and consists of 2 hidden layers and 1 fully connected layer; The structure of the comprehensive disease symptom analysis model is the same as that of the eye feature analysis model.

[0008] As a preferred technical solution of the present invention, it also includes constructing an eye feature analysis model, and the specific steps are: A1. Obtain a number of eye image data and corresponding vital sign data, pre-process the eye image data and the corresponding vital sign data, form an analysis sample with the eye image data and the corresponding vital sign data, add eye feature labels to the analysis samples, and form an eye feature analysis data set, and divide the eye feature analysis data set into a training set, a validation set, and a test set; A2. Obtain a pre-trained model, which is trained using eye image data, extract eye image data parts and labels of some samples in the eye feature analysis data set, and add new similar data to form a pre-training data set, and use the pre-training data set to optimize the pre-trained model to obtain an optimized pre-trained model dedicated to eye features, and use it as the initial image data analysis unit of the eye feature analysis model; Use the same method to train an initial vital sign data analysis unit; A3. Construct an initial eye feature analysis model, which consists of an integrated input layer, a parallel initial image data analysis unit and an initial vital sign data analysis unit, three hidden layers and two fully connected layers; send the eye feature analysis training set and validation set to the initial eye feature analysis model for training, and use the eye feature label as the target to obtain an optimized eye feature analysis model; A4. Send the eye feature analysis data test set to the optimized eye feature analysis for accuracy evaluation. Taking the eye feature label as the target, adjust the hyperparameters of the eye feature analysis model to obtain the final eye feature analysis model. The comprehensive disease symptom analysis model is trained in the same way, the difference is that the classification targets and data sets of the training models are different. Specifically, the classification targets of the comprehensive disease symptom analysis model include but are not limited to eye symptoms, and the data set overlaps with the eye feature analysis data set.

[0009] As a preferred technical solution of the present invention, it also includes an attention adaptation unit; After obtaining the feature map and feature vector, the attention adaptive unit fuses the two. The fusion method is multiplication combined with splicing. First, the feature map and the vital sign feature are multiplied element by element to generate interactive features. The original image features, vital sign features and interactive features after multiplication fusion are spliced ​​together to obtain fusion analysis features. The fusion analysis features are given adaptive weights by optimizing the log content. Through back propagation, the decisive features of the current classification label are obtained in the fully connected layer of the feature analysis unit, which are called guiding features, and the remaining features are called non-guiding features. For the initial analysis samples, the attention adaptive unit does not perform any operation. After obtaining the optimized content of the optimization log, the weight of the guiding feature of each classification is reduced, and the weight of the non-guiding feature is increased. The specific formula for increasing the weight is: θ q =L2(θ p A+B); where L2() represents the normalization function, θ p and θ q They represent the original weight and the updated weight respectively, and A and B are the weight adjustment parameters.

[0010] As a preferred technical solution of the present invention, it also includes optimizing logs; The optimization log stores the optimization records of the analyzed samples, including the image processing methods, changes in vital sign data, and records of comparison errors. The records in the optimization log are used for weight adjustment and generation of auxiliary diagnosis reports.

[0011] As a preferred technical solution of the present invention, it also includes a characteristic disease symptom association table; The characteristic symptom association table records the association between the eye characteristic analysis results and the comprehensive symptom analysis results, as well as the specific image processing method corresponding to the analysis results.

[0012] As a preferred technical solution of the present invention, it also includes image processing; For the initial image data, standard image processing methods are used to process the image, including image standardization and image graying; for the optimized image, it is necessary to process it according to the matched specific image processing method, which includes image standardization and corresponding processing methods for removing interference from specific eye feature results.

[0013] As a preferred technical solution of the present invention, it also includes an auxiliary diagnosis report; The auxiliary diagnosis report is compiled from the final comprehensive disease symptom analysis results, the final eye feature analysis results, and the optimization log. If the analysis sample optimization process is less than 3 times, it means that there is little interference in the analysis process and the result is accurate, and it is directly output as a high-credibility auxiliary diagnosis report. If the optimization process is greater than or equal to 3 times, it means that there is a lot of interference in the analysis process and the result is not credible. The output auxiliary diagnosis report includes the final comprehensive disease symptom analysis results, the final eye feature analysis results, and the category of error analysis in the optimization log.

[0014] An auxiliary diagnosis system based on human eye detection of disease symptoms, comprising: A data acquisition module is used to obtain the patient's eye image data, including but not limited to fundus images, anterior segment images, and corneal images, and obtain the patient's physical sign data, including but not limited to age, gender, blood pressure, blood sugar, body temperature, heart rate, medical history, patient self-report and doctor's preliminary judgment. The patient's self-report and doctor's preliminary judgment are used for corpus identification of symptoms, and the patient's eye image data and physical sign data are preliminarily preprocessed to obtain initial analysis data; A sample optimization module is used to perform standard image processing on the eye image data in the initial analysis data, and combine it with the vital sign data to construct an initial analysis sample, match a specific eye image processing method in the characteristic disease-symptom association table according to the eye feature analysis result, and use the specific eye image processing method to process the eye image data in the initial analysis data to obtain specific processed image data, reconstruct the vital sign data according to the eye feature analysis result, and combine the specific processed image data with the reconstructed vital sign data to construct an optimized analysis sample; The eye feature analysis module is used to send the initial analysis sample into the eye feature analysis model for calculation to obtain the eye feature analysis result, which is used to match the specific eye image processing method. The eye feature analysis model is trained using a multimodal learning method and integrates a convolutional neural network for processing image data and a fully connected neural network for processing numerical data. By integrating the attention mechanism in the network, the model adaptively changes the weights during the loop optimization process to improve the accuracy of the analysis; The comprehensive symptom analysis module is used to send the optimized analysis samples into the comprehensive symptom analysis model for auxiliary diagnosis of the symptom, obtain the comprehensive symptom analysis results, compare the comprehensive symptom analysis results with the eye feature analysis results in the feature symptom association table, and if there is an association result, use the comprehensive symptom analysis results and the eye feature analysis results as the final comprehensive symptom analysis results and the final eye feature analysis results; otherwise, match the specific eye image processing method in the feature symptom association table according to the comprehensive symptom analysis results, process the eye image data in the initial analysis data, reconstruct the patient's vital sign data, and update the optimized analysis samples, save the reconstruction and optimization records in the optimization log, send the optimized analysis samples into the eye feature analysis model for calculation, and obtain the optimized eye feature analysis results; Using the optimized eye feature analysis result to update the optimized analysis sample, and using the optimized analysis sample to calculate the optimized comprehensive symptom analysis result, the optimized comprehensive symptom analysis result and the optimized eye feature analysis result are compared in the feature symptom association table, and if there is an association result, the final comprehensive symptom analysis result and the final eye feature analysis result are obtained, otherwise the current step is looped; The comprehensive report module is used to obtain an auxiliary diagnosis report of the disease according to the final comprehensive disease symptom analysis results and the final eye feature analysis results.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention combines eye image data with patient vital sign data. The method adopts a multimodal learning model, which can not only accurately detect eye symptoms, but also conduct a comprehensive analysis of symptoms related to other systemic diseases, and can more comprehensively perform auxiliary diagnosis of diseases and reduce the risks of misdiagnosis and missed diagnosis; by introducing an adaptive weight adjustment mechanism, the weight distribution of the model can be optimized according to the result association of the two analysis models. In the process of cyclic optimization, the model will continuously adjust the weight of the features, and gradually improve the accuracy of symptom classification and detection, especially in the identification of complex or subtle symptoms. The analysis samples are analyzed by cyclic optimization and the analysis results of the samples are continuously improved; the relationship between the classification results and the features is obtained by back propagation, and the weights of the corresponding results are adaptively adjusted to improve the accuracy of model classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Schematic diagram of the structure of an auxiliary diagnosis system for detecting symptoms of disease through human eyes in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] An auxiliary diagnosis method for detecting disease symptoms through human eyes, comprising: S1. Obtain the patient's eye image data, including but not limited to fundus images, anterior segment images, and corneal images; obtain the patient's physical sign data, including but not limited to age, gender, blood pressure, blood sugar, body temperature, heart rate, medical history, patient self-report and doctor's preliminary judgment; the patient's self-report and doctor's preliminary judgment are used for corpus identification of symptoms; perform preliminary preprocessing on the patient's eye image data and physical sign data to obtain initial analysis data; Corpus identification is matched through a corpus matching model, in which the physician initially determines that the data is not necessary, and the result of corpus identification is only the sub-data of the physical sign data; S2. Perform standard image processing on the eye image data in the initial analysis data, and combine them with the vital sign data to construct an initial analysis sample, match a specific eye image processing method in the characteristic disease-symptom association table according to the eye feature analysis results, and use the specific eye image processing method to process the eye image data in the initial analysis data to obtain specific processed image data, reconstruct the vital sign data according to the eye feature analysis results, and combine the specific processed image data with the reconstructed vital sign data to construct an optimized analysis sample; For the initial image data, the image is processed using a standard image processing method, including image standardization and image graying; for the optimized image, it is necessary to process it according to the matched specific image processing method, which includes image standardization and a corresponding processing method for removing interference from the results of specific eye features; for example, for patients with eye congestion, the R value in the RBG value of the image is reduced, and for patients with eye spots, the eye part in the image is removed; The eye feature analysis model in step S2 is composed of an image data analysis unit, a vital sign data analysis unit, an attention adaptation unit and a feature analysis unit; The image data analysis unit is used to extract the features of the image data. It is composed of a deep convolutional neural network with the output layer removed. The eye image data in the analysis sample enters the unit through the integrated input layer after being sent to the model, and finally outputs the feature map of the image data; The vital sign data analysis unit is used to extract the vital sign data, and is composed of a multi-layer perceptron with the output layer removed. After the vital sign data in the analysis sample is sent to the model, it enters the unit through the integrated input layer, and finally outputs the feature vector of the vital sign data; The attention adaptive unit is used to dynamically adjust the importance of image features and physical features, and fuse multimodal information in an adaptive manner. It consists of 1 hidden layer and 1 fully connected layer. It weights the feature map and feature vector according to the optimization log, and fuses the weighted features to obtain the fusion analysis features. After obtaining the feature map and feature vector, the attention adaptive unit fuses the two. The fusion method is multiplication combined with splicing. First, the feature map and the vital sign feature are multiplied element by element to generate interactive features. The original image features, vital sign features and interactive features after multiplication fusion are spliced ​​together to obtain fusion analysis features. The fusion analysis features are given adaptive weights by optimizing the log content. Through back propagation, the decisive features of the current classification label are obtained in the fully connected layer of the feature analysis unit, which are called guiding features, and the remaining features are called non-guiding features. For the initial analysis samples, the attention adaptive unit does not perform any operation. After obtaining the optimized content of the optimization log, the weight of the guiding feature of each classification is reduced, and the weight of the non-guiding feature is increased. The specific formula for increasing the weight is: θ q =L2(θ p A+B); where L2() represents the normalization function, θ p and θ q Represent the original weight and the updated weight respectively, A and B are the weight adjustment parameters; The feature analysis unit is used to perform the final classification based on the fusion analysis features, and consists of 2 hidden layers and 1 fully connected layer; The structure of the comprehensive disease symptom analysis model is the same as that of the eye feature analysis model.

[0019] Construct an eye feature analysis model. The specific steps are as follows: A1. Obtain a number of eye image data and corresponding vital sign data, pre-process the eye image data and the corresponding vital sign data, form an analysis sample with the eye image data and the corresponding vital sign data, add eye feature labels to the analysis samples, and form an eye feature analysis data set, and divide the eye feature analysis data set into a training set, a validation set, and a test set; A2. Obtain a pre-trained model, which is trained using eye image data, extract eye image data parts and labels of some samples in the eye feature analysis data set, and add new similar data to form a pre-training data set, and use the pre-training data set to optimize the pre-trained model to obtain an optimized pre-trained model dedicated to eye features, and use it as the initial image data analysis unit of the eye feature analysis model; Use the same method to train an initial vital sign data analysis unit; A3. Construct an initial eye feature analysis model, which consists of an integrated input layer, a parallel initial image data analysis unit and an initial vital sign data analysis unit, three hidden layers and two fully connected layers; send the eye feature analysis training set and validation set to the initial eye feature analysis model for training, and use the eye feature label as the target to obtain an optimized eye feature analysis model; A4. Send the eye feature analysis data test set to the optimized eye feature analysis for accuracy evaluation. Taking the eye feature label as the target, adjust the hyperparameters of the eye feature analysis model to obtain the final eye feature analysis model. The comprehensive symptom analysis model is trained in the same way, except that the classification targets and data sets of the training models are different. Specifically, the classification targets of the comprehensive symptom analysis model include but are not limited to eye symptoms, and the data set overlaps with the eye feature analysis data set. The analysis samples are cyclically optimized through two similar models. The two models have different classification capabilities for different classification results. The results are associated, and the analysis samples are optimized based on the associated results. The classification accuracy is improved at the input level. The attention mechanism is used within the model to optimize the weights based on the associated comparison results, which improves the classification accuracy within the model.

[0020] S3. Send the initial analysis sample to the eye feature analysis model for calculation to obtain the eye feature analysis result, which is used to match the specific eye image processing method in step S2. The eye feature analysis model is trained using a multimodal learning method, integrating a deep convolutional neural network for processing image data and a multi-layer perceptron for processing numerical data. By integrating the attention mechanism in the model, the model adaptively changes the weights during the loop optimization process to improve the accuracy of the analysis; S4. Send the optimized analysis sample into the comprehensive disease symptom analysis model for auxiliary diagnosis of the disease symptom, obtain the comprehensive disease symptom analysis result, compare the comprehensive disease symptom analysis result with the eye feature analysis result in the characteristic disease symptom association table, and if there is an association result, use the comprehensive disease symptom analysis result and the eye feature analysis result as the final comprehensive disease symptom analysis result and the final eye feature analysis result; otherwise, match the specific eye image processing method in the characteristic disease symptom association table according to the comprehensive disease symptom analysis result, process the eye image data in the initial analysis data, reconstruct the patient's vital sign data, and update the optimized analysis sample, save the reconstruction and optimization records in the optimization log, send the optimized analysis sample into the eye feature analysis model for calculation, and obtain the optimized eye feature analysis result; The optimization log stores the optimization records of the analyzed samples, including the image processing methods, changes in vital sign data, and records of comparison errors. The records in the optimization log are used for weight adjustment and the generation of auxiliary diagnosis reports. The characteristic symptom association table records the association between the eye characteristic analysis results and the comprehensive symptom analysis results, as well as the specific image processing method corresponding to the analysis results; the association is updated through repeated error records, and the repeated error records are marked in the optimization log and added to the association table after being judged by professionals.

[0021] Using the optimized eye feature analysis result to update the optimized analysis sample, and using the optimized analysis sample to calculate the optimized comprehensive symptom analysis result, the optimized comprehensive symptom analysis result and the optimized eye feature analysis result are compared in the feature symptom association table, and if there is an association result, the final comprehensive symptom analysis result and the final eye feature analysis result are obtained, otherwise the current step is looped; S5. Obtain an auxiliary diagnosis report of the disease based on the final comprehensive disease symptom analysis results and the final eye feature analysis results.

[0022] The auxiliary diagnosis report is compiled from the final comprehensive disease symptom analysis results, the final eye feature analysis results, and the optimization log. If the analysis sample optimization process is less than 3 times, it means that there is little interference in the analysis process and the result is accurate, and it is directly output as a high-credibility auxiliary diagnosis report. If the optimization process is greater than or equal to 3 times, it means that there is a lot of interference in the analysis process and the result is not credible. The output auxiliary diagnosis report includes the final comprehensive disease symptom analysis results, the final eye feature analysis results, and the category of error analysis in the optimization log.

[0023] See also Figure 1 The present invention provides an auxiliary diagnosis system for detecting symptoms through human eyes, comprising: A data acquisition module is used to obtain the patient's eye image data, including but not limited to fundus images, anterior segment images, and corneal images, and obtain the patient's physical sign data, including but not limited to age, gender, blood pressure, blood sugar, body temperature, heart rate, medical history, patient self-report and doctor's preliminary judgment. The patient's self-report and doctor's preliminary judgment are used for corpus identification of symptoms, and the patient's eye image data and physical sign data are preliminarily preprocessed to obtain initial analysis data; A sample optimization module is used to perform standard image processing on the eye image data in the initial analysis data, and combine it with the vital sign data to construct an initial analysis sample, match a specific eye image processing method in the characteristic disease-symptom association table according to the eye feature analysis result, and use the specific eye image processing method to process the eye image data in the initial analysis data to obtain specific processed image data, reconstruct the vital sign data according to the eye feature analysis result, and combine the specific processed image data with the reconstructed vital sign data to construct an optimized analysis sample; The eye feature analysis module is used to send the initial analysis sample into the eye feature analysis model for calculation to obtain the eye feature analysis result, which is used to match the specific eye image processing method. The eye feature analysis model is trained using a multimodal learning method and integrates a convolutional neural network for processing image data and a fully connected neural network for processing numerical data. By integrating the attention mechanism in the network, the model adaptively changes the weights during the loop optimization process to improve the accuracy of the analysis; The comprehensive symptom analysis module is used to send the optimized analysis samples into the comprehensive symptom analysis model for auxiliary diagnosis of the symptom, obtain the comprehensive symptom analysis results, compare the comprehensive symptom analysis results with the eye feature analysis results in the feature symptom association table, and if there is an association result, use the comprehensive symptom analysis results and the eye feature analysis results as the final comprehensive symptom analysis results and the final eye feature analysis results; otherwise, match the specific eye image processing method in the feature symptom association table according to the comprehensive symptom analysis results, process the eye image data in the initial analysis data, reconstruct the patient's vital sign data, and update the optimized analysis samples, save the reconstruction and optimization records in the optimization log, send the optimized analysis samples into the eye feature analysis model for calculation, and obtain the optimized eye feature analysis results; Using the optimized eye feature analysis result to update the optimized analysis sample, and using the optimized analysis sample to calculate the optimized comprehensive symptom analysis result, the optimized comprehensive symptom analysis result and the optimized eye feature analysis result are compared in the feature symptom association table, and if there is an association result, the final comprehensive symptom analysis result and the final eye feature analysis result are obtained, otherwise the current step is looped; The comprehensive report module is used to obtain an auxiliary diagnosis report of the disease according to the final comprehensive disease symptom analysis results and the final eye feature analysis results.

Claims

1. An auxiliary diagnosis method for detecting symptoms of disease through human eyes, characterized in that: include: S1. Obtain the patient's eye image data, including but not limited to fundus images, anterior segment images, and corneal images; obtain the patient's physical sign data, including but not limited to age, gender, blood pressure, blood sugar, body temperature, heart rate, medical history, patient self-report and doctor's preliminary judgment; the patient's self-report and doctor's preliminary judgment are used for corpus identification of symptoms; perform preliminary preprocessing on the patient's eye image data and physical sign data to obtain initial analysis data; S2. Perform standard image processing on the eye image data in the initial analysis data, and combine them with the vital sign data to construct an initial analysis sample, match a specific eye image processing method in the characteristic disease-symptom association table according to the eye feature analysis results, and use the specific eye image processing method to process the eye image data in the initial analysis data to obtain specific processed image data, reconstruct the vital sign data according to the eye feature analysis results, and combine the specific processed image data with the reconstructed vital sign data to construct an optimized analysis sample; S3. Send the initial analysis sample to the eye feature analysis model for calculation to obtain the eye feature analysis result, which is used to match the specific eye image processing method in step S2. The eye feature analysis model is trained using a multimodal learning method, integrating a deep convolutional neural network for processing image data and a multi-layer perceptron for processing numerical data. By integrating the attention mechanism in the model, the model adaptively changes the weights during the loop optimization process to improve the accuracy of the analysis; S4. sending the optimized analysis samples to the comprehensive disease symptom analysis model for auxiliary diagnosis of the disease symptom, obtaining the comprehensive disease symptom analysis results, comparing the comprehensive disease symptom analysis results with the eye feature analysis results in the feature disease symptom association table, and if there is an association result, taking the comprehensive disease symptom analysis results and the eye feature analysis results as the final comprehensive disease symptom analysis results and the final eye feature analysis results; Otherwise, a specific eye image processing method is matched in the characteristic symptom association table according to the comprehensive symptom analysis result, and the eye image data in the initial analysis data is processed, the patient's vital sign data is reconstructed again, and the optimized analysis sample is updated, and the reconstruction and optimization records are saved in the optimization log, and the optimized analysis sample is sent to the eye feature analysis model for calculation to obtain the optimized eye feature analysis result; Using the optimized eye feature analysis result to update the optimized analysis sample, and using the optimized analysis sample to calculate the optimized comprehensive symptom analysis result, the optimized comprehensive symptom analysis result and the optimized eye feature analysis result are compared in the feature symptom association table, and if there is an association result, the final comprehensive symptom analysis result and the final eye feature analysis result are obtained, otherwise the current step is looped; S5. Obtain an auxiliary diagnosis report of the disease based on the final comprehensive disease symptom analysis results and the final eye feature analysis results.

2. An auxiliary diagnosis method for detecting symptoms through human eyes according to claim 1, characterized in that: The eye feature analysis model consists of an image data analysis unit, a vital sign data analysis unit, an attention adaptation unit, and a feature analysis unit; The image data analysis unit is used to extract the features of the image data. It is composed of a deep convolutional neural network with the output layer removed. The eye image data in the analysis sample enters the unit through the integrated input layer after being sent to the model, and finally outputs the feature map of the image data; The vital sign data analysis unit is used to extract the vital sign data, and is composed of a multi-layer perceptron with the output layer removed. After the vital sign data in the analysis sample is sent to the model, it enters the unit through the integrated input layer, and finally outputs the feature vector of the vital sign data; The attention adaptive unit is used to dynamically adjust the importance of image features and physical features, and fuse multimodal information in an adaptive manner. It consists of 1 hidden layer and 1 fully connected layer. It weights the feature map and feature vector according to the optimization log, and fuses the weighted features to obtain the fusion analysis features. The feature analysis unit is used to perform the final classification based on the fusion analysis features, and consists of 2 hidden layers and 1 fully connected layer; The structure of the comprehensive disease symptom analysis model is the same as that of the eye feature analysis model.

3. An auxiliary diagnosis method for detecting symptoms through human eyes according to claim 2, characterized in that: It also includes building an eye feature analysis model, the specific steps are: A1. Obtain a number of eye image data and corresponding vital sign data, pre-process the eye image data and the corresponding vital sign data, form an analysis sample with the eye image data and the corresponding vital sign data, add eye feature labels to the analysis samples, and form an eye feature analysis data set, and divide the eye feature analysis data set into a training set, a validation set, and a test set; A2. Obtain a pre-trained model, which is trained using eye image data, extract eye image data parts and labels of some samples in the eye feature analysis data set, and add new similar data to form a pre-training data set, and use the pre-training data set to optimize the pre-trained model to obtain an optimized pre-trained model dedicated to eye features, and use it as the initial image data analysis unit of the eye feature analysis model; Use the same method to train an initial vital sign data analysis unit; A3. Construct an initial eye feature analysis model, which consists of an integrated input layer, a parallel initial image data analysis unit and an initial vital sign data analysis unit, three hidden layers and two fully connected layers; send the eye feature analysis training set and validation set to the initial eye feature analysis model for training, and use the eye feature label as the target to obtain an optimized eye feature analysis model; A4. Send the eye feature analysis data test set to the optimized eye feature analysis for accuracy evaluation. Taking the eye feature label as the target, adjust the hyperparameters of the eye feature analysis model to obtain the final eye feature analysis model. The comprehensive disease symptom analysis model is trained in the same way, the difference is that the classification targets and data sets of the training models are different. Specifically, the classification targets of the comprehensive disease symptom analysis model include but are not limited to eye symptoms, and the data set overlaps with the eye feature analysis data set.

4. An auxiliary diagnosis method for detecting symptoms through human eyes according to claim 2, characterized in that: Also included is an attention adaptation unit; After obtaining the feature map and feature vector, the attention adaptive unit fuses the two by multiplication and splicing. First, the feature map and the vital sign feature are multiplied element by element to generate interactive features. The original image features, vital sign features and the interactive features after multiplication fusion are spliced ​​together to obtain fusion analysis features. The fusion analysis features are given adaptive weights by optimizing the log content. Through back propagation, the decisive features of the current classification label are obtained in the fully connected layer of the feature analysis unit, which are called guiding features. The remaining features are called non-guiding features. For the initial analysis samples, no attention adaptation unit is performed. After obtaining the optimization content of the optimization log, the weight of the guiding feature of each classification is reduced, and the weight of the non-guiding feature is increased. The specific formula for increasing the weight is: θ q =L2(θ p A+B); where L2() represents the normalization function, θ p and θ q They represent the original weight and the updated weight respectively, and A and B are the weight adjustment parameters.

5. An auxiliary diagnosis method for detecting symptoms of disease through human eyes according to claim 1, characterized in that: Also includes optimization logs; The optimization log stores the optimization records of the analyzed samples, including the image processing methods, changes in vital sign data, and records of comparison errors. The records in the optimization log are used for weight adjustment and generation of auxiliary diagnosis reports.

6. An auxiliary diagnosis method for detecting symptoms through human eyes according to claim 1, characterized in that: It also includes a characteristic disease-symptom association table; The characteristic symptom association table records the association between the eye characteristic analysis results and the comprehensive symptom analysis results, as well as the specific image processing method corresponding to the analysis results.

7. An auxiliary diagnosis method for detecting symptoms of disease through human eyes according to claim 1, characterized in that: It also includes image processing; For the initial image data, standard image processing methods are used to process the image, including image standardization and image graying; for the optimized image, it is necessary to process it according to the matched specific image processing method, which includes image standardization and corresponding processing methods for removing interference from specific eye feature results.

8. An auxiliary diagnosis method for detecting symptoms of disease through human eyes according to claim 1, characterized in that: Also included is an auxiliary diagnostic report; The auxiliary diagnosis report is compiled from the final comprehensive disease symptom analysis results, the final eye feature analysis results, and the optimization log. If the analysis sample optimization process is less than 3 times, it means that there is little interference in the analysis process and the result is accurate, and it is directly output as a high-credibility auxiliary diagnosis report. If the optimization process is greater than or equal to 3 times, it means that there is a lot of interference in the analysis process and the result is not credible. The output auxiliary diagnosis report includes the final comprehensive disease symptom analysis results, the final eye feature analysis results, and the category of error analysis in the optimization log.

9. An auxiliary diagnosis system based on detecting disease symptoms by human eyes, characterized in that: The system uses an auxiliary diagnosis method for detecting symptoms of disease through human eyes as described in any one of claims 1 to 8, including: A data acquisition module is used to obtain the patient's eye image data, including but not limited to fundus images, anterior segment images, and corneal images, and obtain the patient's physical sign data, including but not limited to age, gender, blood pressure, blood sugar, body temperature, heart rate, medical history, patient self-report and doctor's preliminary judgment. The patient's self-report and doctor's preliminary judgment are used for corpus identification of symptoms, and the patient's eye image data and physical sign data are preliminarily preprocessed to obtain initial analysis data; A sample optimization module is used to perform standard image processing on the eye image data in the initial analysis data, and combine it with the vital sign data to construct an initial analysis sample, match a specific eye image processing method in the characteristic disease-symptom association table according to the eye feature analysis result, and use the specific eye image processing method to process the eye image data in the initial analysis data to obtain specific processed image data, reconstruct the vital sign data according to the eye feature analysis result, and combine the specific processed image data with the reconstructed vital sign data to construct an optimized analysis sample; The eye feature analysis module is used to send the initial analysis sample into the eye feature analysis model for calculation to obtain the eye feature analysis result, which is used to match the specific eye image processing method. The eye feature analysis model is trained using a multimodal learning method and integrates a convolutional neural network for processing image data and a fully connected neural network for processing numerical data. By integrating the attention mechanism in the network, the model adaptively changes the weights during the loop optimization process to improve the accuracy of the analysis; The comprehensive symptom analysis module is used to send the optimized analysis samples into the comprehensive symptom analysis model for auxiliary diagnosis of the symptom, obtain the comprehensive symptom analysis results, compare the comprehensive symptom analysis results with the eye feature analysis results in the feature symptom association table, and if there is an association result, use the comprehensive symptom analysis results and the eye feature analysis results as the final comprehensive symptom analysis results and the final eye feature analysis results; otherwise, match the specific eye image processing method in the feature symptom association table according to the comprehensive symptom analysis results, process the eye image data in the initial analysis data, reconstruct the patient's vital sign data, and update the optimized analysis samples, save the reconstruction and optimization records in the optimization log, send the optimized analysis samples into the eye feature analysis model for calculation, and obtain the optimized eye feature analysis results; Using the optimized eye feature analysis result to update the optimized analysis sample, and using the optimized analysis sample to calculate the optimized comprehensive symptom analysis result, the optimized comprehensive symptom analysis result and the optimized eye feature analysis result are compared in the feature symptom association table, and if there is an association result, the final comprehensive symptom analysis result and the final eye feature analysis result are obtained, otherwise the current step is looped; The comprehensive report module is used to obtain an auxiliary diagnosis report of the disease according to the final comprehensive disease symptom analysis results and the final eye feature analysis results.

Images