Meibomian gland data grading method and device, electronic equipment and storage medium

Abstract

The invention provides a meibomian gland data grading method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining sample meibomian gland data; performing grading marking on the sample meibomian gland data; performing unbalance removal processing on the sample meibomian gland data through preset p < x > < 2 > p < x > HD < ga >; performing feature normalization and image registration processing on the sample meibomian gland data after the unbalance removal processing; training and testing a preset meibomian gland data classification model according to the sample meibomian gland data subjected to feature normalization and image registration processing; performing optimization and deviation elimination processing on the trained meibomian gland data classification model; and inputting to-be-detected meibomian gland data into the meibomian gland data classification model after optimization and deviation elimination processing to obtain a target classification result. And the reliability of meibomian gland data classification can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of meibomian gland data grading, and in particular to a meibomian gland data grading method, device, electronic equipment, and storage medium. Background Art

[0002] People are paying more and more attention to the health of the meibomian glands. Currently, the analysis of meibomian gland data mainly relies on experienced personnel, which leads to low efficiency and human analysis is prone to errors. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a method, device, electronic device, and storage medium for grading meibomian gland data, which can improve the reliability of meibomian gland data grading.

[0004] In a first aspect, an embodiment of the present invention provides a method for grading meibomian gland data, comprising:

[0005] Obtain sample meibomian gland data;

[0006] Performing hierarchical labeling on the sample meibomian gland data;

[0007] Perform imbalance removal processing on the sample meibomian gland data by using the preset pix2pixHD gan;

[0008] Performing feature normalization and image registration processing on the sample meibomian gland data after the imbalance removal processing;

[0009] Training and testing a preset meibomian gland data classification model according to the sample meibomian gland data after feature normalization and image registration processing;

[0010] Optimizing and eliminating biases on the trained meibomian gland data classification model;

[0011] The meibomian gland data to be tested are input into the meibomian gland data classification model after optimization and deviation elimination processing to obtain the target classification result.

[0012] In some embodiments of the present invention, the step of performing hierarchical labeling on the sample meibomian gland data includes:

[0013] Obtaining a ratio of an abnormal area of ​​the meibomian gland to an area of ​​the entire meibomian gland in the sample meibomian gland data, wherein the abnormal area of ​​the meibomian gland represents an area of ​​atrophy and blockage of the meibomian gland;

[0014] The sample classification result of the sample meibomian gland data is determined according to the comparison result between the ratio and the preset threshold value.

[0015] In some embodiments of the present invention, the optimization and deviation elimination processing of the trained meibomian gland data classification model includes:

[0016] According to a preset construction rule, a prediction relationship is constructed by using a prediction value of the sample meibomian gland data by a preset explanation model and an average prediction value of the sample meibomian gland data by the meibomian gland data classification model, wherein the construction rule indicates that the prediction value corresponding to the explanation model is equal to the average prediction value corresponding to the meibomian gland data classification model;

[0017] Obtaining a model output of the explanation model including the selected feature, and obtaining a model output excluding the selected feature, calculating the two output results to obtain a contribution value of the selected feature to the meibomian gland data classification model, wherein the selected feature represents a feature arbitrarily selected from preset specific features corresponding to the sample meibomian gland data;

[0018] Obtaining contribution values ​​of the selected features in different feature combinations through the contribution values ​​and the prediction relationships;

[0019] When the contribution value of the selected feature in different feature combinations does not meet the preset threshold, the features in the sample meibomian gland data are adjusted, and the structure of the meibomian gland data classification model is corrected.

[0020] In some embodiments of the present invention, after the sample meibomian gland data is graded and labeled, the step includes:

[0021] Comparing the quantities of the sample meibomian gland data at different levels, and obtaining the level with the smallest quantity;

[0022] The preset PixelGAN model is trained using the sample meibomian gland data corresponding to the smallest level;

[0023] Obtaining random meibomian gland data, and inputting the random meibomian gland data into the trained PixelGAN model to obtain a newly added meibomian gland data set;

[0024] The newly added meibomian gland data set is combined with the sample meibomian gland data to update the original sample meibomian gland data.

[0025] In a second aspect, an embodiment of the present invention provides a meibomian gland data grading device, comprising:

[0026] An acquisition module, used for acquiring sample meibomian gland data;

[0027] A labeling module, used for labeling the sample meibomian gland data in a graded manner;

[0028] An imbalance processing module, used for performing imbalance removal processing on the sample meibomian gland data through a preset pix2pixHD gan;

[0029] A feature normalization and image registration module, used for performing feature normalization and image registration processing on the sample meibomian gland data after the imbalance removal processing;

[0030] A model training module, used to train and test a preset meibomian gland data classification model based on the sample meibomian gland data after feature normalization and image registration processing;

[0031] An optimization module, used for optimizing and eliminating deviations of the trained meibomian gland data classification model;

[0032] The grading module is used to input the meibomian gland data to be tested into the meibomian gland data classification model after optimization and deviation elimination processing to obtain a target grading result.

[0033] In some embodiments of the present invention, the marking module includes:

[0034] An abnormal area acquisition module is used to obtain the ratio of the abnormal area of ​​the meibomian gland to the entire meibomian gland area in the sample meibomian gland data, wherein the abnormal area of ​​the meibomian gland represents the area of ​​meibomian gland atrophy and blockage;

[0035] The first comparison processing module is used to determine the sample classification result of the sample meibomian gland data according to the comparison result between the ratio and the preset threshold value.

[0036] In some embodiments of the present invention, the optimization module includes:

[0037] A prediction relationship construction module, for constructing a prediction relationship according to a preset construction rule, by using a prediction value of the sample meibomian gland data by a preset explanation model and an average prediction value of the sample meibomian gland data by the meibomian gland data classification model, wherein the construction rule indicates that the prediction value corresponding to the explanation model is equal to the average prediction value corresponding to the meibomian gland data classification model;

[0038] A first contribution value calculation module is used to obtain a model output of the explanation model including the selected feature, and obtain a model output excluding the selected feature, and calculate the two output results to obtain a contribution value of the selected feature to the meibomian gland data classification model, wherein the selected feature represents a feature arbitrarily selected from preset specific features corresponding to the sample meibomian gland data;

[0039] A second contribution value calculation module is used to obtain the contribution value of the selected feature in different feature combinations through the contribution value and the prediction relationship;

[0040] The adjustment module is used to adjust the features in the sample meibomian gland data and correct the structure of the meibomian gland data classification model when the contribution value of the selected features in different feature combinations does not meet the preset threshold.

[0041] In some embodiments of the present invention, the meibomian gland data grading device further comprises:

[0042] A second comparison processing module is used to compare the quantities of the sample meibomian gland data at different levels to obtain the level with the smallest quantity;

[0043] A PixelGAN model training module, used to train a preset PixelGAN model using the sample meibomian gland data corresponding to the smallest level;

[0044] Obtaining random meibomian gland data, and inputting the random meibomian gland data into the trained PixelGAN model to obtain a newly added meibomian gland data set;

[0045] A combining module is used to combine the newly added meibomian gland data set with the sample meibomian gland data to update the original sample meibomian gland data.

[0046] In a third aspect, an embodiment of the present invention provides an electronic device comprising the meibomian gland data grading device as described in the second aspect above.

[0047] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the meibomian gland data grading method as described in the first aspect above.

[0048] The method for grading meibomian gland data according to the embodiment of the present invention has at least the following beneficial effects: obtaining sample meibomian gland data; grading and marking the sample meibomian gland data; performing imbalance removal processing on the sample meibomian gland data through the preset pix2pixHD gan; performing feature normalization and image registration processing on the sample meibomian gland data after the imbalance removal processing; training and testing the preset meibomian gland data classification model based on the sample meibomian gland data after the feature normalization and image registration processing; optimizing and eliminating the deviation of the trained meibomian gland data classification model; inputting the meibomian gland data to be tested into the meibomian gland data classification model after the optimization and deviation elimination processing to obtain the target grading result. Performing imbalance removal processing, feature normalization and image registration processing on the sample meibomian gland data in sequence is helpful for feature selection, improving model transparency and reducing data deviation. Thereby improving the reliability of meibomian gland data grading. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of a method for grading meibomian gland data provided by an embodiment of the present invention;

[0050] Figure 2 It is a structural diagram of a meibomian gland data grading device provided by another embodiment of the present invention. DETAILED DESCRIPTION

[0051] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0052] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., and orientations or positional relationships indicated are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0053] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0054] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0055] The control method of the embodiment of the present invention is further described below based on the accompanying drawings.

[0056] Reference Figure 1 , Figure 1 This is a flow chart of a method for grading meibomian gland data provided by an embodiment of the present invention. The method for grading meibomian gland data includes but is not limited to the following steps:

[0057] Step S110, obtaining sample meibomian gland data;

[0058] Step S120, performing graded labeling on the sample meibomian gland data;

[0059] Step S130, performing imbalance removal processing on the sample meibomian gland data by using the preset pix2pixHD gan;

[0060] Step S140, performing feature normalization and image registration processing on the sample meibomian gland data after the imbalance removal processing;

[0061] Step S150, training and testing a preset meibomian gland data classification model based on the sample meibomian gland data after feature normalization and image registration processing;

[0062] Step S160, optimizing and eliminating deviations of the trained meibomian gland data classification model;

[0063] Step S170, inputting the meibomian gland data to be tested into the meibomian gland data classification model after optimization and deviation elimination processing to obtain a target classification result.

[0064] It should be noted that data collection is first performed: meibomian gland data is collected from different devices, illustratively, the devices include but are not limited to LipiView II Ocular Surface Interferometer (LV II; Tearscience, Inc., Italia), Canon CR-2 AF (Canon; Canon, Japan), Keratograph 5M (K5M; Oculus, Wetzlar, Germany), and multi-source meibomian data sets are used to improve the diversity of data, so that the model that completes the training later can adapt to more scenarios and be more reliable. After that, the sample meibomian gland data is graded and labeled, the grade is used as a label, and the sample meibomian gland data is used as a data set for subsequent training of the meibomian gland data classification model. Then, pixelgan is used to generate and solve the problem of data imbalance. Pixelgan is used to generate different levels of data respectively, so that all graded data reach the highest level. Image contrast, dissimilarity, uniformity, energy, correlation and angular second moment (ASM) features play an important role in medical image classification. The data sets collected from different devices show that there are significant differences in these indicators. Therefore, it is necessary to enhance the image quality by normalizing the features. After that, the preset meibomian gland data classification model is trained and tested by the preprocessed sample meibomian gland data. For the trained model, the reliability of the model can be evaluated by using evaluation parameters such as Accuracy for the training and validation dataset, Recall, and F1 score. Furthermore, the model interpretability evaluation standard is introduced to evaluate the accuracy of the trained meibomian gland data classification model based on SHAP interpretability IOU and XAI indicator. Among them, the XAI indicator is judged by relevant professionals to determine whether the important feature heat map marked by the interpretability technology is correctly marked, that is, whether the marked position is helpful for disease judgment. In addition, through the SHAP interpretability mechanism, the self-attention mechanism is added, the model is optimized through two interpretable algorithms, and a complete heat map generation model is built using the strategy of multiple sub-models. The SHAP value of the sub-model and the visualization result are combined with the segmentation MASK to obtain the output IOU result of the model at the current depth. Among them, the segmentation MASK uses the ROI segmentation algorithm to segment the MASK image, and the segmentation algorithm is not limited to DeepLabV3. Explainability can make the model transparent, making the final result easier to understand. After completing the training of the model, the meibomian gland data to be tested is input into the meibomian gland data classification model after optimization and deviation elimination to obtain the target classification result.

[0065] Furthermore, the method for training the meibomian gland data classification model includes:

[0066] A dynamic deep learning model with penalty mechanism of CNN+Transition layer+meta-q learning is adopted. Under the premise that the indicators are not met, the CNN model adds Transition layer and Qlearning mechanism to continue learning. It mainly includes the following steps:

[0067] (1) Construct a CNN machine learning model with the input data being sample meibomian gland data, including meibomian gland images from multiple sources.

[0068] (2) Identify actions and states: Actions are weight assignments and interpretability indicators. Interpretability indicators include feature importance analysis, SHAP (SHapley Additive exPlanations) values, and LIME (Local InterpretatibleModel inspective Expansions). States are different stages of the interpretability process or the current level of model interpretability, i.e., XAI, IOU.

[0069] (3) Define rewards: Determine rewards that reflect the quality of the model explanation. Use accuracy and sensitivity (Accuracy for the training and validation dataset, Recall, F1 score), understandability (how easy the explanation is to understand for relevant professionals, XAI and IOU).

[0070] (4) Initialize the Q-table: Create a Q-table that associates states and actions with their respective Q values. Each entry in the Q-table represents the expected cumulative reward for taking an action in a specific state.

[0071] (5) Training using Q-learning: The model is trained using Q-learning to enhance its interpretability and accuracy. During training, the model explores different explanation techniques and is rewarded based on the quality of the explanation provided. The Q-values ​​are iteratively updated using the Q-learning update rule, which combines the immediate reward received and the estimated future rewards.

[0072] (6) Exploration and Exploitation: Balance exploration and exploitation during training, encourage the model to try different interpretability techniques while assigning different weights and model parameters (convolution kernels, step sizes, learning rates).

[0073] (7) Model improvement: Once the Q-learning process converges, the learned Q-values ​​can guide the model’s interpretability and high-accuracy prediction (i.e., robustness). Based on the learned Q-values, the model can select the most appropriate interpretability method and model parameter determination for a given instance.

[0074] This application uses the penalty mechanism dynamic deep learning model of CNN+Transition layer+meta-q learn-ing and the classic image classification model, which are machine learning models known for their excellent performance in eye disease detection. The selection process considers multiple evaluation indicators, including accuracy, recall, F1 score, XAl indicator, and 10U score. Using the 10-fold validation method, the penalty mechanism dynamic deep learning model of CNN+Transition layer+meta-q learn-ing has shown excellent performance in all dimensions.

[0075] In another embodiment, the sample meibomian gland data is graded and labeled, including:

[0076] Obtain the ratio of the abnormal area of ​​the meibomian gland to the entire meibomian gland area in the sample meibomian gland data. The abnormal area of ​​the meibomian gland represents the area of ​​meibomian gland atrophy and blockage.

[0077] The sample classification result of the sample meibomian gland data is determined according to the comparison result between the ratio and the preset threshold value.

[0078] It should be noted that in the process of meibomian gland grading and marking, the ratio of the area of ​​meibomian gland atrophy and blockage to the area of ​​the entire meibomian gland is used. Meibomian gland abnormalities were classified as grade 0 Level 1 Level 2 Level 3

[0079] In another embodiment, the trained meibomian gland data classification model is optimized and bias eliminated, including:

[0080] According to the preset construction rules, a prediction relationship is constructed by using the prediction value of the preset explanation model for the sample meibomian gland data and the average prediction value of the meibomian gland data classification model for the sample meibomian gland data, and the construction rule represents that the prediction value corresponding to the explanation model is equal to the average prediction value corresponding to the meibomian gland data classification model;

[0081] Obtaining a model output of the explanation model including the selected feature, and obtaining a model output including the selected feature, calculating the two output results to obtain a contribution value of the selected feature to the meibomian gland data classification model, wherein the selected feature represents a feature arbitrarily selected from preset specific features corresponding to the sample meibomian gland data;

[0082] Through the contribution value and prediction relationship, the contribution value of the selected feature in different feature combinations is obtained;

[0083] When the contribution value of the selected feature in different feature combinations does not meet the preset threshold, the features in the sample meibomian gland data are adjusted, and the structure of the meibomian gland data classification model is corrected.

[0084] It should be noted that the optimization and deviation elimination process is based on interpretable model deviation elimination: the SHAP method is used as the mechanism module for interpretability. This is an interpretable method based on Shapley Values. Shapley Values ​​is a concept in game theory. It measures the contribution in a set of sequences to distribute rewards. The marginal contribution of a member to the entire game decision is reflected by the Shapy value. In the field of deep learning (machine learning), the Shapley value reflects the marginal impact of the introduction of a certain feature on the entire model during the entire model training and prediction process, thereby deriving the feature importance of the feature in the entire model prediction process.

[0085] In this Shap calculation, Shapley Values ​​are fitted as an additive feature attribution form. The model first obtains the predicted value on the data set and records the predicted average as φ. The feature of each sample in the data set can be expressed as x = {f1, f2, ..., f M}, the result of the original prediction value of the model is recorded as f(x), and the output result of the Shap model, that is, the explanation model of the present application, is recorded as g(x). The quantitative relationship between the original prediction value f(x) of the meibomian gland data classification model and the output result g(x) of the explanation model is,

[0086]

[0087] This is to ensure that the newly constructed explanation model is consistent with the prediction behavior of the Meibomian Gland Data Classification Model. In this way, the explanation model can reasonably assign the contribution of each feature to the prediction result.

[0088] Use val(x i ) is its value function used to measure the output of the model. S is a subset of sample x, so for a single feature x i For a sample set S, its feature contribution can be expressed as:

[0089] Δval(x i , S) = val(S∪{f i})-val(S) (1-2)

[0090] In formula 1-2, val(SU{fi}) The only set of samples that can be expressed as S∪f i The output value of the model at the moment of the feature vector, f i is x i The feature of is denoted as Output1, so val(S) represents the model output value when only S is in the sample set, denoted as Output2. The difference between Output1 and Output2 is represented as feature x i Feature contribution on subset S.

[0091] Formula (1-2) describes the impact of a single feature on model prediction. The impact of a feature is evaluated by comparing the prediction difference when the model includes and does not include a selected feature to obtain the contribution value.

[0092] Combining formulas 1-1 and 1-2, the feature contribution of feature ji in the entire sample can be expressed as formula 1-3.

[0093]

[0094] In formula 1-3, |S| is the total number of elements in S, and M is the maximum value in sample x. i It is the Shapley Values ​​of the i-th feature variable.

[0095] In the SHAP method, Formula 1-3 considers the average contribution measure of all possible combinations of features. Because Formula (3-2) is aimed at the impact of a single feature. Formula 1-3 is to ensure that the contribution of each feature is considered fairly and evenly, regardless of the order of the features.

[0096] The SHAP method is generally considered to be the core method in the field of explainable artificial intelligence. The SHAP method has strong versatility and can be used in various common deep learning or machine learning models. The additive feature attribution of SHAP usually satisfies the following properties.

[0097] Property 1: The predicted value of the explanatory model g(x) is always equal to the predicted value of the original model f(x), which is what is expressed in formula 3-1.

[0098] Property 2: Under multiple features, if a feature has no value, it will not affect the output of the explanatory model g(x). When the feature is 0, its Shapley Values ​​will also become 0.

[0099] Property 3: When converting from a complex model to a simple model, if the contribution of the characteristic variable increases, its ShapleyValues ​​will also increase.

[0100] The solvability of the SHAP algorithm and deep learning model was studied to optimize the robustness of the meibomian gland data classification model. Multiple SHAP inspection sub-models were inserted in the middle of the model to check the learning process of the model in the middle layer training and the direction of the learning path. According to the visualization results of SHAP, the structural adjustment of the model was completed.

[0101] In another embodiment, after the sample meibomian gland data is graded and labeled, the following steps are performed:

[0102] Compare the quantity of meibomian gland data of samples at different levels and obtain the level with the smallest quantity;

[0103] The preset PixelGAN model is trained using the sample meibomian gland data corresponding to the smallest level.

[0104] Obtain random meibomian gland data, and input the random meibomian gland data into the trained PixelGAN model to obtain a new meibomian gland dataset;

[0105] The newly added meibomian gland data set is combined with the sample meibomian gland data to update the original sample meibomian gland data.

[0106] It should be noted that data imbalance means that the number of data at different levels in the sample is too different, such as too little meibomian gland data at level 0, and too much meibomian gland data at level 2. Categories with a large number will cause the model to overfit, and categories with a small number will make it difficult for the model to learn effective features. Therefore, we must first compare the number of sample meibomian gland data of various types, and obtain the data categories that need to be enhanced in quantity if the number of samples of which categories is too small. Then, the sample meibomian gland data is used as a training sample for PixelGAN, and PixelGAN is trained to learn how to generate meibomian gland data. After PixelGAN completes training, in addition to the sample meibomian gland data used for training, a meibomian gland data that meets the preset regulations is randomly obtained and input into the trained PixelGAN model to generate images of categories that are insufficient in the original sample data set. Thereby increasing the number of samples in these categories. The generated new meibomian glands are added to the sample meibomian gland data, so that the number of samples in each category in the data set used to train the meibomian gland data classification model reaches a preset threshold number. Exemplarily, there are 612 meibomian gland data at each level. This results in a more balanced data.

[0107] Furthermore, in the step of performing feature normalization and image registration processing on the sample meibomian gland data after the imbalance removal processing, the following steps are included:

[0108] It should be noted that image quality enhancement based on explainable technology (1) Image quality enhancement feature normalization based on explainable artificial intelligence.

[0109] Image features such as contrast, dissimilarity, homogeneity, energy, correlation, and angular second moment (ASM) play an important role in medical image classification. Datasets collected from different devices show significant differences in these metrics. Image feature extraction is achieved by applying the scikits-Image-python package. Feature normalization for image quality enhancement is achieved using the cv-python package and the gray-level co-occurrence matrix (GLCM). However, dataset-based normalization may lose important information, thereby introducing significant bias to image classification models.

[0110] Therefore, this application proposes an image quality enhancement method that uses XAI to determine the normalization range of the above-mentioned image features. The XAI method based on feature importance is used to calculate feature performance indicators based on tree models. Tree-based models, such as decision trees or random forests, provide feature importance scores based on the frequency of the feature used for segmentation and the extent to which it reduces impurities or errors. The importance score is regarded as an indicator of FP and is directly extracted from the trained model. This study uses decision trees to perform FP evaluation on image features. The deviation between images collected from the same device is normalized according to function (2-1) (M i ,i=MGD-1k dataset,Real-world dataset 1,Real-world dataset 2,j=contrast,dissimilarity,homogeneity,energy,correlation,ASM,jaccording to FP j Sorted in descending order), where MGD-1k dataset refers to the dataset of meibomian gland dysfunction, Real-wor ld dataset 1 is real-world dataset 1, and Real-wor ld dataset 2 is real-world dataset 2. XAI results (FP j ) is used to regularize the parameters contrast, dissimilarity, homogeneity, energy, correlation and ASM. According to function (2-2), the deviation between images collected from different devices is normalized, and different images are normalized (M,j=contrast, dissimilarity, homogeneity, energy, correlation, ASM,j according to FP j Sort in descending order), XAI results (FP j) is used to regularize the parameters contrast, dissimilar ity, homogeneity, energy, correlation and ASM. The pseudo code of feature normalization for image quality enhancement based on XAI is shown as Algorithm 2. The greycoprops function is used to calculate the inverse difference moment from the modified GLCM matrix, which is used to reconstruct the image. The X input represents the feature matrix and the y input represents the target variable vector.

[0111] A decision tree classifier (DecisionTreeClassifier) ​​is initialized and fit with the input data. The feature_importances_ attribute of the fitted decision tree classifier provides an array of feature importances. These importance values ​​are then divided by the sum of all importance values ​​to get a relative measure of importance. feature_performance is calculated as the feature importance value expressed as a percentage of the total importance. This measure represents the performance or importance of each feature in the decision process. calculate_feature_performance function, by passing the feature matrix X and the target variable vector y. It will return the feature performance values. Further, the features input represents a list or array of feature names and the feature_performance input represents the corresponding list or array of feature performance indicators. A dictionary named feature_dict will be created associating each feature name with its corresponding performance indicator using the zip function. The zip function combines the features and performance indicators into key-value pairs. The sort function is then used to sort the features based on the performance indicator. The key parameter specifies that the sorting should be based on the performance indicator associated with each feature, while the reverse parameter is set to True to sort in descending order. The sorted_features variable will contain the sorted feature names in descending order based on the performance indicator. The sort_features_by_performance function returns the sorted feature names by passing the feature names and performance metrics. Finally, the reconstructed image is normalized to the range of

[0255] using the cv2.normalize function to obtain the reset image. The reset_texture_features function returns the reset image based on these features by passing the original texture feature values. The experiment verifies the effectiveness of the important feature selection method based on XAI.

[0112] M i =GLCM (FP j ,image matrix) (2-1)

[0113] M=GLCM (FP j ,M i ) (2-2)

[0114] Among them, GLCM (parameter) is the gray-level co-occurrence matrix, and image matrix is ​​the image matrix of the collected image.

[0115] (2) Image quality enhancement based on interpretable AI image registration. Machine learning models can perform image registration by learning to match corresponding features in images, such as corners, edges, or spots. In this study, the feature importance analysis XAI technology of SHapley Additive Explanation (SHAP) was studied to reveal which features are most important for accurate registration and guide the improvement of feature detection or matching algorithms. The Python packages shap and cv2.BFMatche are used for important feature identification and image matching, respectively. The pseudo code for image registration using machine learning models and SHAP (SHapley Additive Explanation) for feature importance analysis is shown in Algorithm 3. The image_registration function performs the entire image registration process. In this study, Harris corner detection, Canny and Hessian determinant (DoH) are used to extract corner, edge, and spot detection algorithm features from the source and target images. The machine learning model is trained to learn the correspondence between the features extracted from the source and target images. In this study, Brute ForceMatcher of OpenCV and matplotlib is used. The source image is registered to the target image using the trained model and perspective transformation. Finally, feature importance analysis is performed using the SHAP library to determine the importance of the features for accurate registration. SHAP calculates feature importance scores based on the trained model and source features. The image_registration function, by passing the source image and the target image. It will return the registered image and the importance score indicating the importance of the feature for accurate registration. Using the returned results, the image registration correction process is implemented based on the feature importance values, i.e., if the SHAP scores of all features of corners, edges, and spots are equal, the image will be output, otherwise the image will be registered again.

[0116] Exemplarily, under the guidance of Explainable Artificial Intelligence (XAI), several preprocessing techniques are applied, including feature normalization of each dataset, deep feature normalization of all collected data, and image registration. For the MGD-1k dataset, the feature normalization parameters are set as follows: FP j =1.46,1.41,1.34,1.34,1.29,1.22,(j=

[0117] Homogeneity, Contrast, Correlation, Dissimilarity, ASM, Energy), where jdenotes the feature (Homogeneity, Contrast, Correlation, Dissimilarity, ASM, Energy). Regarding the feature normalization of the real-world dataset 1, the parameters are determined as FP j =0.58,0.52,0.50,0.48,0.46,0.45,(j=Correlation,Dissimilarity,Energy,Contrast,ASM,Homogeneity),j represents the image features (Correlation,Dissimilarity,Energy,Contrast,ASM,Homogeneity). Similarly, for the feature normalization of the real-world dataset 2, the parameter value is FP j =0.65,0.65,0.65,0.64,0.63,0.54,(j=

[0118] Correlation, Dissimilarity, ASM, Homogeneity, Contrast, Energy), j is the image feature (Correlation, Dissimilarity, ASM, Homogeneity, Contrast, Energy). In the case of data set fusion, deep feature normalization is applied to the normalized image to obtain the following parameter value FP j =1.72,1.64,1.61,1.58,1.52,1.35,(j=Correlation,Dissimilarity,Homogeneity,Contrast,Energy,ASM),where j denotes the feature(Correlation,Dissimilarity,Homogeneity,Contrast,Energy,ASM). In addition, the image registration is evaluated and corrected based on the SHAP score. In this embodiment, the threshold Set to 0.6, where i represents corners, edges, and spots.

[0119] like Figure 2 As shown, Figure 2 : is a structural diagram of a meibomian gland data grading device provided by an embodiment of the present invention. The present invention also provides a meibomian gland data grading device, comprising:

[0120] The processor 201 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0121] The memory 202 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 202 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 202, and the processor 201 calls and executes the method of the embodiments of this application;

[0122] Input / output interface 203, used to implement information input and output;

[0123] The communication interface 204 is used to realize the communication interaction between the present apparatus and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0124] Bus 205 , which transmits information between various components of the device (e.g., processor 201 , memory 202 , input / output interface 203 , and communication interface 204 );

[0125] The processor 201 , the memory 202 , the input / output interface 203 and the communication interface 204 are connected to each other in communication within the device via the bus 205 .

[0126] Furthermore, the meibomian gland data grading device comprises:

[0127] An acquisition module, used for acquiring sample meibomian gland data;

[0128] A labeling module, used for hierarchical labeling of sample meibomian gland data;

[0129] The imbalance processing module is used to perform imbalance removal processing on the sample meibomian gland data through the preset pix2pixHD gan;

[0130] A feature normalization and image registration module is used to perform feature normalization and image registration processing on the sample meibomian gland data after imbalance removal processing;

[0131] A model training module is used to train and test a preset meibomian gland data classification model based on the sample meibomian gland data after feature normalization and image registration processing;

[0132] An optimization module is used to optimize and eliminate bias of the trained meibomian gland data classification model;

[0133] The grading module is used to input the meibomian gland data to be tested into the meibomian gland data classification model after optimization and deviation elimination processing to obtain the target grading result.

[0134] Marking module, including:

[0135] The abnormal area acquisition module is used to obtain the ratio of the abnormal area of ​​the meibomian gland to the entire meibomian gland area in the sample meibomian gland data. The abnormal area of ​​the meibomian gland represents the area of ​​meibomian gland atrophy and blockage;

[0136] The first comparison processing module is used to determine the sample classification result of the sample meibomian gland data according to the comparison result between the ratio and the preset threshold value.

[0137] Optimization modules, including:

[0138] A prediction relationship building module is used to build a prediction relationship based on a preset construction rule, using a preset explanation model's prediction value for sample meibomian gland data and an average prediction value of the meibomian gland data classification model for sample meibomian gland data, and the construction rule represents that the prediction value corresponding to the explanation model is equal to the average prediction value corresponding to the meibomian gland data classification model;

[0139] A first contribution value calculation module is used to obtain a model output of the explanation model including the selected feature, and obtain a model output excluding the selected feature, and calculate the two output results to obtain a contribution value of the selected feature to the meibomian gland data classification model, wherein the selected feature represents a feature arbitrarily selected from preset specific features corresponding to the sample meibomian gland data;

[0140] The second congratulations contribution value calculation module is used to obtain the contribution value of the selected feature in different feature combinations through the contribution value and the prediction relationship;

[0141] The adjustment module is used to adjust the features in the sample meibomian gland data and correct the structure of the meibomian gland data classification model when the contribution value of the selected features in different feature combinations does not meet the preset threshold.

[0142] The meibomian gland data grading device further comprises:

[0143] A second comparison processing module is used to compare the quantity of sample meibomian gland data at different levels to obtain the level with the smallest quantity;

[0144] A PixelGAN model training module, used to train the preset PixelGAN model using the sample meibomian gland data corresponding to the minimum number of levels;

[0145] Obtain random meibomian gland data, and input the random meibomian gland data into the trained PixelGAN model to obtain a new meibomian gland dataset;

[0146] The combining module is used to combine the newly added meibomian gland data set with the sample meibomian gland data to update the original sample meibomian gland data.

[0147] An embodiment of the present application also provides an electronic device, comprising the meibomian gland data grading device as described above.

[0148] An embodiment of the present application further provides a storage medium, which is a computer-readable storage medium. The storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned meibomian gland data grading method is implemented.

[0149] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and are implemented to be located in one place, or may also be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0150] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0151] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions under the shared conditions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A method for grading meibomian gland data, characterized in that: include: Obtain sample meibomian gland data; Performing hierarchical labeling on the sample meibomian gland data; Perform imbalance removal processing on the sample meibomian gland data by using the preset pix2pixHD gan; Performing feature normalization and image registration processing on the sample meibomian gland data after the imbalance removal processing; Training and testing a preset meibomian gland data classification model according to the sample meibomian gland data after feature normalization and image registration processing; Optimizing and eliminating biases on the trained meibomian gland data classification model; The meibomian gland data to be tested are input into the meibomian gland data classification model after optimization and deviation elimination processing to obtain the target classification result.

2. The method for grading meibomian gland data according to claim 1, characterized in that: The step of performing hierarchical labeling on the sample meibomian gland data comprises: Obtaining a ratio of an abnormal area of ​​the meibomian gland to an area of ​​the entire meibomian gland in the sample meibomian gland data, wherein the abnormal area of ​​the meibomian gland represents an area of ​​atrophy and blockage of the meibomian gland; The sample classification result of the sample meibomian gland data is determined according to the comparison result between the ratio and the preset threshold value.

3. The method for grading meibomian gland data according to claim 1, characterized in that: The step of optimizing and eliminating deviations from the trained meibomian gland data classification model includes: According to a preset construction rule, a prediction relationship is constructed by using a prediction value of the sample meibomian gland data by a preset explanation model and an average prediction value of the sample meibomian gland data by the meibomian gland data classification model, wherein the construction rule indicates that the prediction value corresponding to the explanation model is equal to the average prediction value corresponding to the meibomian gland data classification model; Obtaining a model output of the explanation model including the selected feature, and obtaining a model output excluding the selected feature, calculating the two output results to obtain a contribution value of the selected feature to the meibomian gland data classification model, wherein the selected feature represents a feature arbitrarily selected from preset specific features corresponding to the sample meibomian gland data; Obtaining contribution values ​​of the selected features in different feature combinations through the contribution values ​​and the prediction relationships; When the contribution value of the selected feature in different feature combinations does not meet the preset threshold, the features in the sample meibomian gland data are adjusted, and the structure of the meibomian gland data classification model is corrected.

4. The method for grading meibomian gland data according to claim 1, characterized in that: After the sample meibomian gland data is graded and labeled, the method includes: Comparing the quantities of the sample meibomian gland data at different levels, and obtaining the level with the smallest quantity; The preset PixelGAN model is trained using the sample meibomian gland data corresponding to the smallest level; Obtaining random meibomian gland data, and inputting the random meibomian gland data into the trained PixelGAN model to obtain a newly added meibomian gland dataset; The newly added meibomian gland data set is combined with the sample meibomian gland data to update the original sample meibomian gland data.

5. A meibomian gland data grading device, characterized in that: include: An acquisition module, used for acquiring sample meibomian gland data; A labeling module, used for labeling the sample meibomian gland data in a graded manner; An imbalance processing module, used for performing imbalance removal processing on the sample meibomian gland data through a preset pix2pixHD gan; A feature normalization and image registration module, used for performing feature normalization and image registration processing on the sample meibomian gland data after the imbalance removal processing; A model training module, used to train and test a preset meibomian gland data classification model based on the sample meibomian gland data after feature normalization and image registration processing; An optimization module, used for optimizing and eliminating deviations of the trained meibomian gland data classification model; The grading module is used to input the meibomian gland data to be tested into the meibomian gland data classification model after optimization and deviation elimination processing to obtain a target grading result.

6. The meibomian gland data grading device according to claim 5, characterized in that: The marking module comprises: An abnormal area acquisition module is used to obtain the ratio of the abnormal area of ​​the meibomian gland to the entire meibomian gland area in the sample meibomian gland data, wherein the abnormal area of ​​the meibomian gland represents the area of ​​meibomian gland atrophy and blockage; The first comparison processing module is used to determine the sample classification result of the sample meibomian gland data according to the comparison result between the ratio and the preset threshold value.

7. The meibomian gland data grading device according to claim 5, characterized in that: The optimization module comprises: A prediction relationship construction module, for constructing a prediction relationship according to a preset construction rule, by using a prediction value of the sample meibomian gland data by a preset explanation model and an average prediction value of the sample meibomian gland data by the meibomian gland data classification model, wherein the construction rule indicates that the prediction value corresponding to the explanation model is equal to the average prediction value corresponding to the meibomian gland data classification model; A first contribution value calculation module is used to obtain a model output of the explanation model including the selected feature, and obtain a model output excluding the selected feature, and calculate the two output results to obtain a contribution value of the selected feature to the meibomian gland data classification model, wherein the selected feature represents a feature arbitrarily selected from preset specific features corresponding to the sample meibomian gland data; A second contribution value calculation module is used to obtain the contribution value of the selected feature in different feature combinations through the contribution value and the prediction relationship; The adjustment module is used to adjust the features in the sample meibomian gland data and correct the structure of the meibomian gland data classification model when the contribution value of the selected features in different feature combinations does not meet the preset threshold.

8. The meibomian gland data grading device according to claim 5, characterized in that: The meibomian gland data grading device further comprises: A second comparison processing module is used to compare the quantities of the sample meibomian gland data at different levels to obtain the level with the smallest quantity; A PixelGAN model training module, used to train a preset PixelGAN model using the sample meibomian gland data corresponding to the minimum number of levels; Obtaining random meibomian gland data, and inputting the random meibomian gland data into the trained PixelGAN model to obtain a newly added meibomian gland dataset; A combining module is used to combine the newly added meibomian gland data set with the sample meibomian gland data to update the original sample meibomian gland data.

9. An electronic device, characterized in that: The device comprises the meibomian gland data grading device according to any one of claims 5 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the meibomian gland data grading method according to any one of claims 1 to 4.

Images