Eye detection method and related device
By extracting representational information from the intermediate layers of the deep learning model through a metacognitive-driven second model, a confidence index is generated, which solves the problem of inaccurate probability score confidence in deep learning models for eye detection and improves the accuracy and interpretability of the detection results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING NORMAL UNIVERSITY
- Filing Date
- 2025-12-31
- Publication Date
- 2026-05-01
AI Technical Summary
Existing deep learning models are susceptible to noisy data and variant samples in eye detection, leading to the generation of false high-confidence indices and reducing the accuracy of detection results.
A metacognitive-driven second model is used to extract the representation information of the intermediate layers of the deep learning model into the global topological structure and density manifold in the representation space, generate a confidence index to quantify the confidence of the type prediction results, and combine it with the type prediction results of the first model to determine the final eye detection results.
By quantifying the offset between samples and distribution, the overconfidence of deep neural networks is avoided, improving the accuracy and robustness of eye detection, and enhancing the interpretability of the model by visualizing feature maps.
Smart Images
Figure CN121963281A_ABST
Abstract
Description
An eye detection method and related device Technical Field
[0001] This application relates to the field of computer technology, and in particular to an eye detection method and related apparatus. Background Technology
[0002] With the development of information technology and the increasing public attention to eye problems, the field of eye testing is undergoing a new generation of transformation. The testing methods are no longer limited to the traditional methods of relying on professionals to observe with their naked eyes and using simple instruments for manual testing.
[0003] Currently, deep learning models, with their feature extraction and pattern recognition capabilities, can extract features from eye images and perform eye detection based on these extracted features. However, current deep learning models use the probability score of the recognition result as a confidence index, which is susceptible to the influence of noisy data, variant samples, and other data, generating false high confidence indices and reducing the accuracy of the detection results.
[0004] Therefore, there is an urgent need for a solution to address the aforementioned technical problems. Summary of the Invention
[0005] In view of the above problems, this application provides an eye detection method and related device to improve the accuracy of eye detection.
[0006] This application discloses the following technical solution: First aspect: This application provides an eye detection method, including: classifying an eye image based on a first model to obtain a type prediction result corresponding to the eye image, and extracting the representation information of the intermediate layer of the first model; extracting the global topological structure and density manifold of the representation information of the intermediate layer in the representation space based on a metacognitive driven second model, and generating a confidence index; the confidence index is used to indicate the confidence level of the type prediction result; and determining the eye detection result corresponding to the eye image based on the type prediction result and the confidence index.
[0007] In one possible implementation, the training steps of the second model include: classifying training samples based on the first model to obtain the representation information of the intermediate layer corresponding to the training samples; extracting the global topological structure and density manifold of the representation information of the intermediate layer corresponding to the training samples based on the initial second model to generate a distribution consistency scalar; and dynamically modulating the intensity distribution of the evidence support space based on the distribution consistency scalar to obtain the trained second model.
[0008] In one possible implementation, the step of dynamically modulating the intensity distribution of the evidence support space based on the distribution consistency scalar to obtain the trained second model includes: mapping the distribution consistency scalar to evidence scaling coefficients based on a family of monotonic response functions; and dynamically modulating the intensity distribution of the evidence support space based on the evidence scaling coefficients to obtain the trained second model.
[0009] In one possible implementation, the training steps of the first model include: preprocessing and classifying multiple eye images to generate an eye image dataset; using a visual processing neural network as the backbone network of the first model, performing end-to-end training on the initial first model for a multi-label classification task of the eye image dataset to generate the trained first model; the eye image dataset includes normal eye images, first type of abnormal eye images, and second type of abnormal eye images.
[0010] In one possible implementation, the preprocessing and classification of multiple eye images to generate an eye image dataset includes: preprocessing and classifying multiple eye images to obtain normal eye images, first-type abnormal eye images, and second-type abnormal eye images; the first type includes multiple subtypes; based on the number of abnormal eye images of the multiple subtypes, performing data augmentation processing on the abnormal eye images of the multiple subtypes to obtain processed first-type abnormal eye images; and generating an eye image dataset based on the normal eye images, the processed first-type abnormal eye images, and the second-type abnormal eye images.
[0011] In one possible implementation, after determining the eye detection result corresponding to the eye image based on the type prediction result and the confidence index, the method further includes: generating a visual feature map based on the multi-layer representation information of the first model and the confidence index; and displaying the visual feature map on the user interface.
[0012] In one possible implementation, generating a visual feature map based on the multi-layer representation information of the first model and the confidence index includes: generating the gradient of the confidence index with respect to each element in the feature map based on the multi-layer representation information of the first model; determining the global average value of the gradient corresponding to each channel as the weight of the channel for each channel, thereby obtaining the weight of each channel; and performing a weighted summation of the feature maps of all channels based on the weight of each channel to generate a visual feature map.
[0013] Secondly, this application provides an eye detection device, including: a first extraction unit, a second extraction unit, and a determination unit; the first extraction unit is used to classify an eye image based on a first model, obtain a type prediction result corresponding to the eye image, and extract the representation information of the intermediate layer of the first model; the second extraction unit is used to extract the global topological structure and density manifold of the representation information of the intermediate layer in the representation space based on a metacognitive driven second model, and generate a confidence index; the confidence index is used to indicate the confidence level of the type prediction result; the determination unit is used to determine the eye detection result corresponding to the eye image based on the type prediction result and the confidence index.
[0014] Thirdly, this application provides a computer device, which includes a processor and a memory; the memory is used to store program code and transmit the program code to the processor; the processor is used to execute the steps of an eye detection method as described above according to the instructions in the program code.
[0015] Fourth aspect: This application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of an eye detection method as described above.
[0016] Fifth aspect: This application provides a computer program product, which, when run on a computer, executes the steps of an eye detection method as described above.
[0017] Sixth aspect: This application provides a chip including a processor coupled to a memory for executing a computer program or instructions stored in the memory, so that the chip implements the steps of an eye detection method as described above.
[0018] Compared to existing technologies, this application offers the following advantages: This application provides an eye detection method. Based on a first model, eye images are classified to obtain type prediction results corresponding to the eye images, and the representation information of the intermediate layers of the first model is extracted. Based on a metacognitive-driven second model, the global topological structure and density manifold of the intermediate layer's representation information in the representation space are extracted to generate a confidence index. The confidence index indicates the confidence level of the type prediction results. Based on the type prediction results and the confidence index, the eye detection result corresponding to the eye image is determined. In this application, a metacognitive-driven second model is used to extract the global topological structure and density manifold of the intermediate layer's representation information in the representation space to determine the eye detection result. This process quantifies the deviation between the sample and the distribution, avoiding the "overconfidence" of deep neural networks caused by using probability scores as confidence indicators, thus improving the accuracy of the detection results. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 is a schematic diagram of an application scenario of an eye detection method provided in an embodiment of this application; Figure 2 is a flowchart of an eye detection method provided in an embodiment of this application; Figure 3 is a flowchart of a first model training method provided in an embodiment of this application; Figure 4 is a flowchart of a visualization feature map generation method provided in an embodiment of this application; Figure 5 is a schematic diagram of the structure of an eye detection device provided in an embodiment of this application; Figure 6 is a schematic diagram of the hardware structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to be a limitation of this application. As used in the specification and appended claims of this application, the singular expressions "a," "an," "the," "the," "the," and "this" are intended to also include expressions such as "one or more," unless the context clearly indicates otherwise. It should also be understood that in the embodiments of this application, "one or more" refers to one, two, or more; "and / or" describes the relationship between related objects, indicating that three relationships may exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.
[0022] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.
[0023] The "multiple" mentioned in the embodiments of this application refers to two or more. It should be noted that in the description of the embodiments of this application, terms such as "first" and "second" are used only for the purpose of distinguishing descriptions and should not be construed as indicating or implying relative importance, nor should they be construed as indicating or implying order.
[0024] Currently, deep learning models, with their feature extraction and pattern recognition capabilities, can extract features from eye images and perform eye detection based on these extracted features. However, current deep learning models use the probability score of the recognition result as a confidence index, which is susceptible to the influence of noisy data, variant samples, and other data, generating false high confidence indices and reducing the accuracy of the detection results.
[0025] For example, when processing fundus images, current deep learning models may still output a confidence score close to 1 even if the fundus images have quality issues such as uneven illumination and motion blur, leading to detection results that do not match the actual situation. This "overconfidence" stems from insufficient internal calibration of the deep learning model, causing the output confidence score to mismatch with the actual classification result, making it difficult to guarantee the accuracy of the detection results.
[0026] Based on this, this application provides an eye detection method. It classifies eye images based on a first model to obtain type prediction results corresponding to the eye images and extracts representation information from the intermediate layers of the first model. Then, based on a metacognitive-driven second model, it extracts the global topological structure and density manifold of the intermediate layer representation information in the representation space to generate a confidence index. The confidence index indicates the confidence level of the type prediction results. Based on the type prediction results and the confidence index, the eye detection result corresponding to the eye image is determined. This application uses a metacognitive-driven second model to extract the global topological structure and density manifold of the intermediate layer representation information in the representation space to determine the eye detection result. This process quantifies the deviation between the sample and the distribution, avoiding the "overconfidence" of deep neural networks caused by using probability scores as confidence indicators, and improving the accuracy of the detection results.
[0027] It should be noted that the user information and data involved in the embodiments of this application (including but not limited to eye images, eye detection results, and data used for analysis, stored data, and displayed data) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0028] As shown in Figure 1, this figure is a schematic diagram of an application scenario of an eye detection method provided in an embodiment of this application. The eye detection method provided in this embodiment of the application can be realized through the interaction between computer device 101 and server 102.
[0029] Computer device 101 acquires an image of the eye and sends the image to server 102.
[0030] Server 102 receives an eye image, classifies the eye image based on a first model, obtains the type prediction result corresponding to the eye image, and extracts the representation information of the intermediate layer of the first model; based on a metacognitive driven second model, it extracts the global topological structure and density manifold of the representation information of the intermediate layer in the representation space to generate a confidence index; the confidence index is used to indicate the confidence of the type prediction result; based on the type prediction result and the confidence index, it determines the eye detection result corresponding to the eye image.
[0031] In one example, server 102 can send eye detection results corresponding to an eye image to computer device 101. Computer device 101 then displays the eye detection results on a user interface.
[0032] In another example, server 102 may generate a visual feature map based on the multi-layer representation information of the first model and the confidence index; and send the visual feature map to computer device 101. Computer device 101 displays the visual feature map on a user interface.
[0033] It is understood that the deployment method of computer equipment and servers is not specifically limited in the embodiments of this application. For example, computer equipment and servers can be deployed separately or jointly.
[0034] The following describes an eye detection method provided by an embodiment of this application with reference to the accompanying drawings. As shown in Figure 2, this figure is a flowchart of an eye detection method provided by an embodiment of this application, including steps S201-S203.
[0035] S201. Based on the first model, classify the eye image to obtain the type prediction result corresponding to the eye image, and extract the representation information of the intermediate layer of the first model.
[0036] The first model in this application embodiment is used to classify eye images and provide intermediate layer representation information so that the subsequent second model can generate a confidence index based on the intermediate layer representation information, as shown in Figure 3. This figure is a flowchart of a first model training method provided in this application embodiment. The training steps of the first model include steps 11-13.
[0037] Step 11: Preprocess and classify multiple eye images to generate an eye image dataset.
[0038] In this embodiment of the application, multiple eye images are preprocessed and classified to obtain normal eye images, first-type abnormal eye images, and second-type abnormal eye images.
[0039] Taking the Ophthalmic Disease Intelligent Recognition Dataset - 5000 Cases (ODIR-5K), which contains 6498 standardized color eye images, as an example, the eye images in the ODIR-5K dataset can be classified into normal eye images and abnormal eye images. Based on this, according to the eye type corresponding to the abnormal eye images, the abnormal eye images can be further divided into Type I abnormal eye images and Type II abnormal eye images.
[0040] The first type can include more common anomaly types; the second type of anomaly in eye images can include less common anomaly types, which is used to improve the generalization ability of the first model.
[0041] In one example, the first type may include multiple subtypes. If the number of eye images of a subtype is small, it may lead to an imbalance in the number of eye images of different types, thereby reducing the detection effect.
[0042] To balance the number of eye images of multiple subtypes, in this embodiment of the application, data augmentation processing can be performed on the abnormal eye images of the multiple subtypes based on the number of abnormal eye images of the multiple subtypes to obtain processed abnormal eye images of the first type; based on the normal eye images, the processed abnormal eye images of the first type and the abnormal eye images of the second type, an eye image dataset is generated.
[0043] The subtypes may include, but are not limited to, diabetic retinopathy, glaucoma, cataracts, hypertensive retinopathy, pathological myopia, and age-related macular degeneration; the second type may include other abnormal types besides the subtypes, such as pigment epithelial hyperplasia and retinal artery aneurysms.
[0044] In this application embodiment, data enhancement processing may include, but is not limited to, one or more of the following: random horizontal flipping processing, random rotation processing (rotation angle can be -15 degrees to 15 degrees), brightness fine-tuning processing (brightness adjustment range can be 0.8-1.2), and contrast processing (contrast adjustment range can be 0.8-1.2).
[0045] For example, in this embodiment of the application, out of 6498 eye images, 3201 eye images can be used for training, 1067 eye images for validation, and 1068 eye images for testing. Simultaneously, the test set also includes 171 eye images of the second type of anomalies that did not appear in the training set, used to evaluate the generalization ability of the first model.
[0046] In this embodiment of the application, by performing data augmentation processing on the abnormal eye images of the multiple subtypes, the number of eye images of each type can be made similar, increasing the diversity of data, avoiding overfitting, and improving the accuracy of eye detection results.
[0047] It should be noted that the embodiments of this application do not specifically limit the source and number of eye images used to train the model; the above only uses the ODIR-5K dataset as an example.
[0048] Step 12: Construct the backbone network of the first model.
[0049] In this embodiment, a visual processing neural network can be used as the backbone network of the first model.
[0050] It is understood that the type of visual processing neural network in this application embodiment is not specifically limited. For example, it can be based on convolution, transformer, multilayer perceptron mixer (MLP-Mixer) or other arbitrary parameterized architecture.
[0051] In this embodiment, the backbone network of the first model has a multi-layer cascaded structure, which can progressively extract hierarchical visual features from the input eye image, ranging from low-level edge textures to high-level semantic patterns.
[0052] Step 13: Train and test the initial first model to generate the trained first model.
[0053] In this embodiment of the application, for the multi-label classification task of the eye image dataset, the initial first model is trained end-to-end to generate the trained first model.
[0054] For example, for the multi-label classification task on the ODIR-5K dataset, the initial first model is trained in an end-to-end manner, starting with random initialization parameters, and regularization strategies are introduced as needed to prevent overfitting.
[0055] During training, by gradually adjusting the parameters of the feature extraction module, the initial first model can be gradually adapted to the domain characteristics of the eye image, such as color distribution, uneven illumination, and specific representations like the cup-shaped structure of the visual disc, to generate the trained first model.
[0056] The eye image to be detected is input into the trained first model. Based on the first model, the eye image can be classified to obtain the type prediction result corresponding to the eye image, and the representation information of the intermediate layer of the first model can be extracted. After obtaining the representation information of the intermediate layer, S202 can be executed.
[0057] S202. Based on the metacognitive-driven second model, extract the global topological structure and density manifold of the representation information of the intermediate layer in the representation space to generate a confidence index.
[0058] The confidence index is used to indicate the confidence level of the prediction result of the type.
[0059] For example, by integrating the representation information of the intermediate layer, the decision process information of the first model can be obtained. Based on this decision process information, the metacognitive-driven second model can generate a confidence index corresponding to the type prediction result.
[0060] As mentioned earlier, current deep learning models typically assume that the distribution of test data is consistent with that of training data, and tend to output artificially high confidence scores during prediction. When the feature distribution of test data deviates from that of training data, current deep learning models cannot detect this deviation. However, in real-world applications, the data to be detected (the eye image to be detected) is often affected by environmental factors, noise, or other factors, and will deviate from the distribution of the training data, resulting in low detection accuracy for current deep learning models.
[0061] Based on this, the second model is constructed by referring to the metacognitive mechanism of the human brain in the embodiments of this application, which can realize the detection of eye images, provide reliable and quantifiable indicators for the detection results, and improve the accuracy of detection.
[0062] Meanwhile, to improve the model's ability to identify offset situations, this application introduces a global representation distribution awareness strategy. This strategy can perform overall distribution modeling of the network's deep representations across all samples during the training phase, constructing an abstract, high-dimensional distribution form. This distribution form does not rely on local neighborhood statistics, nor is it limited to class conditional assumptions, but rather captures the global topological structure and density manifold of the training data in the representation space.
[0063] In this embodiment of the application, the training steps of the second model include: classifying training samples based on the first model to obtain the representation information of the intermediate layer corresponding to the training samples; extracting the global topological structure and density manifold of the representation information of the intermediate layer corresponding to the training samples based on the initial second model to generate a distribution consistency scalar; and dynamically modulating the intensity distribution of the evidence support space based on the distribution consistency scalar to obtain the trained second model.
[0064] In the process of dynamically modulating the intensity distribution of the evidence support space based on the distribution consistency scalar to obtain the trained second model, the distribution consistency scalar can be mapped to the evidence scaling coefficient based on the monotonic response function family; and the intensity distribution of the evidence support space is dynamically modulated based on the evidence scaling coefficient to obtain the trained second model.
[0065] In one possible implementation, to ensure the inherent consistency between evidence representation and distribution perception, an implicit collaborative constraint mechanism can be employed in this embodiment. This mechanism introduces a distribution regularization term during the optimization of the second model, guiding the network representation to evolve in a direction conducive to global distribution modeling. Simultaneously, abstract data augmentation strategies (such as representation space perturbations and manifold preservation transformations) enhance the second model's immunity to potential distribution variations.
[0066] After obtaining the trained second model, the global topological structure and density manifold of the intermediate layer's representation information in the representation space can be extracted based on the trained second model to generate a confidence index.
[0067] In this process, the metacognitive-driven second model can generate a distribution consistency scalar corresponding to the representation information of the intermediate layer based on a global representation distribution awareness strategy. This distribution consistency scalar can reflect the degree of fit between the representation information of the intermediate layer and the overall manifold of the training data.
[0068] In this embodiment, the process of generating the distribution consistency scalar does not involve explicit thresholds or post-processing rules, but is achieved through continuous, differentiable mapping. Based on this distribution consistency scalar, the second model can dynamically modulate the intensity distribution of the evidence support space.
[0069] For example, the metacognitive-driven second model maps the distribution consistency scalar to evidence scaling coefficients based on a family of monotonic response functions. When the representation information of the intermediate layer is located in the high-density core region of the training distribution, the strength of the evidence is preserved or moderately enhanced; when the representation information of the intermediate layer deviates from the known manifold, the second model can progressively suppress the absolute strength of each category of evidence while redirecting the released evidence capacity to the uncertainty dimension.
[0070] Therefore, the process of generating a confidence index based on this trained second model does not require additional training branches or auxiliary networks, thus maintaining end-to-end differentiability. The generated confidence index is negatively correlated with the distribution shift and can be used for risk stratification in subsequent decision-making.
[0071] S203. Based on the type prediction result and confidence index, determine the eye detection result corresponding to the eye image.
[0072] In this embodiment of the application, the accuracy of the type prediction result can be determined based on the confidence index.
[0073] For example, when the confidence index is greater than or equal to the threshold, it can be determined that the distribution shift is small and the type prediction result is highly accurate. In this case, the type prediction result can be determined as the eye detection result corresponding to the eye image.
[0074] When the confidence index is less than the threshold, it can be determined that the distribution shift is large and the accuracy of the type prediction result is low. In this case, the eye image can be re-acquired for detection, or relevant technical personnel can be prompted to intervene and conduct detection to determine the eye detection result corresponding to the eye image.
[0075] In one possible implementation, after determining the eye detection result corresponding to the eye image, the eye detection result can be displayed on the user interface.
[0076] In another possible implementation, considering that the eye detection result is only the final result, and that minor detail deviations may affect the accuracy of the eye detection result, in order to facilitate the understanding of the model's reasoning process by relevant technical personnel and to control the accuracy of the eye detection result, in this embodiment of the application, after determining the eye detection result corresponding to the eye image based on the type prediction result and the confidence index, a visual feature map can be generated based on the multi-layer representation information of the first model and the confidence index; and the visual feature map can be displayed on the user interface.
[0077] Based on the multi-layer representation information of the first model and the confidence index, the process of generating a visual feature map can be shown in Figure 4. This figure is a flowchart of a visual feature map generation method provided by an embodiment of this application, including steps 31-33.
[0078] Step 31: Gradient backpropagation.
[0079] In this embodiment of the application, the gradient of the confidence index with respect to each element in the feature map is generated based on the multi-layer representation information of the first model.
[0080] For example, in this embodiment of the application, the multi-layer representation of the first model can be integrated and used as input based on a metacognitive-driven detection module to evaluate the confidence level of the second model's decision. The gradient of the final output confidence index with respect to each element in the feature map is determined through gradient backpropagation.
[0081] For example, select a confidence index for a specific category as the target and calculate its gradient relative to the feature map. Where conf represents the confidence index, j and k represent spatial locations, and i represents the channel.
[0082] Step 32: Calculate the channel weights.
[0083] In this embodiment of the application, for each channel of the feature map, the global average value of the gradient corresponding to the channel can be determined as the weight of the channel, thereby obtaining the weight of each channel.
[0084] The weight of each channel can be expressed by equation (1) as follows: (1) In equation (1), w i The weights of channel i are represented by H and W, respectively, which represent the height and width of the feature map. i,j,k This represents the feature value of the feature map at position (j, k) in channel i.
[0085] Step 33: Generate a visual feature map.
[0086] In this embodiment of the application, based on the weight of each channel, the feature maps of all channels are weighted and summed to generate a visual feature map.
[0087] For example, the process of weighted summation of feature maps of all channels based on the weight of each channel can be represented by equation (2) as follows: (2) Wherein, heatmap represents a low-resolution heatmap; A i This represents the feature value of the feature map in channel i.
[0088] After obtaining the low-resolution heatmap, it can be upsampled to expand the low-resolution heatmap to the size of the original input eye image, thus obtaining a visualized feature map.
[0089] The visualized feature map is generated by overlaying this low-resolution heatmap onto the eye image, with red areas representing high-contribution regions. For example, when the type prediction result or eye detection result is the first subtype of the first type, the visualized feature map highlights clusters of bleeding points. Based on this visualized feature map, relevant technicians can determine that "the model believes this area is suspected to be the second subtype of the second type and requires further detection."
[0090] After generating the visual feature map, it can be displayed in the user interface.
[0091] For example, the user interface allows relevant technicians to zoom in, zoom out, and download the visualized feature map.
[0092] If the red areas in the visualized feature map are concentrated in non-important areas (such as image noise), relevant technicians can question the reliability of the model, correct the model parameters, and retrain the model.
[0093] In summary, the metacognitive-driven second model adopted in this application embodiment, referencing the metacognitive mechanism of the human brain, can provide reliable and quantifiable evaluation indicators, effectively control the risk of detection errors, and avoid the overconfidence problem of traditional deep neural networks.
[0094] The second model, driven by metacognition, extracts the representation information of the intermediate layer into the global topological structure and density manifold in the representation space to determine the eye detection results. The process combines a global representation distribution perception strategy and dynamic modulation of the intensity distribution of evidence support space, which reflects the distribution shift of the test samples. When the samples are far from the training distribution, uncertainty estimation is increased and confidence is reduced, which quantifies the shift of the samples and the distribution, improves the robustness of the eye detection results, and is suitable for detecting diverse fundus images in the real world.
[0095] Meanwhile, in this embodiment of the application, by generating and displaying a visual feature map on the user interface, the suspected area that contributes the most to the detection results in the eye image can be intuitively displayed based on the visual feature map, thereby improving the interpretability of the "black box" model, making it easier for technicians to understand the decision-making basis, prioritizing the detection of high-risk areas, and improving detection efficiency and accuracy.
[0096] This application provides an eye detection device. Referring to Figure 5, which is a schematic diagram of the structure of an eye detection device provided in this application, its specific implementation method is consistent with the implementation method and the technical effect achieved in the above method embodiments, and some contents will not be repeated.
[0097] This application provides an eye detection device 5100, including: a first extraction unit 5101, a second extraction unit 5102, and a determination unit 5103; the first extraction unit 5101 is used to classify eye images based on a first model, obtain type prediction results corresponding to the eye images, and extract representation information of the intermediate layer of the first model; the second extraction unit 5102 is used to extract the global topological structure and density manifold of the representation information of the intermediate layer in the representation space based on a metacognitive driven second model, and generate a confidence index; the confidence index is used to indicate the confidence level of the type prediction results; the determination unit 5103 is used to determine the eye detection result corresponding to the eye image based on the type prediction results and the confidence index.
[0098] In one possible implementation, the apparatus further includes a first training unit; the first training unit is configured to: classify training samples based on a first model to obtain representation information of intermediate layers corresponding to the training samples; extract the global topological structure and density manifold of the representation information of intermediate layers corresponding to the training samples based on an initial second model to generate a distribution consistency scalar; and dynamically modulate the intensity distribution of the evidence support space based on the distribution consistency scalar to obtain the trained second model.
[0099] In one possible implementation, the first training unit is specifically used to: map the distribution consistency scalar to the evidence scaling factor based on a family of monotonic response functions; and dynamically modulate the intensity distribution of the evidence support space based on the evidence scaling factor to obtain the trained second model.
[0100] In one possible implementation, the device further includes a second training unit; the second training unit is configured to: preprocess and classify multiple eye images to generate an eye image dataset; use a visual processing neural network as the backbone network of the first model, and perform end-to-end training on the initial first model for a multi-label classification task of the eye image dataset to generate the trained first model; the eye image dataset includes normal eye images, first type of abnormal eye images, and second type of abnormal eye images.
[0101] In one possible implementation, the second training unit is specifically configured to: preprocess and classify multiple eye images to obtain normal eye images, first-type abnormal eye images, and second-type abnormal eye images; the first type includes multiple subtypes; based on the number of abnormal eye images of the multiple subtypes, perform data augmentation processing on the abnormal eye images of the multiple subtypes to obtain processed first-type abnormal eye images; and generate an eye image dataset based on the normal eye images, the processed first-type abnormal eye images, and the second-type abnormal eye images.
[0102] In one possible implementation, the apparatus further includes: a visualization unit; the visualization unit is configured to generate a visualization feature map based on the multi-layer representation information of the first model and the confidence index; and display the visualization feature map on a user interface.
[0103] In one possible implementation, the visualization unit is specifically used to: generate the gradient of the confidence index for each element in the feature map based on the multi-layer representation information of the first model; for each channel of the feature map, determine the global average value of the gradient corresponding to the channel as the weight of the channel, and obtain the weight of each channel; and perform a weighted summation of the feature maps of all channels based on the weight of each channel to generate a visualized feature map.
[0104] In summary, the embodiments of this application employ a metacognitive-driven second model to extract the global topological structure and density manifold of the intermediate layer's representation information in the representation space to determine the eye detection results. This process quantifies the deviation between the sample and the distribution, avoiding the "overconfidence" of the deep neural network caused by using probability scores as a confidence index, and improving the accuracy of the detection results.
[0105] This application provides a computer device, as shown in FIG6, which is a schematic diagram of the hardware structure of a computer device provided in this application embodiment.
[0106] The computer device 600 includes a processor 610 and a memory 611; the memory 611 is used to store program code and transmit the program code to the processor 610; the processor 610 is used to execute the steps of an eye detection method as described above according to the instructions in the program code.
[0107] For example, the processor 610 can classify the eye image based on the first model to obtain the type prediction result corresponding to the eye image, and extract the representation information of the intermediate layer of the first model; based on the metacognitive driven second model, extract the global topological structure and density manifold of the representation information of the intermediate layer in the representation space to generate a confidence index; the confidence index is used to indicate the confidence of the type prediction result; based on the type prediction result and the confidence index, determine the eye detection result corresponding to the eye image.
[0108] This application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of an eye detection method as described above.
[0109] This application provides a computer program product that, when run on a computer, executes the steps of an eye detection method as described above.
[0110] This application provides a chip including a processor coupled to a memory for executing a computer program or instructions stored in the memory, thereby enabling the chip to implement the steps of an eye detection method as described above.
[0111] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0112] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. An eye detection method, characterized in that, include: Based on the first model, the eye image is classified to obtain the type prediction result corresponding to the eye image, and the representation information of the intermediate layer of the first model is extracted. The second model, driven by metacognition, extracts the global topological structure and density manifold of the representation information of the intermediate layer in the representation space to generate a confidence index. The confidence index is used to indicate the confidence of the type prediction result. Based on the type prediction result and the confidence index, the eye detection result corresponding to the eye image is determined.
2. The method according to claim 1, characterized in that, The training steps of the second model include: classifying the training samples based on the first model to obtain the representation information of the intermediate layer corresponding to the training samples; extracting the global topological structure and density manifold of the representation information of the intermediate layer corresponding to the training samples based on the initial second model to generate a distribution consistency scalar; and dynamically modulating the intensity distribution of the evidence support space based on the distribution consistency scalar to obtain the trained second model.
3. The method according to claim 2, characterized in that, The step of dynamically modulating the intensity distribution of the evidence support space based on the distribution consistency scalar to obtain the trained second model includes: mapping the distribution consistency scalar to evidence scaling coefficients based on a family of monotonic response functions; and dynamically modulating the intensity distribution of the evidence support space based on the evidence scaling coefficients to obtain the trained second model.
4. The method according to claim 1, characterized in that, The training steps of the first model include: preprocessing and classifying multiple eye images to generate an eye image dataset; using a visual processing neural network as the backbone network of the first model, performing end-to-end training on the initial first model for a multi-label classification task of the eye image dataset to generate the trained first model; the eye image dataset includes normal eye images, first type of abnormal eye images, and second type of abnormal eye images.
5. The method according to claim 4, characterized in that, The step of preprocessing and classifying multiple eye images to generate an eye image dataset includes: preprocessing and classifying multiple eye images to obtain normal eye images, first-type abnormal eye images, and second-type abnormal eye images; the first type includes multiple subtypes; based on the number of abnormal eye images of the multiple subtypes, performing data augmentation processing on the abnormal eye images of the multiple subtypes to obtain processed first-type abnormal eye images; and generating an eye image dataset based on the normal eye images, the processed first-type abnormal eye images, and the second-type abnormal eye images.
6. The method according to any one of claims 1-5, characterized in that, After determining the eye detection result corresponding to the eye image based on the type prediction result and the confidence index, the method further includes: generating a visual feature map based on the multi-layer representation information of the first model and the confidence index; and displaying the visual feature map on the user interface.
7. The method according to claim 6, characterized in that, The step of generating a visual feature map based on the multi-layer representation information of the first model and the confidence index includes: generating the gradient of the confidence index with respect to each element in the feature map based on the multi-layer representation information of the first model; determining the global average value of the gradient corresponding to each channel as the weight of the channel for each channel, thereby obtaining the weight of each channel; and performing a weighted summation of the feature maps of all channels based on the weight of each channel to generate a visual feature map.
8. An eye detection device, characterized in that, include: The system comprises a first extraction unit, a second extraction unit, and a determination unit; The first extraction unit is used to classify eye images based on the first model, obtain type prediction results corresponding to the eye images, and extract the representation information of the intermediate layer of the first model. The second extraction unit is used to extract the global topological structure and density manifold of the representation information of the intermediate layer in the representation space based on the metacognitive driven second model, and generate a confidence index; the confidence index is used to indicate the confidence level of the type prediction result; The determining unit is used to determine the eye detection result corresponding to the eye image based on the type prediction result and the confidence index.
9. A computer device, characterized in that, The computer device includes a processor and a memory; the memory is used to store program code and transmit the program code to the processor; the processor is used to execute the steps of the eye detection method as described in any one of claims 1-7 according to the instructions in the program code.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of an eye detection method as described in any one of claims 1-7.