Intelligent full-automatic diagnosis and analysis method and device for anterior segment diseases, electronic equipment and storage medium

Abstract

The invention provides an intelligent full-automatic diagnosis and analysis method and device for anterior segment diseases, electronic equipment and a storage medium, and the method comprises the steps: obtaining an eye image of a patient; inputting the eye image of the patient into a target detection grading model, identifying a focus area of the eye image of the patient based on the target detection grading model, and determining at least one predicted disease type and a severity level of the predicted disease type based on the focus area of the eye image of the patient; and generating an auxiliary diagnosis report based on the predicted disease type and the severity level of the predicted disease type. The method can assist a doctor in positioning a disease area, a focus type and a severity level, so that the diagnosis efficiency and accuracy are improved.

Description

Technical Field

[0001] This application relates to the field of auxiliary diagnosis of ophthalmic diseases, and more specifically, to an intelligent fully automated diagnostic analysis method, device, electronic device, and storage medium for anterior segment diseases. Background Technology

[0002] Currently, the diagnosis of preganglionic eye diseases in humans primarily relies on manual examination and judgment by physicians. This process typically involves specialists observing and analyzing the lesion area using medical imaging techniques such as slit-lamp scanning and optical coherence tomography (OCT). Because this method is highly dependent on the physician's experience, the diagnostic results may be influenced by the physician's subjective judgment, posing a significant risk of misdiagnosis or missed diagnosis. Furthermore, with the increasing number of ophthalmology patients each year, the workload of physicians is becoming increasingly heavy, and the efficiency of manual examination is insufficient to meet the demands of efficient diagnosis. Summary of the Invention

[0003] The purpose of this application is to provide an intelligent fully automated diagnostic analysis method, device, electronic device, and storage medium for anterior segment diseases, to assist doctors in locating the disease area, lesion type, and severity level, thereby improving diagnostic efficiency and accuracy.

[0004] In a first aspect, the present invention provides an intelligent fully automated diagnostic analysis method for anterior segment diseases, the method comprising:

[0005] Acquire images of the patient's eyes;

[0006] The patient's eye image is input into a target detection grading model, the lesion region of the patient's eye image is identified based on the target detection grading model, and at least one predicted disease type and the severity level of the predicted disease type are determined based on the lesion region of the patient's eye image.

[0007] An auxiliary diagnostic report is generated based on the predicted symptom type and the severity level of the predicted symptom type.

[0008] The method of the first aspect of this application acquires an image of a patient's eye, then inputs the image into a target detection and grading model. Based on the target detection and grading model, it identifies lesion regions in the patient's eye image and determines at least one predicted disease type and a severity level based on the lesion regions. This allows for the generation of an auxiliary diagnostic report based on the predicted disease type and its severity level. Compared with existing technologies, this method, on the one hand, assists in locating lesion regions and provides doctors with the predicted disease type's severity level, predicted disease type, and auxiliary diagnostic information through the auxiliary diagnostic report, thereby improving the doctor's diagnostic efficiency. On the other hand, it can fully extract diagnosis-related image information through computer image processing, thus assisting doctors in improving diagnostic accuracy and reducing the probability of doctors missing relevant information under conditions such as poor image quality or patient diversity.

[0009] In an optional implementation, after acquiring the patient's eye image and before inputting the patient's eye image into the target detection grading model, the method further includes:

[0010] The patient's eye image is preprocessed to meet the image input conditions of the target detection grading model. The preprocessing operations include size unification, brightness adjustment, contrast adjustment, and noise removal.

[0011] This optional implementation can perform operations such as size unification, brightness adjustment, contrast adjustment, and noise removal on the patient's eye image to make the patient's eye image meet the image input conditions of the target detection grading model, thereby enabling the model to output more accurate data based on the standardized input image.

[0012] In an optional implementation, the method further includes, before acquiring the patient's eye image:

[0013] Obtain the raw image data;

[0014] The original image data is annotated to determine the lesion area annotation information, lesion type annotation information, and disease type severity level annotation information of the original image data, and feature image data is obtained;

[0015] The feature image data is divided into a training set, a validation set, and a test set;

[0016] Train a deep learning model based on the training set;

[0017] When the deep learning model is completed, the deep learning model is validated based on the validation set, and the parameters of the deep learning model are adjusted and the deep learning model is retrained based on the validation results until the deep learning model meets the preset conditions.

[0018] When the deep learning model meets the preset conditions, the deep learning model is tested based on the test set, and the deep learning model that passes the test is determined as the target detection classification model.

[0019] This optional implementation acquires raw image data and annotates it to determine the lesion region annotation information, lesion type annotation information, and disease severity level annotation information of the raw image data, thereby obtaining feature image data. Furthermore, by dividing the feature image data into a training set, a validation set, and a test set, a deep learning model can be trained based on the training set. When the deep learning model is completed, it is validated based on the validation set, and the parameters of the deep learning model are adjusted and the deep learning model is retrained based on the validation results until the deep learning model meets preset conditions. When the deep learning model meets the preset conditions, it is tested based on the test set, and the deep learning model that passes the test is determined as the target detection grading model.

[0020] In an optional implementation, the target detection hierarchical model includes a backbone network, a feature fusion layer, and a detection head;

[0021] And, based on the target detection grading model, identify the lesion region of the patient's eye image, and determine at least one predicted disease type and the severity level of the predicted disease type based on the lesion region of the patient's eye image, including:

[0022] Based on the backbone network, multi-scale image features of the patient's eye image are extracted, and a feature map is obtained;

[0023] The feature map is fused based on the feature fusion layer to obtain the feature fusion result;

[0024] The feature fusion results are analyzed based on the detection head to determine the lesion area, predicted disease type, and severity level of the predicted disease type in the patient's eye image.

[0025] This optional implementation extracts multi-scale image features from the patient's eye image based on the backbone network, obtaining a feature map. The feature map is then fused using the feature fusion layer to obtain a feature fusion result. This result can then be analyzed using the detection head to determine the lesion region, predicted disease type, and severity level of the predicted disease type in the patient's eye image. The use of multi-scale features improves the accuracy of the model's output. In this optional implementation, the feature fusion result obtained by fusing the feature map using the feature fusion layer includes:

[0026] The number of tokens is determined based on the height and width of the feature map, and the feature dimension is determined based on the number of channels in the feature map.

[0027] The query matrix, key matrix, and value matrix are determined based on the number of tokens and the feature dimensions.

[0028] Attention information is determined based on the query matrix, the key matrix, and the value matrix;

[0029] The feature map is fused based on the attention information to obtain the feature fusion result.

[0030] This optional implementation can determine the number of tokens based on the height and width of the feature map, and determine the feature dimension based on the number of channels of the feature map. Then, it can determine the query matrix, key matrix, and value matrix based on the number of tokens and the feature dimension. Thus, it can determine attention information based on the query matrix, the key matrix, and the value matrix. Finally, it can fuse the feature map based on the attention information to obtain the feature fusion result. In this case, feature fusion based on the attention mechanism can further improve the accuracy of the model.

[0031] In an optional implementation, the detection head includes a classification branch and a regression branch, the regression branch being used to locate the lesion region in the patient's eye image, and the classification branch being used to determine the predicted disease type and the severity level of the predicted disease type.

[0032] This optional implementation can locate the lesion area in the patient's eye image through regression branching, and on the other hand, can predict the disease type and the severity level of the predicted disease type through classification branching.

[0033] In an optional implementation, the classification branch comprises two depthwise separable convolutions.

[0034] In this optional implementation, the classification branch includes two depthwise separable convolutions. By using two depthwise separable convolutions, the amount of computation and parameters can be reduced, thereby improving the overall efficiency.

[0035] In an optional implementation, the loss function of the target detection grading model is the EIoU loss function.

[0036] This optional implementation can use the EIoU loss function as the loss function of the object detection grading model. The EIoU loss function takes into account the overlap area between the predicted box and the ground truth box, the aspect ratio, and the center point offset, thereby improving the overall prediction accuracy.

[0037] In an optional implementation, the backbone network includes C3k2 modules.

[0038] This optional implementation, through the C3k2 module, can reduce computational overhead and avoid parameter redundancy.

[0039] Secondly, the present invention provides an intelligent diagnostic and analysis device for anterior segment diseases, the device comprising:

[0040] The acquisition module is used to acquire images of the patient's eyes;

[0041] The identification module is used to input the patient's eye image into the target detection grading model, identify the lesion area of ​​the patient's eye image based on the target detection grading model, and determine at least one predicted disease type and the severity level of the predicted disease type based on the lesion area of ​​the patient's eye image.

[0042] The generation module is used to generate an auxiliary diagnostic report based on the predicted symptom type and the severity level of the predicted symptom type.

[0043] The apparatus of the second aspect of this application, by executing an intelligent fully automated diagnostic analysis method for anterior segment diseases, can acquire images of a patient's eye, and then input these images into a target detection and grading model. Based on the target detection and grading model, it identifies lesion regions in the patient's eye images and determines at least one predicted disease type and a severity level based on the lesion regions. This allows it to generate an auxiliary diagnostic report based on the predicted disease type and its severity level. Compared with existing technologies, this apparatus, on the one hand, can assist in locating lesion regions and then provide doctors with the severity level, predicted disease type, and auxiliary diagnostic information through the auxiliary diagnostic report, thereby improving the doctor's diagnostic efficiency. On the other hand, it can fully extract diagnostically relevant image information through computer image processing, thereby assisting doctors in improving diagnostic accuracy and reducing the probability of doctors missing relevant information under conditions such as poor image quality or patient diversity.

[0044] Thirdly, the present invention provides an electronic device, comprising:

[0045] Processor; and

[0046] The memory is configured to store machine-readable instructions that, when executed by the processor, perform the intelligent fully automated diagnostic analysis method for anterior segment diseases as described in any of the foregoing embodiments.

[0047] The electronic device of the third aspect of this application, by executing an intelligent fully automated diagnostic analysis method for anterior segment diseases, can acquire images of a patient's eye, and then input the patient's eye images into a target detection and grading model. Based on the target detection and grading model, it identifies lesion regions in the patient's eye images and determines at least one predicted disease type and a severity level of the predicted disease type based on the lesion regions in the patient's eye images. It can then generate an auxiliary diagnostic report based on the predicted disease type and its severity level. Compared with existing technologies, it can, on the one hand, assist in locating disease areas and then provide doctors with the severity level, predicted disease type, and auxiliary diagnostic information of the predicted disease type through the auxiliary diagnostic report, thereby improving the doctor's diagnostic efficiency. On the other hand, it can fully extract image information related to diagnosis through computer image processing, thereby assisting doctors in improving diagnostic accuracy and reducing the probability of doctors missing relevant information under conditions such as poor image quality or patient diversity.

[0048] Fourthly, the present invention provides a storage medium storing a computer program, the computer program being executed by a processor as described in any of the foregoing embodiments of the intelligent fully automated diagnostic analysis method for anterior segment diseases.

[0049] The storage medium of the fourth aspect of this application, by executing an intelligent fully automated diagnostic analysis method for anterior segment diseases, can acquire patient eye images, and then input the patient eye images into a target detection and grading model. Based on the target detection and grading model, lesion regions in the patient eye images are identified, and at least one predicted disease type and its severity level are determined based on the lesion regions. This allows for the generation of an auxiliary diagnostic report based on the predicted disease type and its severity level. Compared with existing technologies, this method, on the one hand, can assist in locating disease regions, and then provide doctors with the predicted disease type's severity level, predicted disease type, and auxiliary diagnostic information through the auxiliary diagnostic report, thereby improving the doctor's diagnostic efficiency. On the other hand, it can fully extract diagnostically relevant image information through computer image processing, thereby assisting doctors in improving diagnostic accuracy and reducing the probability of doctors missing relevant information under conditions such as poor image quality or patient diversity. Attached Figure Description

[0050] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1This is a flowchart illustrating an intelligent fully automated diagnostic analysis method for anterior segment diseases disclosed in an embodiment of this application;

[0052] Figure 2 This is a schematic diagram of the structure of an intelligent fully automated diagnostic and analysis device for anterior segment diseases disclosed in an embodiment of this application;

[0053] Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. Detailed Implementation

[0054] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0055] Currently, the diagnosis of preganglionic eye diseases in humans primarily relies on manual examination and judgment by physicians. This process typically involves specialists observing and analyzing the lesion area using medical imaging techniques such as slit-lamp scanning and optical coherence tomography (OCT). Because this method is highly dependent on the physician's experience, the diagnostic results may be influenced by the physician's subjective judgment, posing a significant risk of misdiagnosis or missed diagnosis. Furthermore, with the increasing number of ophthalmology patients each year, the workload of physicians is becoming increasingly heavy, and the efficiency of manual examination is insufficient to meet the demands of efficient diagnosis.

[0056] To address the shortcomings of existing technologies, this application provides an intelligent fully automated diagnostic analysis method, device, electronic device, and storage medium for anterior segment diseases. It acquires images of a patient's eye, inputs these images into a target detection and grading model, identifies lesion regions in the patient's eye images based on the model, and determines at least one predicted disease type and its severity level based on these lesion regions. This allows for the generation of an auxiliary diagnostic report based on the predicted disease type and its severity level. Compared to existing technologies, this approach assists in locating lesion regions and provides doctors with the predicted disease type's severity level, severity level, and auxiliary diagnostic information through the auxiliary diagnostic report, thereby improving diagnostic efficiency. Furthermore, it leverages computer image processing to fully extract diagnostically relevant image information, thus assisting doctors in improving diagnostic accuracy and reducing the probability of overlooking relevant information due to poor image quality or patient diversity.

[0057] Please see Figure 1 , Figure 1 This is a flowchart illustrating an intelligent fully automated diagnostic analysis method for anterior segment diseases disclosed in an embodiment of this application. Figure 1 As shown, the method in this application embodiment includes the following steps:

[0058] 101. Obtain images of the patient's eyes;

[0059] 102. Input the patient's eye image into the target detection grading model, identify the lesion area of ​​the patient's eye image based on the target detection grading model, and determine at least one predicted disease type and the severity level of the predicted disease type based on the lesion area of ​​the patient's eye image.

[0060] 103. Generate auxiliary diagnostic reports based on the predicted disease type and the severity level of the predicted disease type.

[0061] The method in this application acquires an image of a patient's eye, then inputs the image into a target detection and grading model. Based on this model, lesion regions in the patient's eye image are identified, and at least one predicted symptom type and its severity level are determined. This allows for the generation of an auxiliary diagnostic report based on the predicted symptom type and its severity level. Compared to existing technologies, this method, on the one hand, assists in locating symptom regions and provides doctors with the predicted symptom severity level, symptom type, and auxiliary diagnostic information through the auxiliary diagnostic report, thereby improving doctors' diagnostic efficiency. On the other hand, it fully extracts diagnostically relevant image information through computer image processing, thus assisting doctors in improving diagnostic accuracy and reducing the probability of doctors missing relevant information due to poor image quality or patient diversity.

[0062] In this embodiment, the patient's eye image refers to an image obtained by imaging the patient's eyes, for example, by imaging the patient's eyes using the main camera of a slit lamp. Furthermore, the patient's eye image can be stored in a patient database after imaging. In the patient database, the patient's eye image is associated with the patient's identification information. Therefore, when diagnostic analysis of the patient is required, the associated patient's eye image can be obtained through the patient's identification information. For example, the shell obtains the associated patient's eye image through the patient's identity information.

[0063] In this application embodiment, the target detection grading model refers to a trained deep learning model that can locate disease areas, detect disease types, and grade diseases, and can be pre-deployed on relevant platforms.

[0064] In the embodiments of this application, the lesion area in the patient's eye image refers to the area in the patient's eye image that represents the possible lesion in the patient's eye, which can be determined by pixel coordinates.

[0065] In this embodiment, the predicted disease type refers to the type of eye disease that the model predicts the patient may have. Further, the predicted disease type may include eye diseases such as cataracts, corneal edema, and corneal ulcers.

[0066] In this embodiment, determining at least one predicted disease type based on the lesion region of a patient's eye image means that one predicted disease type can be determined based on the lesion region of the patient's eye image, or two or more predicted disease types can be determined based on the lesion region of the patient's eye image. For example, based on one lesion region, it can be determined that the patient's eye has two disease types: corneal edema and corneal ulcer. On the other hand, when locating the lesion region in a patient's eye image, the target detection grading model can locate two or more lesion regions, and then determine a predicted disease type based on each lesion region. Ultimately, the target detection grading model can determine two or more predicted disease types.

[0067] In this application embodiment, the predicted severity level of the disease type can refer to the predicted urgency level of the disease type. The predicted severity level of the disease type can be divided into multiple severity levels such as 0, 1, 2, 3, etc., where the smaller the value, the lower the urgency level, so that doctors can prioritize treatment according to the urgency level, thereby improving the efficiency of doctors in diagnosing and treating high-urgency conditions.

[0068] In this embodiment, the auxiliary diagnostic report is pre-stored in a patient database. When diagnosing a patient, the auxiliary diagnostic report related to that patient can be obtained through the patient's identity information. Furthermore, in addition to patient information, predicted symptom type, and predicted severity level, the auxiliary diagnostic report may also include an image of the symptom area, allowing doctors to further verify the predicted symptom type and its severity level based on the image.

[0069] In this embodiment of the application, as an optional implementation, after acquiring the patient's eye image and before inputting the patient's eye image into the target detection grading model, the method of this embodiment of the application further includes the following step:

[0070] Preprocessing operations are performed on the patient's eye images to ensure that the images meet the image input conditions of the target detection grading model. The preprocessing operations include size unification, brightness adjustment, contrast adjustment, and noise removal.

[0071] This optional implementation can perform operations such as size unification, brightness adjustment, contrast adjustment, and noise removal on the patient's eye image to make the patient's eye image meet the image input conditions of the target detection grading model, thereby enabling the model to output more accurate data based on the standardized input image.

[0072] In this optional implementation, the preprocessing operation refers to pre-processing the patient's eye image before inputting it into the target detection grading model, so as to prevent the target detection grading model from outputting incorrect results based on flawed images.

[0073] In this optional implementation, size unification processing refers to adjusting the resolution of the patient's eye image to a uniform resolution value that meets the resolution size requirements of the target detection grading model. For example, the resolution of the patient's eye image can be adjusted to 320pi*640pi.

[0074] In this optional implementation, brightness processing refers to adjusting the image brightness value of the patient's eye image to meet the image brightness requirements of the target detection grading model, thereby avoiding the target detection grading model from outputting erroneous results based on images that do not meet the conditions, such as excessively large and dark images.

[0075] In this optional implementation, contrast adjustment can be performed in conjunction with brightness processing to improve the effectiveness of both brightness processing and contrast adjustment.

[0076] In this optional embodiment, noise removal of the patient's eye image refers to removing image information from the patient's eye image that is unrelated to the lesion area and the disease, so as to reduce the accuracy of the output result of the target detection grading model by reducing irrelevant image information. The noise removal of the patient's eye image can be achieved by Gaussian filtering or median filtering.

[0077] In this optional embodiment, the preprocessing operation of the patient's eye image also includes data augmentation operation of the patient's eye image, wherein other data augmentation operations may include random flipping, random rotation, color jitter, partial occlusion, etc.

[0078] In this optional embodiment, the preprocessing operation on the patient's eye image may also include background processing and corresponding coordinate transformation of the annotation box.

[0079] In this embodiment of the application, as an optional implementation, before acquiring the patient's eye image, the method further includes the following steps:

[0080] Obtain the raw image data;

[0081] The original image data is annotated to determine the annotation information of lesion area, lesion type, and disease severity level, and feature image data is obtained.

[0082] The feature image data is divided into a training set, a validation set, and a test set;

[0083] Train a deep learning model based on the training set;

[0084] When the deep learning model is completed, it is validated on the validation set, and the parameters of the deep learning model are adjusted and the deep learning model is retrained based on the validation results until the deep learning model meets the preset conditions.

[0085] When a deep learning model meets the preset conditions, the deep learning model is tested based on the test set, and the deep learning model that passes the test is determined as the object detection classification model.

[0086] This optional implementation acquires and annotates raw image data to determine the lesion region annotation information, lesion type annotation information, and disease severity level annotation information of the raw image data, thereby obtaining feature image data. Furthermore, by dividing the feature image data into training set, validation set, and test set, a deep learning model can be trained based on the training set. When the deep learning model is completed, it is validated based on the validation set, and the parameters of the deep learning model are adjusted and the deep learning model is retrained based on the validation results until the deep learning model meets the preset conditions. When the deep learning model meets the preset conditions, it is tested based on the test set, and the deep learning model that passes the test is determined as the target detection grading model.

[0087] In this optional implementation, raw image data refers to previously acquired, unlabeled images of the patient's eyes, which can be collected from medical institutions, laboratories, or standard databases. Furthermore, raw image data can include images captured under different shooting environments, lighting conditions, and device types.

[0088] In this optional embodiment, when acquiring the original image data, the original image data can be subjected to quality inspection to remove invalid images that are blurry, severely overexposed, too dark, or cannot be used to identify the disease.

[0089] In this optional embodiment, the lesion region annotation information of the original image data refers to the coordinate information of the positioning box used to locate the lesion region of the original image data.

[0090] In this optional implementation, the lesion type labeling information refers to the name of the lesion type.

[0091] In this optional embodiment, the feature image data refers to comprehensive data including original image data as well as lesion area annotation information, lesion type annotation information, and disease type severity level annotation information.

[0092] In this optional implementation, the deep learning model includes a backbone network for extracting multi-level features from the original image, wherein the backbone network may be a deep convolutional neural network.

[0093] In this optional implementation, the deep learning model also includes a feature fusion layer called Neck, which is used to fuse multi-scale features output by the backbone network in order to better detect lesion regions of different scales and types.

[0094] In this optional implementation, the deep learning model further includes detection heads, which include classification branches and regression branches for predicting disease category, grade, and precise localization of candidate regions.

[0095] In this optional implementation, one specific way to train a deep learning model based on the training set can be:

[0096] Set the initial parameters of the deep learning model. The initial parameters of the deep learning model can be set by using the parameters of an existing pre-trained model or by using a random initialization method.

[0097] The parameters of a deep learning model are iteratively optimized based on the training set.

[0098] In this optional implementation, when dividing the feature image data into a training set, a validation set, and a test set, the ratio of the training set, the validation set, and the test set can be 8:1:1.

[0099] In this optional implementation, the deep learning model satisfying the preset condition can mean that the deep learning model has been trained a specified number of times, or it can mean that the deep learning model has met the convergence condition. In this optional implementation, a specific way to adjust the parameters of the deep learning model based on the validation results can be:

[0100] Based on the validation results, mAP, precision, and recall are calculated, where mAP refers to the mean average precision.

[0101] Adjust the learning rate or data augmentation strategy based on mAP, precision, and recall.

[0102] In this optional implementation, the database storing the original image data can serve as a feature database, while the trained model can be stored in an optimized model library. Furthermore, by constructing, continuously updating, and optimizing the model library and feature database step-by-step, the system can dynamically adapt to emerging disease feature standards and medical diagnostic requirements. For example, when new diagnostic criteria, sub-grading requirements, or additional disease types appear in clinical practice, the model weights and feature information can be flexibly iterated, quickly applying the improved diagnostic strategy to the actual testing process. This ability to dynamically adapt and iteratively update enhances the system's sustainability and sophistication in responding to the increasing demands and complexity of future ophthalmic diagnostic needs.

[0103] In this embodiment of the application, as an optional implementation, the target detection grading model includes a backbone network, a feature fusion layer, and a detection head. Accordingly, based on the target detection grading model, lesion regions in patient eye images are identified, and based on the lesion regions in patient eye images, at least one predicted disease type and the severity level of the predicted disease type are determined, including the following sub-steps:

[0104] Multi-scale image features of patient eye images are extracted based on the backbone network, and feature maps are obtained;

[0105] The feature maps are fused based on the feature fusion layer to obtain the feature fusion result;

[0106] The feature fusion results are analyzed based on the detection head to determine the lesion area in the patient's eye image, predict the disease type, and predict the severity level of the disease type.

[0107] This optional implementation extracts multi-scale image features from the patient's eye image based on the backbone network, obtaining a feature map. The feature map is then fused using a feature fusion layer to obtain a feature fusion result. This result is further analyzed using a detection head to determine the lesion region in the patient's eye image, predict the disease type, and predict the severity level of the disease type. The use of multi-scale features improves the accuracy of the model's output.

[0108] In this optional embodiment, the multi-scale image features of the patient's eye image may include small-scale image features, medium-scale image features, and large-scale image features of the patient's eye image, wherein small-scale, medium-scale, and large-scale each correspond to a resolution.

[0109] In this optional implementation, two or more feature maps are obtained, thus requiring multiple feature maps to be fused.

[0110] In this embodiment of the application, as an optional implementation, the feature fusion result is obtained by fusing feature maps based on the feature fusion layer, including the following steps:

[0111] The number of tokens is determined based on the height and width of the feature map, and the feature dimension is determined based on the number of channels in the feature map.

[0112] The query matrix, key matrix, and value matrix are determined based on the number of tokens and feature dimensions.

[0113] Attention information is determined based on the query matrix, key matrix, and value matrix;

[0114] Feature maps are fused based on attention information to obtain feature fusion results.

[0115] This optional implementation can determine the number of tokens based on the height and width of the feature map, and determine the feature dimension based on the number of channels in the feature map. Then, it can determine the query matrix, key matrix, and value matrix based on the number of tokens and the feature dimension. Thus, it can determine attention information based on the query matrix, key matrix, and value matrix, and then fuse the feature map based on the attention information to obtain the feature fusion result. In this case, feature fusion based on the attention mechanism can further improve the accuracy of the model.

[0116] In this optional implementation, a token refers to a basic discretization unit of data, used to convert continuous or complex raw data into a discrete symbol sequence that the model can process. In this optional implementation, the token converts the feature map into a discrete symbol sequence that the model can process.

[0117] In this optional implementation, a channel of a feature map refers to a dimension in the feature map that carries a specific type of information, wherein a channel of a feature map corresponds to a feature dimension.

[0118] In this optional implementation, the feature map is two-dimensional or three-dimensional data output after operations such as convolutional layers and pooling layers, and is usually represented as [H,W,C] or [C,H,W], where H is the height, W is the width, and C is the number of channels.

[0119] In this optional implementation, the query matrix (Q), key matrix (K), and value matrix (V) refer to the core components of the self-attention mechanism.

[0120] In this optional implementation, the feature fusion layer is responsible for effectively fusing the multi-scale features output by the backbone network to facilitate accurate prediction by subsequent detection heads. In this model, the feature fusion layer introduces the C2PSA (C2withPointwiseSpatialAttention) mechanism, which embeds a multi-head attention mechanism within the C2 module. Specifically, C2PSA uses the height*width of the input feature map as the number of tokens and the channel as the feature dimension to calculate the query (Q), key (K), and value (V), thereby performing attention calculation in the spatial dimension and enhancing the model's focus on important regions. Through C2PSA, the model can more accurately capture important regions in the image in the spatial dimension, improving the detection capability of objects at different locations and scales. Furthermore, the multi-head attention mechanism allows the model to capture features in parallel from multiple subspaces, improving the diversity and expressive power of feature fusion. On the other hand, while fusing multi-scale features, C2PSA automatically adjusts the importance of different feature maps, avoiding information loss or redundancy and improving the overall feature fusion effect.

[0121] In this optional implementation, the C2PSA module inherits the segmented feature processing feature of CSP. The segmented feature processing feature of CSP divides the feature into two parts after a 1x1 convolution. One part is directly transmitted, and the other part is processed by the PSA attention module. Then the two parts of the feature are concatenated and processed by another 1x1 convolution to restore the original number of channels.

[0122] In this optional implementation, the PSA (Pyramid Squeeze Attention) mechanism is an efficient attention mechanism that extracts multi-scale features using various convolutional kernels (such as 3x3, 5x5, 7x7, etc.). Convolutional operations with different kernels are performed in parallel, and the feature maps are then concatenated. The SE (Squeeze-and-Excitation) module is used to weight the feature channels. Finally, attention weights generated by Softmax are applied to each feature map, thereby achieving channel-wise multiplication and increasing attention to important features.

[0123] In this optional implementation, Squeeze-and-Excitation (SE) is an attention mechanism module designed to enhance the feature representation capabilities of convolutional neural networks. It improves the model's representational power by dynamically adjusting the weights of each feature channel, allowing the model to focus more on important features and ignore irrelevant information. Specifically, it compresses the spatial dimension of the input feature map into a global feature vector through global average pooling, obtaining a global feature representation for each channel. This step allows the model to integrate spatial information globally. Then, two fully connected layers are used to model the channel weights and calculate the importance of each channel. In the first fully connected layer, the number of channels is first reduced (through dimensionality reduction, typically by a factor of 16) to reduce computational cost. The second fully connected layer restores the number of channels, and finally, a sigmoid activation function is used to generate the weights for each channel. Finally, the generated channel weights are applied to the input feature map, adjusting the feature intensity of each channel through channel-wise multiplication. In this way, important feature channels are enhanced, while unimportant features are suppressed.

[0124] In this embodiment of the application, as an optional implementation, the head detection includes a classification branch and a regression branch. The regression branch is used to locate the lesion area in the patient's eye image, and the classification branch is used to determine the predicted disease type and the predicted severity level of the disease type.

[0125] This optional implementation can locate the lesion area in the patient's eye image through regression branching, and on the other hand, can predict the disease type and the severity level of the disease type through classification branching.

[0126] In this optional implementation, head detection refers to a component that automatically locates the position and range of a human head in an image or video through an algorithm, and typically outputs key information about the head, such as bounding box coordinates and center point.

[0127] In this optional implementation, the regression branch refers to the sub-network module in the model responsible for predicting the spatial location information of the head target. It learns the features of the input image and outputs the precise coordinates of the head target (such as bounding box, center point, etc.). This optional implementation utilizes the regression branch to locate the lesion region.

[0128] In this optional implementation, the classification branch, responsible for determining "whether it is a head," constitutes a core component of the head detection model. In this optional implementation, the classification branch, through training, can be used to determine the predicted symptom type and the predicted severity level of the symptom type.

[0129] In this embodiment of the application, as an optional implementation, the classification branch includes two depthwise separable convolutions.

[0130] In this optional implementation, the classification branch includes two depthwise separable convolutions. By using two depthwise separable convolutions, the amount of computation and parameters can be reduced, thereby improving the overall efficiency.

[0131] In this optional implementation, depthwise separable convolution refers to an optimized structure for traditional convolution. By separating "spatial convolution" from "channel convolution", it significantly reduces the amount of computation and the number of parameters while maintaining the ability to extract similar features.

[0132] In this optional implementation, depthwise separable convolution decomposes standard convolution into channel-wise convolution and pointwise convolution, significantly reducing computational cost and parameter count. This reduced computational overhead allows the model to process input more quickly during inference, improving real-time detection capabilities. Furthermore, despite employing lightweight convolution operations, depthwise separable convolution (DWConv) effectively captures features, maintaining and even, in some cases, improving classification and regression accuracy.

[0133] In this embodiment of the application, as an optional implementation, the loss function of the target detection hierarchical model is the EIoU loss function.

[0134] This optional implementation can use the EIoU loss function as the loss function of the object detection grading model. The EIoU loss function takes into account the overlap area between the predicted box and the ground truth box, the aspect ratio, and the center point offset, thereby improving the overall prediction accuracy.

[0135] In this optional implementation, the novel EIoU (Extended IoU) loss function considers the overlap area between the predicted bounding box and the ground truth bounding box, aspect ratio, and center point offset, thus improving overall prediction accuracy. Existing IoU-based loss functions cannot adequately describe the bounding box, leading to slow model convergence and inaccuracy; most loss functions ignore the problem of imbalanced positive and negative samples, meaning that a large number of predicted bounding boxes with small overlap areas with the target bounding box contribute the majority of the final bounding box optimization.

[0136] In this optional implementation, the EIoU (Extended IoU) loss function is calculated as follows:

[0137]

[0138] Among them, b p ,b gt These are the center coordinates of the target box and the predicted box, where d represents the Euclidean distance between them, and w...c and h c It is the width and height of the smallest outer rectangle of the predicted bounding box and the target bounding box. Similarly, p and gt represent the information of the predicted bounding box and the target bounding box, respectively.

[0139] In this embodiment of the application, as an optional implementation, the backbone network includes a C3k2 module.

[0140] This optional implementation, through the C3k2 module, can reduce computational overhead and avoid parameter redundancy.

[0141] In this optional implementation, the C3k2 module can flexibly switch between the standard bottleneck and the deeper C3 module depending on the c3k parameter. This allows the network to adjust the depth and complexity of feature extraction according to different task requirements. Specifically, by using the standard bottleneck structure in a shallow layer, the C3k2 module avoids unnecessary computational overhead when c3k = False, maintaining efficient feature extraction capabilities. Furthermore, compared to the traditional C2f module, the C3k2 module reduces the number of model parameters while maintaining the same feature representation capabilities, thus improving overall parameter efficiency.

[0142] Please see Figure 2 , Figure 2 This is a schematic diagram of the structure of an intelligent fully automated diagnostic and analysis device for anterior segment diseases disclosed in an embodiment of this application. Figure 2 As shown, the apparatus in this embodiment includes the following functional modules:

[0143] The acquisition module 201 is used to acquire images of the patient's eyes;

[0144] The identification module 202 is used to input the patient's eye image into the target detection grading model, identify the lesion area of ​​the patient's eye image based on the target detection grading model, and determine at least one predicted disease type and the severity level of the predicted disease type based on the lesion area of ​​the patient's eye image.

[0145] The generation module 203 is used to generate an auxiliary diagnostic report based on the predicted symptom type and the severity level of the predicted symptom type.

[0146] The device in this embodiment of the application, by executing an intelligent fully automated diagnostic analysis method for anterior segment diseases, can acquire images of a patient's eye, and then input these images into a target detection and grading model. Based on the target detection and grading model, lesion regions in the patient's eye images are identified, and at least one predicted disease type and a severity level of the predicted disease type are determined based on these lesion regions. This allows for the generation of an auxiliary diagnostic report based on the predicted disease type and its severity level. Compared with existing technologies, this device, on the one hand, can assist in locating disease regions, and then provide doctors with the predicted disease type's severity level, predicted disease type, and auxiliary diagnostic information through the auxiliary diagnostic report, thereby improving the doctor's diagnostic efficiency. On the other hand, it can fully extract diagnostically relevant image information through computer image processing, thereby assisting doctors in improving diagnostic accuracy and reducing the probability of doctors missing relevant information under conditions such as poor image quality or patient diversity.

[0147] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. For example... Figure 3 As shown, the electronic device includes:

[0148] Processor 301; and

[0149] The memory 302 is configured to store machine-readable instructions that, when executed by the processor 301, perform the intelligent fully automated diagnostic analysis method for anterior segment diseases as described in any of the foregoing embodiments.

[0150] The electronic device in this application embodiment, by executing an intelligent fully automated diagnostic analysis method for anterior segment diseases, can acquire images of a patient's eye. These images are then input into a target detection and grading model. Based on this model, lesion regions in the patient's eye images are identified, and at least one predicted disease type and its severity level are determined. This allows for the generation of an auxiliary diagnostic report based on the predicted disease type and its severity level. Compared to existing technologies, this method, on the one hand, assists in locating disease areas and provides doctors with the predicted disease type's severity level, severity level, and auxiliary diagnostic information through the auxiliary diagnostic report, thereby improving diagnostic efficiency. On the other hand, it can fully extract diagnostically relevant image information through computer image processing, thus assisting doctors in improving diagnostic accuracy and reducing the probability of missing relevant information due to poor image quality or patient diversity.

[0151] Furthermore, this application also provides a storage medium storing a computer program, which is executed by a processor as described in any of the foregoing embodiments, a fully automated intelligent diagnostic analysis method for anterior segment diseases.

[0152] The storage medium in this application embodiment, by executing an intelligent fully automated diagnostic analysis method for anterior segment diseases, can acquire patient eye images. These images are then input into a target detection and grading model. Based on this model, lesion regions in the patient's eye images are identified, and at least one predicted disease type and its severity level are determined. This allows for the generation of an auxiliary diagnostic report based on the predicted disease type and its severity level. Compared to existing technologies, this method, on the one hand, assists in locating disease areas and provides doctors with the predicted disease type's severity level, severity level, and auxiliary diagnostic information through the auxiliary diagnostic report, thereby improving doctors' diagnostic efficiency. On the other hand, it can fully extract diagnostically relevant image information through computer image processing, thus assisting doctors in improving diagnostic accuracy and reducing the probability of doctors missing relevant information due to poor image quality or patient diversity.

[0153] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0154] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0155] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0156] It should be noted that if a function is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0158] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A fully automated intelligent diagnostic and analysis method for anterior segment diseases, characterized in that, The method includes: Acquire images of the patient's eyes; The patient's eye image is input into a target detection grading model, the lesion region of the patient's eye image is identified based on the target detection grading model, and at least one predicted disease type and the severity level of the predicted disease type are determined based on the lesion region of the patient's eye image. An auxiliary diagnostic report is generated based on the predicted symptom type and the severity level of the predicted symptom type.

2. The method as described in claim 1, characterized in that, After acquiring the patient's eye image and before inputting the patient's eye image into the target detection grading model, the method further includes: The patient's eye image is preprocessed to meet the image input conditions of the target detection grading model. The preprocessing operations include size unification, brightness adjustment, contrast adjustment, and noise removal.

3. The method as described in claim 1, characterized in that, Before acquiring images of the patient's eyes, the method further includes: Obtain the raw image data; The original image data is annotated to determine the lesion area annotation information, lesion type annotation information, and disease type severity level annotation information of the original image data, and feature image data is obtained; The feature image data is divided into a training set, a validation set, and a test set; Train a deep learning model based on the training set; When the deep learning model is completed, the deep learning model is validated based on the validation set, and the parameters of the deep learning model are adjusted and the deep learning model is retrained based on the validation results until the deep learning model meets the preset conditions. When the deep learning model meets the preset conditions, the deep learning model is tested based on the test set, and the deep learning model that passes the test is determined as the target detection classification model.

4. The method as described in claim 1, characterized in that, The target detection hierarchical model includes a backbone network, a feature fusion layer, and a detection head; And, based on the target detection grading model, identify the lesion region of the patient's eye image, and determine at least one predicted disease type and the severity level of the predicted disease type based on the lesion region of the patient's eye image, including: Based on the backbone network, multi-scale image features of the patient's eye image are extracted, and a feature map is obtained; The feature map is fused based on the feature fusion layer to obtain the feature fusion result; The feature fusion results are analyzed based on the detection head to determine the lesion area, predicted disease type, and severity level of the predicted disease type in the patient's eye image.

5. The method as described in claim 4, characterized in that, The feature maps are fused based on the feature fusion layer to obtain the feature fusion result, including: The number of tokens is determined based on the height and width of the feature map, and the feature dimension is determined based on the number of channels in the feature map. The query matrix, key matrix, and value matrix are determined based on the number of tokens and the feature dimensions. Attention information is determined based on the query matrix, the key matrix, and the value matrix; The feature map is fused based on the attention information to obtain the feature fusion result.

6. The method as described in claim 4, characterized in that, The detection head includes a classification branch and a regression branch. The regression branch is used to locate the lesion area in the patient's eye image, and the classification branch is used to determine the predicted disease type and the severity level of the predicted disease type.

7. The method as described in claim 6, characterized in that, The classification branch consists of two depthwise separable convolutions.

8. The method as described in claim 1, characterized in that: The loss function of the target detection grading model is the EIoU loss function.

9. The method as described in claim 4, characterized in that, The backbone network includes C3k2 modules.

10. An intelligent fully automated diagnostic and analysis device for anterior segment diseases, characterized in that, The device includes: The acquisition module is used to acquire images of the patient's eyes; The identification module is used to input the patient's eye image into the target detection grading model, identify the lesion area of ​​the patient's eye image based on the target detection grading model, and determine at least one predicted disease type and the severity level of the predicted disease type based on the lesion area of ​​the patient's eye image. The generation module is used to generate an auxiliary diagnostic report based on the predicted symptom type and the severity level of the predicted symptom type.

11. An electronic device, characterized in that, include: processor; as well as A memory configured to store machine-readable instructions that, when executed by the processor, perform the intelligent fully automated diagnostic analysis method for anterior segment diseases as described in any one of claims 1-9.

12. A storage medium, characterized in that, The storage medium stores a computer program, which is executed by a processor as described in any one of claims 1-9: the intelligent fully automated diagnostic analysis method for anterior segment diseases.

Images