An intelligent screening method, system and device for multiple diseases of the anterior segment of the eye
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANTOU UNIV·CHINESE UNIV OF HONG KONG JOINT SHANTOU INT OPHTHALMOLOGY CENT
- Filing Date
- 2026-07-08
- Publication Date
- 2026-08-04
AI Technical Summary
[0004]针对现有技术中白内障、结膜充血、翼状胬肉等11类眼前节疾病诊断主观性强、多疾病综合筛查能力不足、报告生成效率低、缺乏标准化图文呈现及生成式报告专业性欠缺等问题，本发明提出了一种眼前节多疾病智能筛查方法、系统及设备
1.对白内障、结膜充血等11类疾病的诊断AUC值均≥0.93，其中结膜充血与结膜出血鉴别 AUC≥0.95，角膜混浊与角膜水肿鉴别 AUC≥0.94；病情分级准确率≥89%，早期微小病变（如早期白内障混浊、轻微结膜充血）识别灵敏度≥86%，优于人工诊断水平。
Smart Images
Figure CN122511554A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical artificial intelligence technology, and more specifically, to an intelligent screening method, system, and device for multiple diseases of the anterior segment. Background Technology
[0002] With the increasing penetration of artificial intelligence (AI) technology into the medical field, research on ophthalmic AI diagnosis focusing on fundus diseases has become relatively mature. However, the application of AI in the diagnosis of anterior segment diseases is still in its early stages. Slit-lamp examination of the anterior segment is a routine ophthalmic examination method that can observe the anterior segment structures of the eye, such as the cornea, conjunctiva, iris, and lens. It is a core diagnostic basis for various diseases, including conjunctivitis, keratitis, cataracts, and glaucoma.
[0003] Existing technologies for anterior segment disease diagnosis and report generation have several limitations: First, diagnosis relies on doctors' subjective observation and experience, which can easily lead to missed or misdiagnosed cases due to a shortage of professional personnel in primary healthcare institutions. Second, traditional slit-lamp examination results lack standardized recording, and manual report writing is time-consuming, inconsistent in expression, and difficult to track disease progression. Third, existing AI technologies are mostly developed for single diseases, lacking the ability to comprehensively screen for multiple diseases, and only output diagnostic results without accompanying professional graphic reports, which cannot directly support clinical diagnosis and treatment decisions. Fourth, reports lack visual annotation and quantitative analysis data of lesion areas, requiring doctors to re-verify images, reducing diagnostic efficiency. Fifth, the application of generative AI in medical reports is not yet mature, and existing report generation tools are mostly template-filling, lacking personalized and professional descriptions of disease characteristics. Summary of the Invention
[0004] To address the problems in existing technologies regarding the diagnosis of 11 types of anterior segment diseases, including cataracts, conjunctival congestion, and pterygium, such as high subjectivity, insufficient comprehensive screening capabilities for multiple diseases, low report generation efficiency, lack of standardized graphic presentation, and insufficient professionalism in generated reports, this invention proposes an intelligent screening method, system, and device for multiple anterior segment diseases. The method specifically includes the following steps:
[0005] S1. Acquire an anterior segment slit lamp image and input the image into a pre-trained multi-task detection model to detect key anatomical points and lesion areas of the eye, and obtain lesion detection boxes, disease categories and coordinates of key anatomical points of the eye. S2. Based on the coordinates of the key points of the eye anatomy, establish an eye polar coordinate system and perform geometric calculations on the lesion detection frame to obtain the geometric parameters between the lesion and the eye anatomy structure. S3. Generate structured semantic description information based on the geometric parameters; S4. Input the structured semantic description information into the large language model to generate a graphic report on the diagnosis of anterior segment diseases.
[0006] In step S1, the coordinates of the key anatomical points are used to locate the positions of ocular anatomical structures. In step S2, the polar point is selected at the center of the pupil to establish an ocular polar coordinate system. The reference positions of ocular anatomical structures in the image are determined by the coordinates of the key points. The coordinates of the lesion detection box vertices are transformed into this polar coordinate system for geometric calculations to obtain the geometric parameters between the lesion and the ocular anatomical structures, such as the polar angle and polar diameter of the lesion, as well as the overlap, intersection, and union ratio with the pupil region and the limbus region.
[0007] The geometric data obtained in step S2 exists in numerical form and is not suitable for direct text generation. Therefore, step S3 converts these parameters into semantic descriptions, such as the location of the lesion, whether it is close to the pupil area, and whether it covers the limbus. This process transforms the visual detection results into structured information that a language model can understand. Step S4 inputs the structured semantic description information obtained in step S3 into a large language model to generate a diagnostic image-text report. The report includes the disease category, lesion location description, and corresponding explanations.
[0008] Furthermore, the multi-task detection model includes a backbone network, a feature fusion network, and a detection head; The multi-task detection model introduces a C2PSA attention mechanism into the backbone network, disables the c3k parameter in the shallow structure of the backbone network, and uses an asymmetric multi-task joint loss function for training, in which the regression loss weight of anatomical key points is higher than the regression loss weight of lesion detection boxes.
[0009] In the multi-task detection model, the backbone network is responsible for extracting image features, while the feature fusion network integrates features at different scales. The detection head outputs lesion detection boxes, disease categories, and coordinates of key anatomical points of the eye. A C2PSA attention mechanism is introduced into the backbone network. This attention mechanism captures long-distance dependencies in the eye image (e.g., associating the spatial locations of "pterygium head" and "pupil center") by calculating the global context of the feature map.
[0010] The c3k parameter is disabled in the shallow structure of the backbone network to preserve the texture features of small lesions (such as early neovascularization and subtle corneal opacities) to the greatest extent possible, preventing information loss during downsampling. The model training assigns high weights to the ocular anatomical keypoint regression loss. The priority of the anatomical keypoint localization task is placed above that of the lesion detection task. Only when the anatomical keypoint regression is accurate does the determination of the relative location of lesions have clinical significance, thereby effectively suppressing false detections caused by background noise.
[0011] Furthermore, the key anatomical points of the eye in S1 include at least the pupil center, the pupil edge point, and the corneal limbus edge point; Step S2 specifically involves establishing an eye polar coordinate system with the center of the pupil as the pole.
[0012] Key points in ocular anatomy include the pupil center, pupil edge point, and limbal edge point. The pupil center is used to determine the location of the pole in the polar coordinate system, the pupil edge point is used to fit the pupil contour, and the limbal edge point is used to fit the limbal contour.
[0013] Furthermore, the geometric parameters in step S2 include the polar radius and polar angle of the vertex of the lesion detection frame, as well as the overlap ratio of the lesion detection frame with the pupil region and the limbus region.
[0014] The polar angle indicates the direction of the lesion in the eye, such as its location above, below, or to the side. The polar diameter indicates its relative position from the center of the pupil.
[0015] Furthermore, the pupil region and the limbal region are obtained by performing least-squares circle fitting or ellipse fitting on the pupil edge points and the limbal edge points.
[0016] By fitting key points of ocular anatomy, the boundaries of the region can be obtained. Only a small number of key points are needed to determine the range of the pupil region and the limbus region.
[0017] Furthermore, step S3 includes converting the geometric parameters into structured semantic information through a preset interval mapping rule.
[0018] After this mapping process, all the geometric parameters obtained in step S2 are converted into readable structured semantic information.
[0019] Furthermore, the structured semantic information includes disease category, lesion location, invasion depth, and occlusion relationship.
[0020] Disease categories are derived from the disease category results obtained by the S1-step multi-task detection model. Lesion orientation describes the directional position of the lesion within the eye, obtained based on the polar radius and polar angle of the lesion detection frame vertex. Invasion depth is determined by the relationship between the polar radius of the lesion detection frame vertex and the pupil radius and limbal radius, describing the degree of outward expansion of the lesion relative to the pupil center. Occlusion relationship is determined based on the overlap, intersection, and union ratio (IoU) between the lesion detection frame and the pupillary region or limbal region.
[0021] Furthermore, the method also includes performing a consistency check on the generated diagnostic report. The consistency check includes extracting key entity information from the anterior segment disease diagnostic graphic report; inversely mapping the key entity information into virtual geometric coordinates; comparing the virtual geometric coordinates with the lesion detection box output in step S1, and triggering report regeneration if the overlap is lower than a threshold.
[0022] Key entity information, such as lesion location descriptions and related ocular anatomical structures, is converted into virtual geometric coordinates. These virtual geometric coordinates are then compared with the lesion detection bounding box obtained in step S1. If the overlap between the two is less than a set threshold, the text description is considered inconsistent with the detection result, and a report regeneration process is triggered.
[0023] This invention also provides an AI-powered intelligent screening system for multiple diseases of the anterior segment that integrates generative graphic reports, characterized by comprising: The detection module is used to acquire anterior segment slit lamp images and input the images into a pre-trained multi-task detection model to detect key anatomical points and lesion areas of the eye, and obtain lesion detection boxes, disease categories and coordinates of key anatomical points of the eye. The geometric calculation module is used to establish an ocular polar coordinate system based on the coordinates of the key anatomical points of the eye, and to perform geometric calculations on the lesion detection box to obtain the geometric parameters between the lesion and the ocular anatomical structure. A semantic transformation module is used to generate structured semantic description information based on the geometric parameters; The report generation module is used to input the structured semantic description information into a large language model to generate a graphic report on the diagnosis of anterior segment diseases.
[0024] The present invention also includes a computer device, characterized in that it comprises: a processor; a memory storing a computer program; and when the computer program is executed by the processor, it causes the processor to perform the above-described intelligent screening method for multiple diseases of the anterior segment.
[0025] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. The diagnostic AUC values for 11 types of diseases, including cataracts and conjunctival congestion, are all ≥0.93. Among them, the AUC for differentiating conjunctival congestion from conjunctival hemorrhage is ≥0.95, and the AUC for differentiating corneal opacity from corneal edema is ≥0.94. The accuracy rate of disease grading is ≥89%, and the sensitivity for identifying early minimal lesions (such as early cataract opacity and mild conjunctival congestion) is ≥86%, which is better than the level of manual diagnosis.
[0026] 2. The total time for single image analysis and generation of a complete text and image report is ≤10 seconds. The report text description conforms to clinical professional standards and can accurately present the pathological characteristics and diagnostic basis of each type of disease. The combination of text and images is intuitive and easy to understand, and no additional report writing is required.
[0027] 3. It provides unified diagnostic standards, grading systems and quantitative data for 11 types of diseases, avoiding subjective bias in manual diagnosis and facilitating disease progression tracking and multi-center data comparison; the visual annotation of lesion areas helps doctors quickly locate lesions and improve diagnostic and treatment efficiency.
[0028] 4. Compatible with conventional slit lamps / mobile phone cameras, requiring no additional hardware modifications, and can be directly deployed in primary healthcare institutions; reports support export in multiple formats and are compatible with electronic medical record systems, meeting clinical data management needs.
[0029] 5. Reduce reliance on doctors' experience in diagnosis, improve the diagnosis and treatment level of 11 common anterior segment diseases in primary healthcare institutions, expand the accessibility of high-quality medical resources, reduce the increase in medical costs caused by missed diagnoses and misdiagnoses, and reduce the workload of clinicians. Attached Figure Description
[0030] Figure 1 This is a schematic diagram of the overall process of the intelligent screening method for multiple diseases of the anterior segment of the present invention.
[0031] Figure 2 This is a schematic diagram of the Visual-Net network structure of the multi-task detection model of the present invention.
[0032] Figure 3 This is a schematic diagram illustrating the mapping logic from visual features to medical semantic information in this invention.
[0033] Figure 4 This is a flowchart illustrating the generation and verification process of the generative graphic report of this invention. Detailed Implementation
[0034] Example 1 like Figure 1 As shown, this embodiment provides an intelligent screening method for multiple anterior segment diseases. In this embodiment, anterior segment diseases include cataracts, conjunctival hyperemia, pterygium, irregular pupils, corneal edema, conjunctival hemorrhage, corneal opacity, neovascularization, trichiasis, corneal ulcers, and blepharitis. The method specifically includes the following steps: S1. Acquire an anterior segment slit lamp image and input the image into a pre-trained multi-task detection model to detect key anatomical points and lesion areas of the eye, and obtain lesion detection boxes, disease categories and coordinates of key anatomical points of the eye. S2. Based on the coordinates of the key points of the eye anatomy, establish an eye polar coordinate system and perform geometric calculations on the lesion detection frame to obtain the geometric parameters between the lesion and the eye anatomy structure. S3. Generate structured semantic description information based on the geometric parameters; S4. Input the structured semantic description information into the large language model to generate a graphic report on the diagnosis of anterior segment diseases.
[0035] Preferably, in step S1, the acquired anterior segment slit lamp image is first preprocessed. The original images at different resolutions are scaled proportionally to 640x640 pixels. To avoid image distortion caused by traditional direct stretching, this method calculates a scaling factor during the scaling process and fills the remaining edge areas. The fill value uses gray (RGB: 114, 114, 114) to maintain the proportional relationship of ocular anatomical structures such as the pupil and cornea.
[0036] To address the characteristics of slit lamp light sources, slit lamp images were converted to the HSV color space, and hue, saturation, and luminance were enhanced using specific parameters: hue gain of 0.015, saturation gain of 0.7, and luminance gain of 0.4. This parameter combination specifically enhances the contrast between lesion features (such as the grayish-white opacity of corneal ulcers and the fine red streaks of conjunctival hyperemia) and normal ocular surface tissue.
[0037] During the model training phase, to improve the model's ability to recognize multiple disease scenarios, a Mosaic data augmentation strategy was used to randomly stitch together training images, combining four images into one training sample. When constructing batch data, low-frequency diseases (such as occult corneal foreign bodies and early neovascularization) were forcibly included to address the imbalance in the recognition of 11 disease categories caused by the long-tail distribution of medical data.
[0038] Preferably, before generating a diagnostic report for anterior segment diseases, the system first identifies the disease category and its severity based on structured semantic description information, and retrieves relevant treatment guidelines from the local medical knowledge base. For example, when "pterygium invading the pupillary area" is identified, the system automatically retrieves relevant treatment recommendations from the consensus on pterygium treatment and extracts corresponding knowledge fragments as prompts. Subsequently, the structured semantic description information and the retrieved medical knowledge are combined to construct prompt words, which are then input into a large language model to guide the generation of a diagnostic report that conforms to clinical standards.
[0039] Preferably, after generating the diagnostic text, the system combines the generated diagnostic report with the original slit-lamp image for display. The image portion includes the lesion detection bounding box, key point connections, and corresponding confidence level annotations output by the model. Subsequently, the system automatically completes the text and image layout and generates a diagnostic report file, such as a PDF file, for doctors to view or archive.
[0040] In step S1, the coordinates of the key anatomical points are used to locate the positions of ocular anatomical structures. In step S2, the polar point is selected at the center of the pupil to establish an ocular polar coordinate system. The reference positions of ocular anatomical structures in the image are determined by the coordinates of the key points. The coordinates of the lesion detection box vertices are transformed into this polar coordinate system for geometric calculations to obtain the geometric parameters between the lesion and the ocular anatomical structures, such as the polar angle and polar diameter of the lesion, as well as the overlap, intersection, and union ratio with the pupil region and the limbus region.
[0041] The geometric data obtained in step S2 exists in numerical form and is not suitable for direct text generation. Therefore, step S3 converts these parameters into semantic descriptions, such as the location of the lesion, whether it is close to the pupil area, and whether it covers the limbus. This process transforms the visual detection results into structured information that a language model can understand. Step S4 inputs the structured semantic description information obtained in step S3 into a large language model to generate a diagnostic image-text report. The report includes the disease category, lesion location description, and corresponding explanations.
[0042] Furthermore, the multi-task detection model includes a backbone network, a feature fusion network, and a detection head; The multi-task detection model introduces a C2PSA attention mechanism into the backbone network, disables the c3k parameter in the shallow structure of the backbone network, and uses an asymmetric multi-task joint loss function for training, in which the regression loss weight of anatomical key points is higher than the regression loss weight of lesion detection boxes.
[0043] like Figure 2 As shown, this multi-task detection model includes a backbone network, a feature fusion network (Neck), and a head. The backbone network is responsible for extracting image features, the feature fusion network integrates features at different scales, and the head outputs lesion detection boxes, disease categories, and coordinates of key anatomical points of the eye. A C2PSA attention mechanism is introduced into the backbone network. This attention mechanism captures long-distance dependencies in the eye image (e.g., associating the spatial locations of "pterygium head" and "pupil center") by calculating the global context of the feature map.
[0044] Preferably, in the feature fusion network and detection head, depthwise separable convolution (DWConv) is introduced to replace standard convolution. By decomposing the convolution into two parts, channel-wise convolution and point-wise convolution, the computational load and parameter scale of the multi-task detection model are effectively reduced, enabling the model to run on medical terminal devices with limited computing power.
[0045] The c3k parameter is disabled in the shallow structure of the backbone network to preserve the texture features of small lesions (such as early neovascularization and subtle corneal opacities) to the greatest extent possible, preventing information loss during downsampling. The model training assigns high weights to the ocular anatomical keypoint regression loss. The priority of the anatomical keypoint localization task is placed above that of the lesion detection task. Only when the anatomical keypoint regression is accurate does the determination of the relative location of lesions have clinical significance, thereby effectively suppressing false detections caused by background noise.
[0046] Preferably, the weight of the anatomical keypoint pose estimation loss is set to λpose=12.0, the weight of the lesion detection box regression loss is set to λbox=7.5, and the weight of the keypoint target loss is set to λobj=1.0.
[0047] Furthermore, the key anatomical points of the eye in S1 include at least the pupil center, the pupil edge point, and the corneal limbus edge point; Step S2 specifically involves establishing an eye polar coordinate system with the center of the pupil as the pole.
[0048] Key points in ocular anatomy include the pupil center, pupil edge point, and limbal edge point. The pupil center is used to determine the location of the pole in the polar coordinate system, the pupil edge point is used to fit the pupil contour, and the limbal edge point is used to fit the limbal contour.
[0049] Furthermore, the geometric parameters in step S2 include the polar radius and polar angle of the vertex of the lesion detection frame, as well as the overlap ratio of the lesion detection frame with the pupil region and the limbus region.
[0050] The polar angle indicates the direction of the lesion in the eye, such as its location above, below, or to the side. The polar diameter indicates its relative position from the center of the pupil.
[0051] Furthermore, the pupil region and the limbal region are obtained by performing least-squares circle fitting or ellipse fitting on the pupil edge points and the limbal edge points.
[0052] By fitting key points of ocular anatomy, the boundaries of the region can be obtained. Only a small number of key points are needed to determine the range of the pupil region and the limbus region.
[0053] Preferably, after fitting the geometry of the pupil and limbus, the radius parameters of the fitted circle or ellipse are directly calculated to obtain the pupil radius and limbus radius.
[0054] Furthermore, step S3 includes converting the geometric parameters into structured semantic information through a preset interval mapping rule.
[0055] like Figure 3 As shown, this invention establishes a polar coordinate system with the pupil center as the origin and performs geometric calculations on the position of the lesion detection frame to obtain the lesion's orientation information and invasion depth.
[0056] Preferably, when the vertex radius ρ of the lesion detection frame is less than the pupil radius, the structured semantic slot "Invasion Depth" is marked as "Obscures Pupil"; when ρ is greater than the pupil radius and less than the limbal radius, the invasion depth is marked as "Invades Within the Limbal Region". This interval mapping rule can convert geometric calculation results into structured semantic descriptors, such as: {Disease: Pterygium, Laterality: Left Eye, Orientation: Nasal Side, Invasion: Obscures Pupil}.
[0057] Furthermore, the structured semantic information includes disease category, lesion location, invasion depth, and occlusion relationship.
[0058] Disease categories are derived from the disease category results obtained by the S1-step multi-task detection model. Lesion orientation describes the directional position of the lesion within the eye, obtained based on the polar radius and polar angle of the lesion detection frame vertex. Invasion depth is determined by the relationship between the polar radius of the lesion detection frame vertex and the pupil radius and limbal radius, describing the degree of outward expansion of the lesion relative to the pupil center. Occlusion relationship is determined based on the overlap, intersection, and union ratio (IoU) between the lesion detection frame and the pupillary region or limbal region.
[0059] Furthermore, the method also includes performing a consistency check on the generated diagnostic report. The consistency check includes extracting key entity information from the anterior segment disease diagnostic graphic report; inversely mapping the key entity information into virtual geometric coordinates; comparing the virtual geometric coordinates with the lesion detection box output in step S1, and triggering report regeneration if the overlap is lower than a threshold.
[0060] Preferably, the threshold is set between 0.4 and 0.6.
[0061] Key entity information, such as lesion location descriptions and related ocular anatomical structures, is converted into virtual geometric coordinates. These virtual geometric coordinates are then compared with the lesion detection bounding box obtained in step S1. If the overlap between the two is below a set threshold, the text description is considered inconsistent with the detection result, triggering a report regeneration process. Figure 4 As shown, the report generation process includes three stages: prompt word construction, text generation and consistency verification, and report output. First, prompt words are constructed based on structured medical description information and an ophthalmology medical knowledge base. Then, the prompt words are input into a large language model to generate preliminary diagnostic text. Next, the generated text is compared with the visual detection results. When the detection results match the text description, the final report is generated. Finally, the diagnostic text is combined with the original image with lesion annotations to output an AI-assisted diagnostic graphic report.
Claims
1. An intelligent screening method, system and device for multiple diseases of the anterior segment of the eye, characterized in that, Includes the following steps: S1. Acquire an anterior segment slit lamp image and input the image into a pre-trained multi-task detection model to detect key anatomical points and lesion areas of the eye, and obtain lesion detection boxes, disease categories and coordinates of key anatomical points of the eye. S2. Based on the coordinates of the key points of the eye anatomy, establish an eye polar coordinate system and perform geometric calculations on the lesion detection frame to obtain the geometric parameters between the lesion and the eye anatomy structure. S3. Generate structured semantic description information based on the geometric parameters; S4. Input the structured semantic description information into the large language model to generate a graphic report on the diagnosis of anterior segment diseases.
2. The method according to claim 1, characterized in that: The multi-task detection model includes a backbone network, a feature fusion network, and a detection head; The multi-task detection model introduces a C2PSA attention mechanism into the backbone network, disables the c3k parameter in the shallow structure of the backbone network, and uses an asymmetric multi-task joint loss function for training, in which the regression loss weight of anatomical key points is higher than the regression loss weight of lesion detection boxes.
3. The method according to claim 1, characterized in that: The key anatomical points of the eye in S1 include at least the center of the pupil, the edge of the pupil, and the edge of the corneal limbus; Step S2 specifically involves establishing an eye polar coordinate system with the center of the pupil as the pole.
4. The method according to claim 3, characterized in that: The geometric parameters in step S2 include the polar diameter and polar angle of the vertex of the lesion detection frame, as well as the overlap ratio of the lesion detection frame with the pupil region and the limbus region.
5. The method according to claim 4, characterized in that: The pupil region and limbal region are obtained by performing least-squares circle fitting or ellipse fitting on the pupil edge points and limbal edge points.
6. The method according to claim 5, characterized in that: Step S3 includes converting the geometric parameters into structured semantic information using a preset interval mapping rule.
7. The method according to claim 6, characterized in that: The structured semantic information includes disease category, lesion location, invasion depth, and occlusion relationship.
8. The method according to any one of claims 1-7, characterized in that: The method further includes performing a consistency check on the generated diagnostic report. The consistency check includes extracting key entity information from the anterior segment disease diagnostic image and text report; inversely mapping the key entity information into virtual geometric coordinates; comparing the virtual geometric coordinates with the lesion detection box output in step S1, and triggering report regeneration if the overlap is lower than a threshold.
9. An AI intelligent screening system for multiple diseases of the anterior segment of the eye for generating a fusion report, characterized in that, include: The detection module is used to acquire anterior segment slit lamp images and input the images into a pre-trained multi-task detection model to detect ocular anatomical structures and lesion areas, and obtain lesion detection boxes, disease categories and coordinates of key ocular anatomical points; The geometric calculation module is used to establish an ocular polar coordinate system based on the coordinates of the key anatomical points of the eye, and to perform geometric calculations on the lesion detection box to obtain the geometric parameters between the lesion and the ocular anatomical structure. A semantic transformation module is used to generate structured semantic description information based on the geometric parameters; The report generation module is used to input the structured semantic description information into a large language model to generate a graphic report on the diagnosis of anterior segment diseases.
10. A computer device, comprising: include: processor; A memory storing a computer program; when executed by the processor, the computer program causes the processor to perform the method as described in any one of claims 1 to 8.