Adaptive resampling guidance method and system based on information gain driving

Abstract

The present application relates to the technical field of ophthalmic medical image processing and intelligent analysis, and in particular to an adaptive supplementary sampling guidance method and system based on information gain driving, comprising: collecting an initial fundus image, constructing a region weight model and a supplementary sampling direction mapping model, obtaining a fundus region coverage map and a screening task region weight map; calculating a screening completeness score; determining whether the screening completeness score is greater than or equal to a second completeness threshold and less than a first completeness threshold; in the case that the screening completeness score is greater than or equal to the second completeness threshold and less than the first completeness threshold, using the supplementary sampling direction mapping model to obtain a candidate supplementary sampling direction set; calculating the information gain value and the eye movement cost value of the candidate supplementary sampling direction set, and obtaining an optimal supplementary sampling direction. Through the above technical solution, the adaptive supplementary sampling decision mechanism driven by information gain is introduced to realize the supplementary sampling of the most valuable screening region within a limited number of times, thereby meeting the real screening demand.

Description

Technical Field

[0001] This invention relates to the field of ophthalmic medical image processing and intelligent analysis technology, specifically to an adaptive supplementary acquisition guidance method and system based on information gain. Background Technology

[0002] Diseases such as diabetic retinopathy, high myopia-related fundus diseases, macular degeneration, glaucoma risk, and hypertension-related fundus changes all rely on fundus images for screening, follow-up, and risk assessment. The clinical value of fundus images depends not only on image clarity but also on whether the screening covers the key areas required for the current disease assessment. In primary care screening scenarios, factors such as patient cooperation, operator experience, eye gaze direction, eyelid occlusion, reflection, pupil size, and device stability can all affect the acquisition results. Even if an image appears relatively clear, it may fail to meet early screening requirements because it does not cover the macular region, optic disc region, superior and inferior vascular arches, or other high-risk areas. In primary care fundus disease early screening, the key issues are whether the screening captures enough images, accurately, and in the key areas. If insufficient images are captured, which direction should the system prioritize for additional images? If the patient is fatigued or has experienced multiple failed acquisitions, how should the system balance screening value and eye comfort?

[0003] Current fundus image acquisition systems typically interpret image quality as indicators such as sharpness, exposure, or focus, focusing on the sharpness, exposure, focus, or lesion identification results of a single image, but lacking an evaluation of the completeness of whether the screening covers key regions. However, for early fundus screening, the absence of a certain region has a greater impact on clinical risk assessment than a slightly blurred single image. Furthermore, existing multi-view fundus acquisition protocols often employ fixed fixation points or fixed directional sequences, lacking a mechanism to dynamically adjust the next acquisition direction based on the results of already acquired images. A fixed sequence may lead to unnecessary repeated acquisitions and may also result in the omission of more important regions in the current disease task. Simultaneously, existing supplementary acquisition strategies rely on a simple logic of filling in missing regions, i.e., if a certain direction is missing, the examinee is prompted to look in that direction. This approach does not consider the different values ​​of different regions in different disease screening tasks. For example, the weight of key regions differs in diabetic retinopathy screening, high myopia-related fundus disease screening, macular degeneration screening, and glaucoma risk screening. Furthermore, existing supplementary sampling strategies lack constraints on eye movement comfort. If the system only aims to cover more areas, it may frequently guide subjects to make large eye movements, leading to eye fatigue, decreased cooperation, and an increase in sampling failures, thus reducing the efficiency of primary care screening. Most existing fundus image processing methods focus on generating the stitched image itself, lacking quality gating mechanisms to determine the effectiveness of the current screening, whether supplementary sampling is needed, whether it should be included in AI risk analysis, and whether physician review is required.

[0004] Therefore, there is an urgent need for an adaptive supplementary sampling guidance method and system based on information gain to solve the problem that existing technologies cannot meet screening requirements due to the lack of judgment of key screening areas. Summary of the Invention

[0005] The purpose of this invention is to provide an adaptive supplementary sampling guidance method and system based on information gain to solve the technical problem in the prior art that the screening requirements cannot be met due to the lack of judgment of key screening areas.

[0006] To achieve the above objectives, embodiments of the present invention provide an adaptive supplementary sampling guidance method based on information gain driving, comprising: Acquire initial fundus images, construct a region weight model and a supplementary acquisition direction mapping model, and obtain a fundus region coverage map and a screening task region weight map; Based on the fundus region coverage map and the screening task region weight map, the screening completeness score is calculated; Determine whether the screening completeness score is greater than or equal to the second completeness threshold and less than the first completeness threshold; If the screening integrity score is greater than or equal to the second integrity threshold and less than the first integrity threshold, a candidate set of supplementary sampling directions is obtained based on the current fundus region coverage map using a supplementary sampling direction mapping model. Calculate the information gain value and eye-tracking cost of the candidate supplementary sampling direction set to obtain the optimal supplementary sampling direction.

[0007] Optionally, initial fundus images are acquired, a region weight model and a supplementary acquisition direction mapping model are constructed, and fundus region coverage maps and screening task region weight maps are obtained, including: Acquire initial fundus images and extract the effective imaging area; Based on the fundus anatomy and screening requirements, a fundus region coverage map is constructed based on the effective imaging area. Construct a regional weight model and a supplementary mining direction mapping model; The regional weight map of the screening task is obtained based on the aforementioned regional weight model.

[0008] Optionally, constructing a regional weight model includes: Establish a set of screening tasks; Based on the clinically critical areas for different fundus screening needs, corresponding weights are established for the preset fundus areas; Based on the set of screening tasks, obtain the regional weights corresponding to different screening tasks and construct a task weight graph; Based on the task weight graph, a regional weight model is constructed.

[0009] Optionally, constructing the supplementary sampling direction mapping model includes: Based on the preset gaze directions supported by the device gaze guidance module, a set of preset supplementary sampling directions is established; Based on the correspondence between each gaze direction and the fundus imaging area, a mapping relationship between the supplementary imaging direction and the expected area is established; A supplementary sampling direction mapping model is constructed based on the aforementioned mapping relationship.

[0010] Optionally, the screening completeness score is calculated based on the fundus region coverage map and the screening task region weight map, including: The coverage state value is calculated using formula (1). (1) in, Let r be the coverage status value of the r-th region. For the r-th preset fundus region, This represents the effective coverage area after merging the currently acquired and qualified images. Indicates area; The screening completeness score is calculated using formula (2). (2) in, To score the completeness of the screening, Let r be the region weight of the r-th fundus region in the current screening task. The overall image quality score for the r-th region.

[0011] Optionally, if the screening integrity score is greater than or equal to the second integrity threshold and less than the first integrity threshold, based on the current fundus region coverage map, a supplementary sampling direction mapping model is used to obtain a candidate supplementary sampling direction set, including: Based on the current fundus region coverage map, obtain the uncovered areas and low-quality covered areas; A supplementary sampling direction mapping model is used to obtain the fundus coverage area that can be covered by each preset supplementary sampling direction; The direction in the preset supplementary imaging direction that can cover the uncovered area or the low-quality coverage area of ​​the fundus is selected as the initial candidate direction; The initial candidate directions are filtered to obtain a set of candidate supplementary sampling directions.

[0012] Optionally, calculating the information gain value and eye-tracking cost value of the candidate supplementary acquisition direction set to obtain the optimal supplementary acquisition direction includes: The information gain value of each candidate sampling direction in the candidate sampling direction set is calculated using formula (3). (3) in, This is the information gain value. To estimate the area of ​​new coverage, For regional weights, Contribution to coverage of key anatomical structures. To improve the screening completeness score, To improve the overall image quality score, These are adjustable weighting coefficients.

[0013] Optionally, calculating the information gain value and eye-tracking cost value of the candidate supplementary acquisition direction set to obtain the optimal supplementary acquisition direction includes: The eye-tracking cost of each candidate replenishment direction in the candidate replenishment direction set is calculated using formula (4). (4) in, For the value of eye movement, For offset angle, This represents the number of consecutive fixations that have been completed so far. The penalty is for extreme gaze direction. This is a penalty item for failed historical data collection. This is a penalty item for the examinee's cooperation. , , , , These are adjustable weighting coefficients.

[0014] Optionally, calculating the information gain value and eye-tracking cost value of the candidate supplementary acquisition direction set to obtain the optimal supplementary acquisition direction includes: The reward function is constructed using formula (5). (5) in, For the reward function, This is the information gain value. To improve the screening completeness score, To improve the overall image quality score, For the value of eye movement, A penalty for repeatedly collecting low-value regions. To include penalties for failed data collection, To achieve the completion reward after reaching the preset screening completeness threshold, These are the weighting coefficients; Based on the reward function, a reinforcement learning strategy is employed to obtain the optimal replenishment direction.

[0015] On the other hand, the present invention provides an adaptive supplementary acquisition guidance system based on information gain drive, the system comprising: The image acquisition and model building module is used to acquire initial fundus images, build a region weight model and a supplementary acquisition direction mapping model, and obtain a fundus region coverage map and a screening task region weight map. The screening completeness scoring module is used to calculate the screening completeness score based on the fundus region coverage map and the screening task region weight map; The screening completeness score determination module is used to determine whether the screening completeness score is greater than or equal to the second completeness threshold and less than the first completeness threshold; The candidate supplementary sampling direction set construction module is used to obtain the candidate supplementary sampling direction set based on the current fundus region coverage map and using a supplementary sampling direction mapping model when the screening integrity score is greater than or equal to the second integrity threshold and less than the first integrity threshold. The optimal supplementary sampling direction generation module is used to calculate the information gain value and eye-tracking cost value of the candidate supplementary sampling direction set to obtain the optimal supplementary sampling direction; The processor is connected to the image acquisition and model building module, the screening completeness scoring module, the screening completeness scoring judgment module, the candidate supplementary sampling direction set construction module, and the optimal supplementary sampling direction generation module, and the processor is configured to execute any of the methods described above.

[0016] The beneficial effects of this invention are: This invention constructs a regional weight model to obtain a regional weight map for screening tasks, and adjusts the task weights according to different screening tasks. This allows the same fundus acquisition device to adapt to early screening tasks with different fundus screening needs, reducing real-time computing overhead. By calculating and judging the screening completeness score, it clarifies whether the current screening can be used for subsequent AI analysis, reducing the risk of misjudgment caused by directly performing AI risk stratification on low-quality images or images missing key areas. Furthermore, by calculating the information gain value and eye movement cost value, the optimal re-import direction is obtained, improving the screening efficiency with a limited number of acquisitions, reducing large, continuous, and disordered eye movements of the examinee, and improving the acquisition success rate for the elderly, diabetic examinees, and screening populations at the grassroots level.

[0017] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of an adaptive supplementary sampling guidance method based on information gain drive according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating a specific method according to an embodiment of the present invention for acquiring initial fundus images, constructing a region weight model and a supplementary acquisition direction mapping model, and obtaining a fundus region coverage map and a screening task region weight map; Figure 3 This is a block diagram of an adaptive supplementary sampling guidance system based on information gain drive according to an embodiment of the present invention. Detailed Implementation

[0019] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0020] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0021] like Figure 1 The diagram shows a flowchart of an adaptive supplementary sampling guidance method based on information gain driving according to an embodiment of the present invention. Figure 1 The supplementary sampling guidance method may include the following steps: In step S10, initial fundus images are acquired, a regional weight model and a supplementary acquisition direction mapping model are constructed, and a fundus region coverage map and a screening task region weight map are obtained. In step S11, the screening completeness score is calculated based on the fundus region coverage map and the screening task region weight map; In step S12, it is determined whether the screening completeness score is greater than or equal to the second completeness threshold and less than the first completeness threshold; In step S13, if the screening integrity score is greater than or equal to the second integrity threshold and less than the first integrity threshold, a candidate supplementary sampling direction set is obtained based on the current fundus region coverage map using a supplementary sampling direction mapping model. In step S14, the information gain value and eye-tracking cost of the candidate sampling direction set are calculated to obtain the optimal sampling direction.

[0022] In such Figure 1In the information gain-driven adaptive supplementary acquisition guidance method shown, step S10 is used to acquire initial fundus images, construct a region weight model and a supplementary acquisition direction mapping model, and obtain a fundus region coverage map and a screening task region weight map. In this embodiment, the acquired initial fundus images may include images in the central gaze direction and multiple fundus images under several preset main gaze directions.

[0023] In this embodiment, the specific method for step S10—acquiring initial fundus images, constructing a region weight model and a supplementary acquisition direction mapping model, and obtaining a fundus region coverage map and a screening task region weight map—can be of various forms known to those skilled in the art. In this embodiment, step S10 can be as follows: Figure 2 The steps are shown. In this Figure 2 In this context, step S10 may include: In step S20, an initial fundus image is acquired, and the effective imaging area is extracted; In step S21, a fundus region coverage map is constructed based on the effective imaging area, according to the fundus anatomy and screening task requirements. In step S22, a regional weight model and a supplementary mining direction mapping model are constructed; In step S23, the regional weight map of the screening task is obtained based on the regional weight model.

[0024] In such Figure 2 In the method shown, step S20 is used to acquire an initial fundus image and extract the effective imaging region. In this embodiment, the effective imaging region is extracted from the acquired initial fundus image, removing black areas at the image edges, obvious reflective areas, severely occluded areas, exposure failure areas, and invalid background areas.

[0025] Step S21 is used to construct a fundus region coverage map based on the fundus anatomy and screening task requirements, using the effective imaging area. In this embodiment, the specific method for constructing the fundus region coverage map based on the effective imaging area in step S21 can be various forms known to those skilled in the art, such as constructing a regional fundus coverage map, using preset regions such as the macula, optic disc, vascular arch, nasal region, temporal region, and mid-peripheral region as units to mark the coverage and quality status. In this embodiment, an approximate coordinate system can be established based on the image center, optic disc position, and vascular arch direction. By mapping the regions in the current image that are effectively imaged and meet the quality requirements to the preset fundus region division map, the coverage status of each region is marked. The coverage status of each region can include three categories: uncovered, low-quality coverage, and effective coverage. Uncovered indicates that the region does not appear in the currently acquired image; low-quality coverage indicates that although the region appears in the image, it suffers from problems such as blurriness, reflection, occlusion, underexposure, motion artifacts, or unrecognizable vascular structures; effective coverage indicates that the region has been clearly acquired and can be used for subsequent screening and judgment. If the current image is detected to have full-image glare, large-area eyelid occlusion, severe exposure failure, severe defocus, or image sensor malfunction, it can be directly determined as an invalid image and will not be included in the coverage map update. If only a portion of the image has glare or occlusion, the affected area can be marked as low-quality coverage or uncovered, while the remaining clear areas continue to be included in the coverage map. Subsequently, the coverage results of multiple acquired fundus images are merged to form the current screening coverage map. This coverage map can be continuously updated after each acquisition to determine currently covered areas, missing areas, low-quality areas, and areas that need to be prioritized for further acquisition.

[0026] In another embodiment of the present invention, the specific method for constructing the fundus region coverage map in step S21 can also be: recording the coverage status, image quality score, and region weight in units of pixels or small grid cells, thereby further improving the accuracy of missing region identification and supplementary sampling direction calculation. For low-computing-power devices, a region-level coverage map can be used; for high-computing-power devices or cloud-based collaborative devices, a pixel-level coverage map or a combination of region-level and pixel-level coverage maps can be used.

[0027] Because existing fixed acquisition methods cannot adjust acquisition priorities according to different screening tasks and cannot predict the fundus coverage corresponding to different gaze directions, a regional weight model and a supplementary acquisition direction mapping model are constructed in step S22. In this embodiment, the specific method for constructing the regional weight model can be as follows: First, a set of screening tasks is established, which may include screening for diabetic retinopathy fundus image integrity, macular fundus image integrity, glaucoma risk screening, high myopia-related fundus image integrity, and hypertensive fundus changes. Second, based on the clinically critical areas required for different fundus screenings, corresponding weights are assigned to preset fundus regions. Further, the regional weights corresponding to different screening tasks are saved as a task weight map. Based on this task weight map, a regional weight model is constructed.

[0028] In this embodiment, the regional weights can be set and adjusted based on ophthalmologist experience, clinical guidelines focusing on specific regions, screening task requirements, or subsequent prototype validation data. These regional weights can be stored locally on the device as preset parameter sets or updated via the backend system. Different groups of examinees can access different parameter sets, such as parameter sets for screening high myopia in adolescents, screening parameter sets for middle-aged and elderly diabetic examinees, and glaucoma risk screening parameter sets.

[0029] The specific method for constructing the supplementary acquisition direction mapping model can be as follows: First, establish a preset direction set based on the gaze directions supported by the device's gaze guidance module. When the acquisition device uses a nine-grid gaze guidance module, the preset direction set can include nine directions: center, top, bottom, left, right, upper left, upper right, lower left, and lower right. Then, based on the correspondence between each gaze direction and the fundus imaging area, establish a mapping relationship between the supplementary acquisition direction and the expected coverage area. Further, save this mapping relationship as a direction coverage mapping table for subsequent determination of the fundus areas that may be newly covered by different candidate supplementary acquisition directions.

[0030] Step S23 is used to obtain a regional weight map for the screening task based on a regional weight model. The regional weight map for the screening task is used to characterize the importance of different fundus regions in a specific screening task. In this embodiment, the regional weight parameters are adjusted based on medical knowledge, expert experience, guideline-recommended regions, and historical screening data. For example, in the diabetic retinopathy fundus image integrity screening task, the macular region, peripapillary region, superior and inferior vascular arches, posterior pole vascularized areas, and high-incidence areas such as hemorrhage, exudation, microaneurysms, and cotton wool spots can be set as high-weight regions; in the high myopia-related fundus image integrity screening task, the posterior pole, macular region, peripapillary region, and peripheral degeneration or hole risk areas can be set as high-weight regions; in the macular fundus image integrity screening task, the fovea and its surrounding area can be set as the highest-weight region, and areas related to continuous macular coverage can be set as high-weight regions; in the glaucoma risk screening task, the optic disc region, cup-to-disc ratio related regions, optic nerve head periapical region, and nerve fiber layer related regions can be set as high-weight regions. When a user or device selects a screening task, the regional weight model is used to obtain the task regional weight map of the current screening task.

[0031] Step S10 involves identifying effective regions in the acquired fundus images and constructing a fundus coverage map based on the identification results. This indicates whether each fundus region has been effectively acquired during the screening process and whether the image quality of the corresponding region meets the screening requirements. This solves the problem that existing technologies cannot standardize the determination of which regions have been effectively acquired and which regions are still missing or have low-quality coverage. By pre-establishing a region weight model to represent the importance of different fundus regions in different screening tasks, this solves the problem that existing fixed acquisition methods cannot adjust the acquisition priority according to different screening tasks. By pre-establishing a supplementary acquisition direction mapping model to represent the fundus regions that are expected to be covered by different gaze directions, this solves the problem that existing fixed acquisition methods cannot predict the fundus coverage range corresponding to different gaze directions.

[0032] Since existing fundus screening lacks quantitative standards to assess the image acquisition completeness of this screening task, a screening completeness score is calculated in step S11 based on the fundus region coverage map and the screening task region weight map. Specifically, in this embodiment, the current fundus region coverage map is first read to determine the coverage status of each fundus region. For effective coverage areas, a higher coverage status value is assigned; for low-quality coverage areas, a lower coverage status value is assigned; and for uncovered areas, a zero coverage status value is assigned. Subsequently, the region weight map corresponding to the current screening task is retrieved, and the coverage status values ​​of each region, the comprehensive image quality score, and the region weight are weighted and calculated to obtain the screening completeness score. The specific method for assigning the coverage status value can be any form known to those skilled in the art, such as setting the coverage status value to 0 for uncovered areas; setting the coverage status value to a value between 0 and 1 for low-quality coverage areas; and setting the coverage status value to 1 for effective coverage areas. In this embodiment, specifically, the coverage status value is first calculated according to formula (1): (1) in, Let r be the coverage status value of the r-th region. For the r-th preset fundus region, This represents the effective coverage area after merging the currently acquired and qualified images. Indicates area.

[0033] Then, the overall image quality score is calculated according to the following formula: ; in, The overall image quality score is given. For clarity, For appropriate exposure, The percentage of reflective area. To determine the percentage of the area to be covered, For the effective area proportion, For the visibility of blood vessels, These are the weighting coefficients.

[0034] Furthermore, the screening completeness score is calculated using formula (2): (2) in, To score the completeness of the screening, Let r be the region weight of the r-th fundus region in the current screening task. The overall image quality score for the r-th region.

[0035] Step S11 calculates the screening completeness score based on the current fundus region coverage map and the current screening task weight map to evaluate whether the current fundus screening covers the key areas required by the current screening task, solves the lack of quantitative standards in the existing fundus screening, and assesses the image acquisition completeness of the current screening task and whether it meets the prerequisites for entering AI risk analysis.

[0036] Step S12 is used to determine whether the screening completeness score is greater than or equal to the second completeness threshold and less than the first completeness threshold. In this embodiment, the range of the first completeness threshold can be... The preferred value is 0.85; the range of the second integrity threshold can be... The optimal value is 0.60. The completeness threshold can be adjusted based on the screening task, the imaging field of view of the equipment, the subject population, and the actual screening standards. When the screening completeness score reaches the first completeness threshold, the screening is considered basically effective and can proceed to the AI ​​risk analysis process. When the screening completeness score is between the first and second completeness thresholds, the screening is considered partially effective, but further sampling is recommended. When the screening completeness score is below the second completeness threshold, the screening quality is considered insufficient, and re-sampling or manual review is required.

[0037] Because existing technologies rely on manual experience for supplementary imaging directions, the system cannot automatically generate and filter these directions. Therefore, in step S13, when continued supplementary imaging is recommended, a candidate supplementary imaging direction mapping model is used based on the current fundus region coverage map to obtain a set of candidate supplementary imaging directions. In this embodiment, when the acquisition device uses a nine-grid gaze guidance module, the preset supplementary imaging directions include nine directions: center, top, bottom, left, right, upper left, upper right, lower left, and lower right. First, the current fundus region coverage map is read to identify uncovered areas and low-quality coverage areas. Then, the supplementary imaging direction mapping model is used to determine the fundus region that each preset supplementary imaging direction can cover after this supplementary imaging. Further, the directions among the preset supplementary imaging directions whose covered fundus regions can encompass uncovered or low-quality coverage areas are used as initial candidate directions. Further, the initial candidate directions are filtered to obtain a candidate supplementary imaging direction set. Specifically, the method for screening initial candidate directions can be as follows: for directions that have failed to acquire images multiple times consecutively, their priority can be reduced based on the number of failures; for extreme fixation directions, if the current subject has completed multiple fixations consecutively, or the device detects that the subject has limited eye movement, strabismus, significant eyelid occlusion, or inability to maintain stable fixation, the priority of that direction can be reduced or it can be removed. For example, in a diabetic retinopathy fundus image integrity screening task, if the current coverage map shows insufficient coverage in both the temporal region and the inferior vascular arch region, the temporal re-acquisition direction and the inferior re-acquisition direction can be included in the candidate re-acquisition direction set simultaneously. If a blurred image of the central macular region is detected, the central re-acquisition direction can be included in the candidate re-acquisition direction set. If a certain re-acquisition direction has failed to acquire images multiple times consecutively, its priority can be reduced within a preset number of attempts; if the direction corresponds to a primary critical region, the direction can be retained but its consecutive re-acquisition frequency reduced, thereby balancing screening value and acquisition success rate.

[0038] In another embodiment of the present invention, the candidate supplementary sampling direction is not limited to the nine-grid gaze direction. If the gaze guidance display module supports continuous gaze guidance, an interpolated direction can be generated based on preset directions such as center, top, bottom, left, right, upper left, upper right, lower left, and lower right. For example, a middle direction can be generated between the top and upper right, or between the bottom and temporal sides, to improve the supplementary sampling accuracy of edge-deficient areas or locally low-quality areas.

[0039] Step S14 is used to calculate the information gain value and eye-tracking cost of the candidate supplementary sampling direction set to obtain the optimal supplementary sampling direction. The specific method for calculating the information gain value can be to calculate the information gain value for each candidate direction in the candidate supplementary sampling direction set according to formula (3): (3) in, The information gain value for the candidate sampling direction. The expected increase in coverage area for the candidate supplementary retrieval direction, in this embodiment, can be defined as the ratio of the expected increase in effective coverage area for the candidate supplementary retrieval direction to the total area of ​​the preset fundus target region. Assigning regional weights to newly covered areas in the current screening task. This is to determine the contribution of this candidate re-extraction direction to the coverage of key anatomical structures. This represents the expected increase in screening completeness score after supplementary sampling for this candidate direction. This represents the expected improvement in overall image quality score after re-imaging in this candidate direction. These are adjustable weighting coefficients.

[0040] In this embodiment, Discrete values ​​can be used: if the direction is expected to completely cover key anatomical structures such as the macula, optic disc, major vascular arches, or arteriovenous crossings, Take 1; if this direction is expected to partially cover the aforementioned key anatomical structures, Take 0.5; if this direction is not expected to cover key anatomical structures, Take 0.

[0041] In another embodiment of the invention, Alternatively, values ​​can be continuously selected based on the coverage ratio of key anatomical structures, with a range of values ​​being [missing information]. The definition of key anatomical structures may differ depending on the screening task. For example, in a diabetic retinopathy fundus image integrity screening task, the macula, the peripapillary region, and the superior and inferior vascular arches can be considered key anatomical structures; in a glaucoma risk screening task, the optic disc and cup-to-disc ratio related regions can be considered key anatomical structures; and in a macular fundus image integrity screening task, the macula and its surrounding region can be considered key anatomical structures.

[0042] In this embodiment, The non-negative weighting coefficients can be normalized to be set as follows: It can also be set to a value range of . Non-negative coefficients. For example, for a diabetic retinopathy fundus image integrity screening task, the coefficients related to the weight of the screening task area, the contribution of key anatomical structures, and the improvement of screening integrity can be appropriately increased; for subjects with poor image quality, the contribution of the overall image quality score to information gain can be appropriately increased.

[0043] Simultaneously, a preset information gain lower limit is set to determine whether further sampling is needed. In this embodiment, its value range can be set to... The preferred value is 0.10. When the expected improvement in screening completeness or overall information gain from a candidate supplementary sampling direction is lower than this lower limit, it can be determined that continuing supplementary sampling is of low benefit, and the automatic supplementary sampling process is terminated or transferred to the manual review process. The lower limit of information gain can be adjusted according to the device's imaging field of view, the current screening task, the subject's cooperation, and the maximum allowed number of samplings.

[0044] Subsequently, the specific method for calculating the eye-tracking cost can be to calculate the eye-tracking cost for each candidate direction in the candidate supplementary sampling direction set according to formula (4): (4) in, The value of eye-tracking for candidate supplementary sampling. The offset angle between the current gaze point and the candidate target gaze point. This represents the number of consecutive fixations that have been completed so far. This is a penalty term for the extreme gaze direction of the candidate retrieval direction. This is a penalty item for historical acquisition failures in the candidate acquisition direction or adjacent directions. This is a penalty item for the examinee's cooperation. , , , , These are adjustable weighting coefficients.

[0045] In this embodiment, The data can be quantified based on the subject's historical data acquisition success rate, current gaze stability, blink frequency, eyelid occlusion frequency, number of consecutive failures, and the operator's input level of cooperation. The value range can be set to... The less cooperative the examinee is, the more likely they are to be affected. The higher the value, the better the subject's cooperation and the more stable their gaze. The value is relatively low. If the subject blinks, deviates from the fixation point, or fails to collect data multiple times in a certain direction, the corresponding candidate re-collection direction will be considered lower. It can automatically increase, thereby reducing the priority of the candidate supplementary sampling direction in subsequent supplementary sampling decisions. If the subject has a high success rate in sampling in adjacent directions, the adjacent supplementary sampling direction with lower eye-tracking cost and easier for the subject to complete can be selected first.

[0046] In this embodiment, , , , , These are non-negative weighting coefficients, which can be normalized or set to a range of values. The non-negative coefficient. It can be calculated based on the geometric offset angle between the current gaze point and the target gaze point, or it can be discretized and estimated based on the adjacency relationship between the nine grid directions. A higher value can be taken when the target direction is in an extreme direction such as upper left, upper right, lower left, or lower right. It can increase with the number of historical failures.

[0047] Furthermore, the optimal re-scanning direction is obtained based on the information gain value and eye-tracking cost. The specific method for obtaining the optimal re-scanning direction can take many forms known to those skilled in the art. In one embodiment of the present invention, the optimal re-scanning direction is determined using a weighted scoring strategy according to the following formula: ; in, The overall score for candidate supplementary sampling directions. This represents the information gain value for the candidate sampling direction. This provides the eye-tracking value for the candidate supplementary sampling direction. and These are non-negative weighting coefficients, representing the weighting coefficients for information gain and eye movement comfort, respectively. They can be adjusted based on the screening task, the age of the examinee, the examinee's cooperation level, and the equipment scenario.

[0048] In another embodiment of the present invention, the specific method for obtaining the optimal supplementary sampling direction can also be to use the Pareto optimal strategy for direction selection, that is, to select a set of candidate directions from the candidate direction set that have no other directions and have higher information gain and lower eye movement cost, and then determine the optimal supplementary sampling direction based on the current screening task priority, subject cooperation, remaining allowable sampling times and key area missingness.

[0049] In another embodiment of the present invention, a reinforcement learning strategy is employed to determine the optimal re-sampling direction. Specifically: First, the current fundus region coverage map, missing regions, low-quality regions, current fixation point, number of acquisitions, overall image quality score, screening completeness score, current screening task, number of historical acquisition failures, and subject cooperation parameters are taken as states, and candidate re-sampling directions are taken as actions. The reward function of the reinforcement learning strategy is constructed using formula (5) to learn the optimal re-sampling strategy: (5) in, The reward value obtained after executing the candidate replenishment direction. This is the information gain value. To improve the screening completeness score, To improve the overall image quality score, For the value of eye movement, A penalty for repeatedly collecting low-value regions. To include penalties for failed data collection, To achieve the completion reward after reaching the preset screening completeness threshold, The weighting coefficient can be set normally or preset according to the screening task, equipment computing power, and the population of examinees.

[0050] Subsequently, a trained reinforcement learning strategy is employed to output the optimal supplementary sampling direction based on the current screening status. In this embodiment, when the candidate supplementary sampling directions are limited to directions such as center, top, bottom, left, right, upper left, upper right, lower left, and lower right, discrete action decision algorithms such as DQN or Double DQN can be used; when the candidate supplementary sampling directions consist of continuous gaze points or interpolated directions, strategy optimization algorithms such as PPO or Actor-Critic can be used.

[0051] Simultaneously, the device can periodically store screening logs, including comprehensive image quality scores, changes in fundus region coverage, direction of supplementary sampling, success or failure of supplementary sampling, changes in integrity scores, and final screening results. These screening logs can be uploaded to the backend system after the subject authorizes and completes data anonymization, for subsequent offline optimization of model parameters. Updated model parameters can be distributed to the local device via the communication module, enabling periodic iterative optimization without requiring real-time on-site training.

[0052] The acquired direction for supplementary image capture can be displayed to the examinee via the gaze guidance display module, or through voice prompts, a human-computer interaction interface, or a mini-program to guide the examinee to complete the next gaze step. Simultaneously, the device display shows the operator the remaining number of images, a schematic diagram of the current missing area in the fundus, the current screening completeness score, and the next direction for supplementary image capture. For fundus images with insufficient screening completeness, the background screening system automatically marks them for doctor review; for fundus images with sufficient screening completeness, the images and screening results can be sent to the fundus lesion AI segmentation and recognition module or risk stratification module to generate a screening report, doctor review suggestions, and referral recommendations.

[0053] If the screening completeness score remains below the second completeness threshold after multiple re-scans, automatic re-scanning can be stopped and a manual review will be prompted. This re-scanning can be performed 2 to 6 times. If the device detects an image sensor malfunction, lighting malfunction, communication malfunction, or continuous image exposure failure, the current screening process will be terminated and the device will be prompted for inspection. If the examinee actively terminates the screening, or if the device detects that the examinee cannot complete extreme gaze directions, the current screening effectiveness level will be output, and the incomplete areas will be marked for doctor review.

[0054] Step S14 evaluates the effective contribution of each candidate supplementary sampling direction to the screening results by calculating the information gain value of each candidate supplementary sampling direction; and calculates the eye movement cost of each candidate supplementary sampling direction to represent the eye movement burden required to turn from the current gaze direction to the candidate target direction and the risk of sampling failure that the direction may cause, thereby solving the problem that the existing supplementary sampling method only performs mechanical supplementary sampling based on the missing area and cannot determine the optimal supplementary sampling direction.

[0055] On the other hand, embodiments of the present invention also provide an adaptive supplementary sampling guidance system based on information gain driving. The structural block diagram of this system is shown below. Figure 3 As shown. In this Figure 3 The system comprises an image acquisition and model building module, a screening completeness scoring module, a screening completeness scoring judgment module, a candidate supplementary imaging direction set construction module, and an optimal supplementary imaging direction generation module. The image acquisition and model building module acquires initial fundus images, constructs a region weight model and a supplementary imaging direction mapping model, and obtains a fundus region coverage map and a screening task region weight map. The screening completeness scoring module calculates the screening completeness score based on the fundus region coverage map and the screening task region weight map. The screening completeness scoring judgment module determines whether the screening completeness score is greater than or equal to a second completeness threshold and less than a first completeness threshold. The candidate supplementary imaging direction set construction module, when the screening completeness score is greater than or equal to the second completeness threshold and less than the first completeness threshold, obtains a set of candidate supplementary imaging directions based on the current fundus region coverage map and using the supplementary imaging direction mapping model. The optimal supplementary imaging direction generation module calculates the information gain and eye-tracking cost of the candidate supplementary imaging direction set to obtain the optimal supplementary imaging direction.

[0056] Example: Screening for Fundus Image Integrity in Diabetic Retinal Retinopathy Taking the screening of fundus image integrity in diabetic retinopathy as an example, the system first acquires fundus images in the central gaze direction and identifies the coverage of the macula, optic disc, superior and inferior vascular arches, and posterior pole regions. If the system detects insufficient coverage of the temporal region and the inferior vascular arch region in the current image, the system does not directly supplement the image according to the size of the missing area, but instead calculates the information gain of the two candidate directions separately.

[0057] If the temporal region has a large missing area, but its screening task weight is lower than that of the lower vascular arch region; while the lower vascular arch region has a small missing area, but its weight in the DR screening task is high, and supplementary sampling can significantly improve the integrity score of key areas, then the system can prioritize the lower gaze direction as the next supplementary sampling direction.

[0058] If the subject's current gaze point is already close to the downward direction, the eye movement cost of the re-sampling direction is low, and the system further confirms that downward re-sampling is the optimal direction. If the subject has failed to gaze downward multiple times consecutively, the system can reduce the overall score for that direction and select an adjacent direction with slightly lower information gain but easier to complete for re-sampling.

[0059] The beneficial effects of this invention are: The embodiments of this invention construct a fundus region coverage map, a screening task weight map, information gain evaluation, eye movement cost constraint, and adaptive supplementary sampling direction selection. This intelligently determines whether a single fundus screening has completely covered the key areas required for the target screening needs. Within a limited number of sampling attempts, the device prioritizes supplementary sampling of the most valuable areas for screening and areas that are easier for the examinee to cooperate with, thus meeting the real needs of grassroots screening. After reaching the screening completeness threshold, it automatically triggers subsequent artificial intelligence risk stratification, thereby significantly improving screening efficiency and the reliability of risk stratification while reducing invalid sampling.

[0060] This invention, through effective region identification of acquired fundus images and the construction of a fundus coverage map, achieves standardized and quantitative judgment of fundus screening acquisition areas, distinguishing between effective acquisition areas, missing areas, and low-quality coverage areas, thereby improving the consistency and reliability of image acquisition integrity assessment. By constructing a region weight model, the acquisition priority of each region is dynamically adjusted according to different fundus screening tasks, overcoming the limitation of existing fixed acquisition methods that cannot adapt to the needs of various screening tasks. Furthermore, by constructing a supplementary acquisition direction mapping model, the fundus coverage areas corresponding to different gaze directions are predicted, effectively avoiding blind acquisition.

[0061] This invention calculates a screening completeness score to determine whether the fundus coverage area map meets the conditions for subsequent AI analysis, effectively reducing the risk of misjudgment caused by low-quality images or images missing key areas directly entering AI risk stratification. By calculating the information gain value, it prioritizes acquiring the most valuable areas for the current screening task, avoiding redundant acquisition caused by a fixed gaze order, thereby significantly improving screening efficiency within a limited number of acquisitions. Simultaneously, it introduces eye movement cost constraints to reduce large, continuous, and disordered eye movements by the examinee, improving the success rate of fundus image acquisition for the elderly, diabetic examinees, and screening populations at the grassroots level.

[0062] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0063] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0064] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0065] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0066] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0067] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0068] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0069] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0070] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An adaptive supplementary sampling guidance method based on information gain, characterized in that, The method includes: Acquire initial fundus images, construct a region weight model and a supplementary acquisition direction mapping model, and obtain a fundus region coverage map and a screening task region weight map; Based on the fundus region coverage map and the screening task region weight map, the screening completeness score is calculated; Determine whether the screening completeness score is greater than or equal to the second completeness threshold and less than the first completeness threshold; If the screening integrity score is greater than or equal to the second integrity threshold and less than the first integrity threshold, a candidate set of supplementary sampling directions is obtained based on the current fundus region coverage map using a supplementary sampling direction mapping model. Calculate the information gain value and eye-tracking substitution value of the candidate supplementary sampling direction set to obtain the optimal supplementary sampling direction.

2. The method according to claim 1, characterized in that, Acquire initial fundus images, construct a region weight model and a supplementary image direction mapping model, and obtain a fundus region coverage map and a screening task region weight map, including: Acquire initial fundus images and extract the effective imaging area; Based on the fundus anatomy and screening requirements, a fundus region coverage map is constructed based on the effective imaging area. Construct a regional weight model and a supplementary mining direction mapping model; The regional weight map of the screening task is obtained based on the aforementioned regional weight model.

3. The method according to claim 2, characterized in that, Constructing the regional weight model includes: Establish a set of screening tasks; Based on the clinically critical areas for different fundus screening needs, corresponding weights are established for the preset fundus areas; Based on the set of screening tasks, obtain the regional weights corresponding to different screening tasks and construct a task weight graph; Based on the task weight graph, a regional weight model is constructed.

4. The method according to claim 2, characterized in that, The construction of the supplementary mining direction mapping model includes: Based on the preset gaze directions supported by the device gaze guidance module, a set of preset supplementary sampling directions is established; Based on the correspondence between each gaze direction and the fundus imaging area, a mapping relationship between the supplementary imaging direction and the expected area is established; A supplementary sampling direction mapping model is constructed based on the aforementioned mapping relationship.

5. The method according to claim 1, characterized in that, Based on the fundus region coverage map and the screening task region weight map, the screening completeness score is calculated as follows: The coverage state value is calculated using formula (1). ，（1） in, Let r be the coverage status value of the r-th region. For the r-th preset fundus region, This represents the effective coverage area after merging the currently acquired and qualified images. Indicates area; The screening completeness score is calculated using formula (2). ，（2） in, To score the completeness of the screening, Let r be the region weight of the r-th fundus region in the current screening task. The overall image quality score for the r-th region.

6. The method according to claim 1, characterized in that, If the screening integrity score is greater than or equal to the second integrity threshold and less than the first integrity threshold, based on the current fundus region coverage map, a candidate supplementary screening direction mapping model is used to obtain the following: Based on the current fundus region coverage map, obtain the uncovered areas and low-quality covered areas; A supplementary sampling direction mapping model is used to obtain the fundus coverage area that can be covered by each preset supplementary sampling direction; The direction in the preset supplementary imaging direction that can cover the uncovered area or the low-quality coverage area of ​​the fundus is selected as the initial candidate direction; The initial candidate directions are filtered to obtain a set of candidate supplementary sampling directions.

7. The method according to claim 1, characterized in that, Calculating the information gain and eye-tracking cost of the candidate supplementary acquisition direction set to obtain the optimal supplementary acquisition direction includes: The information gain value of each candidate sampling direction in the candidate sampling direction set is calculated using formula (3). ，（3） in, This is the information gain value. To estimate the area of ​​new coverage, For regional weights, Contribution to coverage of key anatomical structures. To improve the screening completeness score, To improve the overall image quality score, These are adjustable weighting coefficients.

8. The method according to claim 1, characterized in that, Calculating the information gain and eye-tracking cost of the candidate supplementary acquisition direction set to obtain the optimal supplementary acquisition direction includes: The eye-tracking cost of each candidate replenishment direction in the candidate replenishment direction set is calculated using formula (4). ，（4） in, For the value of eye movement, For offset angle, This represents the number of consecutive fixations that have been completed so far. The penalty is for extreme gaze direction. This is a penalty item for failed historical data collection. This is a penalty item for the examinee's cooperation. , , , , These are adjustable weighting coefficients.

9. The method according to claim 1, characterized in that, Calculating the information gain and eye-tracking cost of the candidate supplementary acquisition direction set to obtain the optimal supplementary acquisition direction includes: The reward function is constructed using formula (5). ，（5） in, For the reward function, This is the information gain value. To improve the screening completeness score, To improve the overall image quality score, For the value of eye movement, A penalty for repeatedly collecting low-value regions. To include penalties for failed data collection, To achieve the completion reward after reaching the preset screening completeness threshold, These are the weighting coefficients; Based on the aforementioned reward function, a reinforcement learning strategy is employed to obtain the optimal replenishment direction.

10. An adaptive supplementary sampling guidance system based on information gain, characterized in that, The system includes: The image acquisition and model building module is used to acquire initial fundus images, build a region weight model and a supplementary acquisition direction mapping model, and obtain a fundus region coverage map and a screening task region weight map. The screening completeness scoring module is used to calculate the screening completeness score based on the fundus region coverage map and the screening task region weight map; The screening completeness score determination module is used to determine whether the screening completeness score is greater than or equal to the second completeness threshold and less than the first completeness threshold; The candidate supplementary sampling direction set construction module is used to obtain the candidate supplementary sampling direction set based on the current fundus region coverage map and using a supplementary sampling direction mapping model when the screening integrity score is greater than or equal to the second integrity threshold and less than the first integrity threshold. The optimal supplementary sampling direction generation module is used to calculate the information gain value and eye-tracking cost value of the candidate supplementary sampling direction set to obtain the optimal supplementary sampling direction; The processor is connected to the image acquisition and model building module, the screening integrity scoring module, the screening integrity scoring judgment module, the candidate supplementary sampling direction set construction module, and the optimal supplementary sampling direction generation module, and the processor is configured to perform the method as described in any one of claims 1 to 9.

Images