A method and system for feature analysis of ocular images

Abstract

This invention discloses a method and system for feature analysis of eye images, belonging to the field of ophthalmic clinical information processing technology. It includes: extracting key eye features to obtain initial information text; identifying sub-specialty directions of the initial information text to obtain ophthalmic specialty category labels; performing a first knowledge retrieval in a sub-specialty-specific vector knowledge base to obtain first medical knowledge; combining the initial information text and the first medical knowledge to obtain multiple preliminary analysis results; summarizing the initial information text and all preliminary analysis results to obtain a comprehensive reasoning context; performing a second knowledge retrieval in a multi-factor interactive knowledge base to obtain second medical knowledge; and performing comprehensive reasoning on the comprehensive reasoning context and the second medical knowledge to obtain intermediate reasoning conclusions, followed by structured parsing and formatting to obtain a multi-source eye feature comprehensive processing report. This invention effectively improves the accuracy and reliability of complex eye information processing.

Description

Technical Field

[0001] This invention relates to the field of ophthalmic clinical information processing technology, and in particular to a method and system for feature analysis of eye images. Background Technology

[0002] With the increasing aging population and rising prevalence of chronic underlying diseases, the complexity of ocular health issues in clinical practice is growing. Many patients present with multiple coexisting ocular signs, such as lens opacity in the context of diabetic retinopathy, or abnormal corneal changes during glaucoma progression. These multifactorial ocular manifestations not only interact physiologically but also exhibit high complexity in clinical data presentation, posing significant challenges to the understanding and integration of medical information.

[0003] To overcome the static nature of knowledge in existing general-purpose large models (such as OpenAI's ChatGPT series, Google's Med-PaLM 2, and Baidu's "Lingyi Zhihui"), some researchers have proposed a Retrieval Enhancement Generation (RAG) framework. The general architecture and technical process of such systems include: collecting clinical guidelines, technical guidelines, and standard textbooks from authoritative industry institutions such as the National Health Commission and the Chinese Medical Association; segmenting these into semantically complete paragraphs; converting them into vector form using Chinese embedding models (such as BERT-based Chinese embeddings); storing the vectors in vector databases (such as Faiss and Milvus); forming initial query statements from multimodal input information entered by medical personnel or extracted by the system; vectorizing the query statements; calculating cosine similarity across the entire knowledge base; and returning the top-k most relevant medical knowledge fragments; concatenating the original input with the search results; feeding the result into a large language model; prompting it to "answer the question based on the following information"; and generating analytical output supported by external knowledge.

[0004] However, when faced with mixed input of multiple vital signs, this model cannot distinguish the professional categories to which different features belong, resulting in mixed search results, fragmented knowledge points, and a lack of logical connection. The final analysis content may contain contradictions and a shift in focus. Summary of the Invention

[0005] Therefore, it is necessary to provide a feature analysis method for eye images to address the aforementioned technical problems.

[0006] This invention provides a feature analysis method for eye images, including: The system acquires multimodal clinical input information from electronic medical record systems or medical staff, and extracts key eye features from the multimodal clinical input information using medical named entity recognition technology to obtain the initial information text. The initial information text is identified by sub-specialty direction using the set prompt words in the large language model to obtain at least one ophthalmology specialty category label; the first knowledge retrieval is performed in the sub-specialty exclusive vector knowledge base corresponding to each ophthalmology specialty category label to obtain the first medical knowledge. By combining the initial information text and primary medical knowledge through a large language model, independent feature analysis of different professional directions can be achieved, and preliminary analysis results corresponding to each ophthalmology professional category label can be obtained. The initial information text and all preliminary analysis results are summarized by a structured splicing method to obtain a comprehensive reasoning context containing multi-dimensional analysis clues; a second knowledge retrieval is performed in the multi-factor interactive knowledge base based on the comprehensive reasoning context to obtain second medical knowledge describing the potential correlation mechanism between different ocular features; By using a large language model to perform integrated reasoning based on the context of comprehensive reasoning and secondary medical knowledge, cross-domain information fusion and consistent reasoning are achieved to obtain intermediate reasoning conclusions. The intermediate reasoning conclusions are then subjected to structured parsing and formatting to obtain a comprehensive processing report of multi-source eye features.

[0007] Optionally, multimodal clinical input information includes: visual complaint text, clinical record summary, ocular examination indicators, and imaging report summary; Key ocular features include: visual abnormalities, anatomical locations, types of abnormalities, numerical indicators, and imaging abnormalities. Before extracting key ocular features from multimodal clinical input information using medical named entity recognition technology, the process also includes text cleaning and terminology standardization of the multimodal clinical input information, including removing irrelevant characters, standardizing expression methods, and expanding preset medical abbreviations.

[0008] Optionally, the construction of a sub-specialty-specific vector knowledge base and a multi-factor interactive knowledge base specifically includes: Obtain medical literature and clinical reference materials; Medical literature and clinical references are divided into themes according to ophthalmology subspecialties, resulting in multiple independent knowledge sets. The documents in each knowledge set are segmented into multiple paragraph units, and metadata is labeled for each paragraph unit to obtain a subspecialty-specific vector knowledge base that includes medical knowledge fragments of the corresponding specialty. Content describing the potential correlation mechanisms between various ocular features was extracted from medical literature and clinical references to obtain a multi-factor interaction knowledge base.

[0009] Optionally, an initial knowledge retrieval is performed in the subspecialty-specific vector knowledge base corresponding to each ophthalmology specialty category label to obtain the first medical knowledge, specifically including: For each ophthalmology specialty category tag, construct the corresponding query statement; The query statement is input into the sub-specialty-specific vector knowledge base with the corresponding name. Semantic retrieval is performed in parallel using the cosine similarity algorithm, and a preset number of the most relevant medical knowledge fragments are returned to obtain the first medical knowledge.

[0010] Optionally, the preliminary analysis results may include at least: feature matching judgment, confidence level, supporting evidence, exclusion criteria, and a prompt for whether additional information is needed.

[0011] Optionally, second medical knowledge describing the potential association mechanisms between different ocular features includes at least one of the following: co-occurrence patterns of multiple ocular manifestations, potential pathological association mechanisms between manifestations, and clues for the order of examination or intervention.

[0012] Optionally, a large language model is used to perform integrated reasoning on the context of comprehensive reasoning and secondary medical knowledge to complete cross-domain information fusion and consistent reasoning, and to obtain intermediate reasoning conclusions, specifically including: Identify logical contradictions among the preliminary analysis results using a large language model; Based on logical contradictions, the preliminary analysis results are ranked according to their primary and secondary relationships, and the secondary associations corresponding to each preliminary analysis result are deduced. Preliminary analysis results where the ranking results are greater than the set order and the secondary associations are greater than the set threshold are used as intermediate inference conclusions. Intermediate inference conclusions include: the naming of primary and secondary manifestations, the logical relationship between manifestations, the explanation of key points of divergence, and the citation of key evidence on which the inference process depends.

[0013] Optionally, the multi-source ocular feature comprehensive processing report shall include at least the following fields: summary of primary manifestations, description of secondary manifestations, inference of relationships between manifestations, list of supporting evidence, suggestions for differential analysis, and recommendations for observation directions.

[0014] This invention provides a feature analysis system for eye images, comprising: The data acquisition module is used to acquire multimodal clinical input information from the electronic medical record system or entered by medical staff. It extracts key eye features from the multimodal clinical input information through medical named entity recognition technology to obtain the initial information text. The first-level retrieval module is used to identify the sub-specialty direction of the initial information text through the set prompt words in the large language model to obtain at least one ophthalmology specialty category label; and to perform the first knowledge retrieval in the sub-specialty exclusive vector knowledge base corresponding to each ophthalmology specialty category label to obtain the first medical knowledge. The first-level enhancement generation module is used to combine the initial information text and primary medical knowledge through a large language model to achieve independent feature analysis of different professional directions and obtain preliminary analysis results corresponding to each ophthalmology professional category label. The second-level retrieval module is used to summarize the initial information text and all preliminary analysis results through structured splicing to obtain a comprehensive reasoning context containing multi-dimensional analysis clues; based on the comprehensive reasoning context, a second knowledge retrieval is performed in the multi-factor interactive knowledge base to obtain second medical knowledge describing the potential correlation mechanism between different ocular features; The second-level enhancement generation module is used to perform comprehensive reasoning on the comprehensive reasoning context and second medical knowledge through a large language model to complete cross-domain information fusion and consistent reasoning, and obtain intermediate reasoning conclusions; and to perform structured parsing and formatting processing on the intermediate reasoning conclusions to obtain a multi-source eye feature comprehensive processing report.

[0015] The feature analysis method and system for eye images provided in this invention have the following advantages compared with the prior art: This invention first performs standardized preprocessing on the input multimodal clinical information to extract key ocular features; then it activates the first-level RAG mechanism, uses a large language model to identify the potential subspecialty direction to which each feature belongs, and retrieves relevant medical knowledge in the corresponding dedicated knowledge base to generate multiple independent subspecialty dimension analysis results, thereby realizing the classification and processing of different signs and clues.

[0016] Based on this, the invention enters the second-level RAG stage: the preliminary analysis results are integrated with the original input information to construct a comprehensive reasoning context, and a secondary retrieval is performed in the "multi-factor interactive knowledge base" to obtain professional evidence on the potential correlation mechanism between vital signs; finally, with the help of the deep reasoning capability of the large language model, consistency checks, conflict identification and priority ranking of multi-path outputs are performed to generate a unified, coherent and traceable comprehensive analysis conclusion.

[0017] As can be seen, this invention effectively improves the accuracy and reliability of processing complex ocular information through a two-stage knowledge retrieval mechanism of "first diverting, then fusing". Compared with existing technologies, this invention can simulate multidisciplinary analysis logic without relying on the collaborative participation of multiple experts, significantly improving processing efficiency while ensuring output quality. It is particularly suitable for the initial screening and auxiliary understanding of highly complex ocular manifestations in primary healthcare environments, and has good scalability and practical value. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the multi-source eye feature analysis process for an eye image feature analysis method provided in one embodiment; Figure 2 This is a schematic diagram illustrating the data source and professional knowledge base construction for a feature analysis method of eye images provided in one embodiment; Figure 3 This is a schematic diagram illustrating multimodal disease processing of an eye image feature analysis method provided in one embodiment; Figure 4 This is a schematic diagram illustrating the eye feature splitting and preliminary knowledge matching processing of an eye image feature analysis method provided in one embodiment; Figure 5 This is a flowchart of the parallel first-level RAG feature analysis of an eye image feature analysis method provided in one embodiment; Figure 6 This is a schematic diagram illustrating the preliminary analysis results and context fusion reconstruction of a feature analysis method for eye images provided in one embodiment. Figure 7 This is a schematic diagram of multi-level RAG integrated inference analysis of a feature analysis method for eye images provided in one embodiment. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] Traditional ophthalmological information processing relies primarily on individual subspecialists manually interpreting patient complaints, examination records, and other input information. However, when faced with multi-source vital signs data spanning different professional dimensions, individual doctors struggle to comprehensively connect pathological clues from different systems, easily leading to the omission or misinterpretation of key features, which in turn affects the consistency and completeness of subsequent judgments.

[0021] Currently, in high-level medical institutions, a multidisciplinary team (MDT) approach is often used to address such complex situations. This involves multiple subspecialties, such as ophthalmologists, glaucoma specialists, and cataract specialists, jointly reviewing data and exchanging opinions to arrive at a comprehensive analytical conclusion. While this approach can improve information utilization efficiency to some extent, it relies heavily on the coordination and participation of a large number of professional human resources, resulting in long process cycles and high organizational costs, making it difficult to meet the needs of rapid response scenarios. Especially in emergency situations or when the condition is progressing rapidly, waiting for discussion may delay critical intervention opportunities. Furthermore, this process is highly dependent on the subjective experience of experts and lacks standardized information processing paths, leading to discrepancies in the outputs of different teams.

[0022] The above problems are even more pronounced in primary healthcare units, such as township health centers or community hospitals. Due to a lack of professional and technical personnel with multi-disciplinary knowledge, junior medical staff can usually only handle common ocular signs based on a limited cognitive framework. Once they encounter complex manifestations involving multiple subspecialties, they find it difficult to complete an effective analysis independently and often need to be referred to higher-level institutions, further prolonging the information flow chain and reducing service efficiency.

[0023] In recent years, the development of artificial intelligence technology has provided new technical pathways for the automated processing of complex medical information. In particular, Large Language Models (LLMs), with their powerful natural language understanding and contextual reasoning capabilities, have been widely explored for assisting in clinical information integration tasks. However, general-purpose large models (such as ChatGPT and Qwen3) are mainly trained on general corpora and have not fully absorbed the structured knowledge system of specialized fields. When directly applied to ophthalmology-related information processing, they are prone to generating unfounded content, producing the so-called "hallucination" phenomenon, which seriously affects the reliability of the output results.

[0024] Meanwhile, some existing technologies attempt to introduce retrieval-augmented generation (RAG) mechanisms to improve the professionalism of model output by connecting to external knowledge bases. However, most current mainstream solutions adopt a single-hop RAG architecture, which only performs one round of global knowledge matching.

[0025] This invention provides a feature analysis method for eye images, such as... Figure 1 As shown, the method includes: The system acquires multimodal clinical input information from electronic medical record systems or medical staff, and extracts key eye features from the multimodal clinical input information using medical named entity recognition technology to obtain the initial information text.

[0026] The initial information text is analyzed using predefined prompts in a large language model to identify sub-specialty directions, resulting in at least one ophthalmology specialty category label. A first knowledge retrieval is then performed in the sub-specialty-specific vector knowledge base corresponding to each ophthalmology specialty category label to obtain the first piece of medical knowledge.

[0027] By combining the initial information text and primary medical knowledge through a large language model, independent feature analysis of different professional directions can be achieved, and preliminary analysis results corresponding to each ophthalmology professional category label can be obtained.

[0028] The initial information text and all preliminary analysis results are summarized using a structured concatenation method to obtain a comprehensive reasoning context containing multi-dimensional analytical clues. Based on this comprehensive reasoning context, a second knowledge retrieval is performed in a multi-factor interactive knowledge base to obtain second medical knowledge describing the potential correlation mechanisms between different ocular features.

[0029] By integrating the comprehensive reasoning context and secondary medical knowledge through a large language model, cross-domain information fusion and consistent reasoning are achieved to obtain intermediate reasoning conclusions. These intermediate reasoning conclusions are then subjected to structured parsing and formatting to generate a comprehensive report of multi-source ocular features.

[0030] A specific embodiment of the present invention is provided: Step 1: Obtain multimodal clinical input information from the electronic medical record system or entered by medical staff, including patient complaints, eye examination records and image report summaries; clean, denoise, and standardize the terminology of the original text, and extract key eye features through medical named entity recognition technology to generate semantically complete and formatted initial information text.

[0031] Step 2: Input the initial information text into the large language model, guide it to complete the sub-specialty identification task by setting prompt words, and output one or more possible ophthalmology specialty category labels for subsequent knowledge path distribution.

[0032] Step 3: For each identified professional tag, perform an independent knowledge retrieval operation in the corresponding sub-professional vector knowledge base, calculate semantic similarity using an embedding model, and return the Top-k medical knowledge fragments most relevant to the field.

[0033] Step 4: For each specialization, combine the retrieved knowledge fragments with the initial information text, and call the large language model to generate a structured preliminary feature analysis report, which includes: feature matching judgment, confidence level, supporting evidence, exclusion basis, and prompts on whether supplementary information is needed.

[0034] Step 5: Summarize all preliminary feature analysis reports and original information texts, and construct a comprehensive reasoning context containing multi-dimensional analysis clues through structured splicing, as the basic input for the second-level processing.

[0035] Step 6: Using the comprehensive reasoning context as the query statement, perform a second knowledge retrieval in the "Multi-Factor Interactive Knowledge Base" to obtain knowledge fragments about the potential correlation mechanisms between different ocular signs, such as "lens expansion may lead to shallowing of the anterior chamber" and "diabetes background aggravates the progression of macular edema" and other cross-disciplinary medical evidence.

[0036] Step 7: Input the comprehensive reasoning context and the secondary retrieval results into the large language model to guide it to complete the logical consistency test, contradiction identification, and deduction of the relationship between primary and secondary features, and generate a unified intermediate reasoning conclusion.

[0037] Step 8: Perform structured analysis and formatting on the intermediate inference conclusions to generate a comprehensive multi-source eye feature processing report. The report should include at least the following: summary of main manifestations, explanation of the correlation between signs, reference to supporting knowledge, identification analysis tips and suggested observation directions. The report should be output to external devices or information systems via API or file.

[0038] Figure 1 The illustrated process represents a preferred embodiment of the present invention, with each step executed sequentially to form a closed-loop information processing chain. Steps 2 to 4 constitute the First-Level Retrieval Enhancement Generation (FAG) process, enabling independent feature analysis of different professional directions; steps 5 to 7 constitute the Second-Level Retrieval Enhancement Generation (FAG) process, completing cross-domain information fusion and consistent reasoning. The overall architecture simulates the "subject-specific assessment—comprehensive reasoning" information integration mechanism in multidisciplinary collaboration, but its technical implementation is fully automated and requires no manual intervention.

[0039] A specific embodiment of the present invention is provided: S0: Method for building the basic information processing environment for multi-level RAG (preliminary preparation steps).

[0040] To support the accurate retrieval of professional medical information during subsequent multi-level retrieval enhancement processes, this invention pre-constructs a structured, hierarchical medical knowledge support environment before executing the multi-source eye feature processing flow. This environment is the technical foundation for achieving high-confidence reasoning analysis, such as... Figure 2 As shown, its construction method includes the following steps:

[0041] 1. Collect medical literature and clinical reference materials from authoritative sources, including industry standards issued by the National Health Commission, technical guidelines issued by the Ophthalmology Branch of the Chinese Medical Association, UpToDate clinical decision support system entries, core journal articles (such as the Chinese Journal of Ophthalmology), and recognized standard textbook content.

[0042] 2. Divide the original documents into thematic categories according to ophthalmology subspecialties, forming multiple independent knowledge sets. The professional categories should include at least: glaucoma, cataracts, fundus diseases (including diabetic retinopathy and macular degeneration), corneal diseases, uveitis, ocular surface diseases, strabismus and amblyopia, etc.

[0043] 3. Perform text preprocessing operations on each category of documents, specifically including: (1) Remove non-text content such as headers, footers, numbered lists, and references; (2) Divide long documents into semantically complete paragraph units, with each paragraph not exceeding 512 Chinese characters in length, in order to adapt to the input limitations of the embedding model; (3) Label each paragraph with metadata, including its professional category, source, publication time and version identifier.

[0044] 4. Use a Chinese text embedding model to convert each paragraph into a vector representation and store it in a vector database that supports efficient similarity retrieval.

[0045] 5. Construct two types of dedicated knowledge base systems to serve information processing tasks at different stages: (1) Sub-specialty exclusive knowledge base: Each sub-base contains only medical knowledge fragments in the corresponding specialty field, which are used for targeted knowledge matching and preliminary feature analysis in the first-level RAG.

[0046] (2) Multi-factor interaction knowledge base: Extract content describing the potential correlation mechanism between various ocular signs from the above data, such as "lens expansion can lead to shallow anterior chamber" and "diabetic background may aggravate fluid accumulation in the macular area" and other cross-domain medical expressions, which are specifically used for correlation mining and consistency reasoning in the second-level RAG.

[0047] 6. Establish a knowledge base maintenance mechanism, regularly update the content of each database, and record the timestamp, scope of modification, and review status of each change through a version control system to ensure the timeliness, completeness, and traceability of the knowledge content.

[0048] Once the aforementioned knowledge support environment is constructed, it serves as a fundamental component of the multi-source eye feature processing method of this invention and is invoked in every information processing task. Specifically, a sub-specialty-specific knowledge base is used to classify, identify, and independently analyze different vital sign cues, while a multi-factor interactive knowledge base supports cross-path information fusion and comprehensive reasoning. Together, they complete the technical closed loop from "path-specific initial screening" to "unified output."

[0049] S1: Standardized processing method for multi-source ocular vital signs information.

[0050] This step is the initial information access stage in the multi-level retrieval enhancement generation process. It aims to convert unstructured or semi-structured medical text from various channels into semantically consistent, formatted, and machine-parsable standard input data, serving as the foundational information source for subsequent feature recognition and knowledge matching. This process does not involve any medical judgment; it is solely an information preprocessing operation for large language models, specifically including the following sequential sub-steps:

[0051] S1.1 Receives multimodal clinical input information.

[0052] Multimodal input content related to the eyes is obtained from hospital information systems (HIS), electronic health record platforms, doctor input interfaces, or mobile terminal devices, mainly including the following four types of information: (1) Visual complaints: The main visual abnormalities described by the individual, such as: "Blurred vision in the left eye for 3 days" or "Swelling pain in the right eye accompanied by headache on the same side for 12 hours".

[0053] (2) Clinical record summary: The symptom development process recorded by medical staff, including qualitative descriptions such as onset time, trend of change, and previous intervention measures.

[0054] (3) Eye examination indicators: Objective measurement values ​​obtained through professional instruments, including but not limited to: 1) Visual acuity (uncorrected / corrected); 2) Intraocular pressure (unit: mmHg); 3) Anterior chamber depth; 4) Lens condition (LOCS III classification); 5) Description of fundus observation (e.g., "the optic disc margin is clear, and the macular reflection is absent").

[0055] (4) Imaging report summary: textual conclusions of auxiliary examinations, such as OCT showing "subfoveal fluid-filled dark area", fundus photography showing "scattered punctate hemorrhage and hard exudate".

[0056] All inputs are in the form of natural language text, allowing a mix of numerical fields and qualitative expressions.

[0057] S1.2 Perform text cleaning and terminology standardization.

[0058] Perform the following cleaning and standardization operations on the above raw inputs: (1) Remove irrelevant characters, such as repeated punctuation, garbled text, emoticons, and non-medical terms.

[0059] (2) Standardize the expression methods and convert colloquial and vague expressions into standardized terms, for example: 1) "High intraocular pressure" → "Elevated intraocular pressure"; 2) "Cannot see" → "Deteriorating vision"; 3) "Seeing dark shadows floating in front of your eyes" → "Floaters-like symptoms".

[0060] (3) Expand common abbreviations to ensure terminology consistency, for example: 1) "DR" → "Diabetes-related retinopathy"; 2) "POAG" → "Primary open-angle glaucoma-like appearance".

[0061] Normalization was achieved using a mapping mechanism based on a rule engine and a medical terminology dictionary, with priority given to the standard terminology set published by the National Health Commission and the ICD-11 ophthalmology coding system.

[0062] S1.3 Extract key eye features and annotate contextual attributes.

[0063] Using medical named entity recognition (NER) technology, the following five key features are automatically extracted from the cleaned text, and corresponding contextual attribute information is attached to them, as shown in Table 1: Table 1. Attribute Information Table Corresponding to Feature Categories Entity extraction can be achieved through pre-trained Chinese medical NER models (such as Baidu's "Lingyi Zhihui" open-source model or fine-tuned BERT-CRF architecture), supporting automated batch processing and high-accuracy output.

[0064] S1.4 Construct a structured representation of the text.

[0065] The aforementioned processing results are integrated into a standard input text that is semantically coherent, hierarchically structured, and free of redundant expressions. The organizational structure is shown in the following example:

Basic Information

[0066] [Clinical Record Summary] The patient suddenly experienced right eye swelling and pain early this morning, accompanied by nausea and significant blurred vision. There is no history of trauma. The patient has a 5-year history of cataracts but has not undergone surgical treatment.

[0067] [Ocular vital signs data] Right eye visual acuity: 0.05 (no improvement after correction), intraocular pressure: 42 mmHg (N: 10–21); Slit-lamp examination: corneal edema, shallow anterior chamber, moderately dilated pupil, and sluggish pupillary light reflex; Lens opacity (LOCS III:NC3).

[0068] [Summary of Auxiliary Examinations] OCT examination revealed mild macular edema in the right eye; fundus photography showed no obvious hemorrhage or exudation.

[0069] During this process, the system automatically identifies and corrects spelling errors or terminological deviations, such as correcting "cataract-like changes" to "cataract-like changes," thereby improving the quality of input data.

[0070] S1.5 Outputs standardized information text.

[0071] The structured text described above is taken as the final output of this step, and denoted as... T 0 is used as a unified input source for subsequent inference modules.

[0072] The output format is plain text (UTF-8 encoding), which supports API interface transmission and direct reading of large language models, ensuring stable access for downstream modules.

[0073] S2: Perform first-level search enhancement generation to generate preliminary feature analysis results from multiple professional directions.

[0074] This step, as the first stage of the multi-level information processing mechanism of this invention, aims to simulate the information organization logic of "specialty assessment" in clinical collaboration. By identifying sub-specialty dimensions of the input information and independently conducting knowledge retrieval and feature matching within each specialty's knowledge system, it achieves accurate triage and professional initial screening of multi-source ocular signs. For example... Figure 4 and Figure 5 As shown, this step includes the following consecutive sub-steps:

[0075] S2.1 Sub-specialty identification based on large language model.

[0076] The standardized information text output in step S1 T Inputting 0 into a large language model, and using prompts (Prompt Engineering) to drive the model to complete a multi-label classification task, the model can identify the ophthalmology specialty that the current input may involve.

[0077] The models used are large language models (LLMs) with strong Chinese understanding capabilities, such as Qwen-Max, ChatGLM3-Turbo, or ERNIE Bot 4.5, running in local or cloud inference environments.

[0078] The output is denoted as the set of professional direction labels. Each element represents a technical path that requires further in-depth analysis.

[0079] S2.2 Construct targeted queries and perform professional domain knowledge retrieval.

[0080] For tag collection Each specialization Each operation triggers an independent knowledge retrieval operation, ensuring that medical information from different fields does not interfere with each other.

[0081] The specific implementation method is as follows: 1. Provides multiple pre-built sub-specialty-specific vector knowledge bases, namely: 1) A knowledge base on glaucoma-related symptoms; 2) Cataract-like changes knowledge base; 3) Fundus anomaly knowledge base; 4) Knowledge base of corneal disease related indicators.

[0082] The content stored in each knowledge base comes from industry standards issued by the National Health Commission, guidelines of the Chinese Medical Association, authoritative textbooks and core journal literature. After text slicing, cleaning and metadata annotation, it is converted into vector form using Chinese embedding models (such as text2vec-large-chinese or bge-large-zh-v1.5) to establish an index structure that supports efficient similarity matching.

[0083] 2. For each tag Construct a special query statement Its form is: "Combined with" [Special expertise in the field] Analyze whether the following individuals exhibit relevant characteristics in this area? Key signs include: [From...] T [Relevant descriptions extracted from 0].

[0084] 3. The data is input into a knowledge base with the corresponding name, and a semantic retrieval is performed using the cosine similarity algorithm to return the top-k most relevant knowledge fragments (usually...). k =3), denoted as .

[0085] 4. All retrieval operations are executed in parallel without affecting each other, ensuring the independence and professional focus of each analysis path.

[0086] S2.3 Generate a structured preliminary analysis report on professional directions.

[0087] For each activated specialty Call the large language model and combine it with its corresponding search results { } with the original input text T 0. Generate an independent "Preliminary Analysis Report on Professional Direction". This process controls the output format through customized prompt templates to ensure that each report has a consistent structure and comparability.

[0088] The model outputs a structurally complete piece of natural language text, called the first... Preliminary analysis results in each direction D i This reflects an independent interpretation of information from this professional perspective.

[0089] all D iAfter generation, the results are compiled into a preliminary multi-path analysis set. This serves as the foundation for information input in the next stage.

[0090] S2.4 Outputs intermediate results and passes them to subsequent processes.

[0091] The analysis result set With the original information text T 0 is encapsulated together as a standard data packet, with a timestamp and version identifier added, ready to proceed to step S3.

[0092] In this step, if the confidence in a certain direction is too low (such as the model outputs "no obvious evidence to support it" and no relevant knowledge is detected), the processing of that path is allowed to terminate, realizing dynamic path pruning and improving the overall processing efficiency.

[0093] S3: Execute the second-level retrieval enhancement generation to complete multi-path information fusion and consistent reasoning.

[0094] This step, as the advanced stage of the multi-level information processing mechanism of this invention, aims to simulate the "comprehensive consultation" information integration logic in multidisciplinary collaboration. It globally integrates the preliminary analysis results from multiple professional directions output by the first-level RAG, combining cross-disciplinary medical knowledge to complete contradiction identification, primary and secondary relationship deduction, and the generation of consistent conclusions. This step includes the following sequential sub-steps:

[0095] S3.1 Construct a comprehensive reasoning context.

[0096] The standardized information text output in step S1 T 0 and the preliminary multipath analysis result set generated in step S2 The text is structured and pieced together to form a semantically complete and traceable composite text. This serves as the input basis for this stage.

[0097] The splicing rules are as follows: (1) First, retain the original information text. T 0; (2) Then add each "Preliminary Analysis Report on Professional Direction" in sequence. D i And its source is marked with a special identifier.

[0098] By using the above methods, we can ensure that subsequent models can accurately distinguish the output sources of each analysis path and avoid information confusion.

[0099] income It not only includes the individual's original eye signs records, but also integrates independent judgment clues from different professional dimensions, forming a comprehensive context that supports high-level reasoning.

[0100] S3.2 Perform cross-disciplinary related knowledge retrieval.

[0101] Will As a query statement, it is input into an independently constructed multi-factor interactive knowledge base to perform a second knowledge retrieval.

[0102] The content stored in this multi-factor interaction knowledge base focuses on the following types of information: (1) Multiple ocular manifestations co-occurring pattern (such as macular edema combined with lens opacity) (2) Potential correlation mechanisms between manifestations (e.g., increased lens volume leads to shallower anterior chamber, increasing the risk of angle closure) (3) Suggestions for the order of examination or intervention (e.g., certain surgical procedures should not be performed immediately before intraocular pressure is controlled) (4) Recommendations for the management of complex conditions in national clinical guidelines The query is vectorized using a Chinese embedding model consistent with S2 (e.g., bge-large-zh-v1.5), and semantic similarity is calculated in the knowledge base, returning Top- k The most relevant knowledge fragment (usually) k =4), denoted as .

[0103] The focus of this search is to discover potential connections or conflicts between different analytical approaches, rather than single feature matching, such as verifying whether "cataract-like changes may induce glaucoma-related manifestations".

[0104] S3.3 invokes a large language model to complete integrated reasoning and conflict identification.

[0105] Integrating the context Related knowledge fragments retrieved A high-performance large language model is input as a common input. By designing guiding prompts, the model outputs an intermediate inference conclusion in natural language form. The content must include: (1) Naming of primary and secondary manifestations; (2) Logical relationships between manifestations (e.g., "caused by," "existing concurrently," "aggravating factors"); (3) Explanation of key points of disagreement; (4) The citation of key evidence relied upon in the reasoning process.

[0106] S3.4 Generate and output the comprehensive analysis results.

[0107] right The data was standardized and extracted to generate a unified format for a comprehensive analysis of multi-source eye features. The results have a clear logical chain and evidence-based support, and can be used to help medical personnel quickly grasp the overall situation of complex ocular information.

[0108] like Figure 6 As shown, based on multi-source eye features, a first-level RAG is used to achieve accurate routing and targeted retrieval of knowledge bases related to various professional directions, and to generate preliminary feature analysis results for the corresponding paths. These results, together with the original input, constitute the contextual input for comprehensive reasoning analysis. The advantage of this design is that it can ensure accurate matching of professional knowledge, as well as the effective integration and efficient reuse of the analysis results of each path.

[0109] S4: Generate and output a comprehensive analysis report of multi-source eye features.

[0110] This step serves as the final output stage of the information processing flow of this invention, such as... Figure 7 As shown, the aim is to standardize and structure the intermediate inference conclusions generated by S3, forming a comprehensive multi-source ocular feature analysis report with a unified format, complete content, and easy system integration, for subsequent information system access or medical personnel reference. Specifically, it includes the following fields:

[0111] (1) Summary of main manifestations: Based on the comprehensive reasoning results, extract the most likely core ocular manifestations (such as “abnormal anterior chamber structure with elevated intraocular pressure”) and attach a brief description; (2) Description of minor manifestations: List other relevant signs or co-occurring features that have been identified (e.g., “lens opacity (LOCS III:NC3)”); (3) Inference of relationships between manifestations: Explain the potential logical connections between each feature, such as non-deterministic statements such as "caused by", "existing concurrently", "may aggravate", etc., to avoid causal assertions; (4) List of supporting evidence: Cite the knowledge base entries found in the second-level search in the form of numbers (e.g., Rel-KB-2024-017) and indicate the source type (guide / document / textbook); (5) Distinguishing analysis tips: List similar cases that need further differentiation (e.g., "need to be differentiated from primary open-angle mechanism"), only as information comparison suggestions; (6) Suggestions for follow-up observation: Provide non-mandatory data attention tips, such as "It is recommended to continuously monitor the trend of intraocular pressure changes" and "OCT can be repeated to assess the status of the macular area"; (7) Inspection coordination prompts: When multiple tests are involved, prompts are given on the order of priority or the method of cooperation (e.g., “It is recommended to complete intraocular pressure control before surgical operation assessment”).

[0112] All content is organized using natural language combined with structured tags, and supports generating multiple output formats such as JSON, XML, or PDF.

[0113] This comprehensive analysis report can be transmitted to the Hospital Information System (HIS), electronic health record platform, or physician workstation via standard interfaces (such as HTTP API, HL7-compatible protocols), achieving seamless integration with existing business processes. Simultaneously, an operation log for this processing is recorded (for system auditing, quality traceability, and subsequent optimization analysis), including:

[0114] (1) Enter the timestamp; (2) Time consumed at each stage; (3) The version of the knowledge base used; (4) Model call ID.

[0115] This invention achieves a leap from "single path judgment" to "multi-dimensional information integration," and its core lies in: 1. Through a two-level retrieval enhancement generation (Multi-Level RAG) architecture, the two processing stages of "preliminary screening of professional direction" and "comprehensive reasoning analysis" are separated: the first level focuses on targeted knowledge matching and independent analysis by sub-professional dimension to avoid information mixing from different fields; the second level focuses on cross-path clue fusion and logical consistency verification, which significantly improves the processing accuracy under complex input.

[0116] 2. Introduce a targeted retrieval mechanism to ensure the professionalism and relevance of knowledge access at each stage.

[0117] In the first-level RAG, the query is routed to the corresponding professional knowledge base based on the preliminary identification results, realizing "dedicated database for dedicated inspection", which effectively prevents noise interference and low-priority information coverage problems caused by general retrieval.

[0118] 3. Leveraging the deep semantic understanding and contextual inference capabilities of large language models, we can complete the consistency verification and conflict identification of multi-source analysis results.

[0119] In the second-level RAG, the association rules in the multi-factor interaction knowledge base are combined to further reason about the preliminary conclusions, identify potential contradictions, clarify the primary and secondary relationship, and form a comprehensive output with interpretability.

[0120] The entire process requires no expert intervention, enabling efficient and systematic processing of multiple ocular signs. It achieves near-multidisciplinary collaboration levels in terms of the integrity of the reasoning chain and the stability of the results. This method is particularly suitable for primary healthcare settings or rapid screening scenarios, assisting medical personnel in grasping the overall picture of complex clinical information with limited resources.

[0121] Based on the same inventive concept, embodiments of the present invention provide a feature analysis system for eye images, the system comprising: The data acquisition module is used to acquire multimodal clinical input information from electronic medical record systems or medical staff. It extracts key eye features from the multimodal clinical input information using medical named entity recognition technology to obtain the initial information text.

[0122] The first-level retrieval module identifies the sub-specialty direction of the initial information text using predefined prompts from the large language model, obtaining at least one ophthalmology specialty category label. A first knowledge retrieval is then performed in the sub-specialty-specific vector knowledge base corresponding to each ophthalmology specialty category label to obtain the first piece of medical knowledge.

[0123] The first-level enhancement generation module combines the initial information text and primary medical knowledge through a large language model to achieve independent feature analysis of different professional directions and obtain preliminary analysis results corresponding to each ophthalmology professional category label.

[0124] The second-level retrieval module is used to summarize the initial information text and all preliminary analysis results through structured splicing, resulting in a comprehensive reasoning context containing multi-dimensional analytical clues. Based on the comprehensive reasoning context, a second knowledge retrieval is performed in the multi-factor interactive knowledge base to obtain second medical knowledge describing the potential correlation mechanisms between different ocular features.

[0125] The second-level enhancement generation module is used to perform comprehensive reasoning based on the integrated reasoning context and secondary medical knowledge through a large language model, in order to complete cross-domain information fusion and consistent reasoning, and obtain intermediate reasoning conclusions. The intermediate reasoning conclusions are then subjected to structured parsing and formatting to generate a multi-source ocular feature comprehensive processing report.

[0126] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A feature analysis method for eye images, characterized in that, include: The system acquires multimodal clinical input information from electronic medical record systems or medical staff, and extracts key eye features from the multimodal clinical input information using medical named entity recognition technology to obtain initial information text. The initial information text is subjected to sub-specialty direction recognition by setting prompt words in the large language model to obtain at least one ophthalmology specialty category label; the first knowledge retrieval is performed in the sub-specialty exclusive vector knowledge base corresponding to each ophthalmology specialty category label to obtain the first medical knowledge. By combining the initial information text and the first medical knowledge through a large language model, independent feature analysis of different professional directions can be achieved, and preliminary analysis results corresponding to each ophthalmology professional category label can be obtained. The initial information text and all preliminary analysis results are summarized by a structured splicing method to obtain a comprehensive reasoning context containing multi-dimensional analysis clues; a second knowledge retrieval is performed in the multi-factor interactive knowledge base based on the comprehensive reasoning context to obtain second medical knowledge describing the potential correlation mechanism between different ocular features; The comprehensive reasoning context and the second medical knowledge are comprehensively reasoned using a large language model to complete cross-domain information fusion and consistent reasoning, and intermediate reasoning conclusions are obtained. The intermediate reasoning conclusions are then structured and formatted to obtain a multi-source eye feature comprehensive processing report.

2. The feature analysis method for an eye image as described in claim 1, characterized in that, The multimodal clinical input information includes: visual complaint text, clinical record summary, eye examination indicators, and image report summary; The key ocular features include: visual abnormalities, anatomical locations, types of manifestations, numerical indicators, and imaging abnormalities. Before extracting key eye features from the multimodal clinical input information using medical named entity recognition technology, the process further includes text cleaning and terminology standardization of the multimodal clinical input information, including removing irrelevant characters, standardizing expression methods, and expanding preset medical abbreviations.

3. The feature analysis method for an eye image as described in claim 1, characterized in that, The construction of the sub-specialty-specific vector knowledge base and the multi-factor interaction knowledge base specifically includes: Obtain medical literature and clinical reference materials; The medical literature and clinical references are divided into themes according to ophthalmology subspecialties to obtain multiple independent knowledge sets; the documents in each knowledge set are segmented into multiple paragraph units, and metadata is labeled for each paragraph unit to obtain a subspecialty-specific vector knowledge base including medical knowledge fragments of the corresponding specialty. Content describing the potential correlation mechanisms between various ocular features was extracted from the medical literature and clinical references to obtain a multi-factor interaction knowledge base.

4. The feature analysis method for an eye image as described in claim 1, characterized in that, The first knowledge retrieval is performed in the sub-specialty-specific vector knowledge base corresponding to each ophthalmology specialty category label to obtain the first medical knowledge, specifically including: For each of the aforementioned ophthalmology specialty category tags, construct its corresponding query statement; The query statement is input into the sub-specialty-specific vector knowledge base with the corresponding name. Semantic retrieval is performed in parallel using the cosine similarity algorithm, and a preset number of the most relevant medical knowledge fragments are returned to obtain the first medical knowledge.

5. The feature analysis method for an eye image as described in claim 1, characterized in that, The preliminary analysis results include at least: feature matching judgment, confidence level, supporting evidence, exclusion criteria, and prompts regarding whether additional information is needed.

6. The feature analysis method for an eye image as described in claim 1, characterized in that, The second medical knowledge describing the potential correlation mechanism between different ocular features includes at least one of the following: co-occurrence patterns of multiple ocular manifestations, potential pathological correlation mechanisms between manifestations, and clues for the order of examination or intervention.

7. The feature analysis method for an eye image as described in claim 1, characterized in that, The process of using a large language model to perform comprehensive reasoning on the integrated reasoning context and the second medical knowledge to complete cross-domain information fusion and consistent reasoning, and to obtain intermediate reasoning conclusions, specifically includes: Logical contradictions among the preliminary analysis results are identified using a large language model. Based on logical contradictions, the preliminary analysis results are ranked according to their primary and secondary relationships, and the secondary associations corresponding to each preliminary analysis result are deduced. Preliminary analysis results where the ranking result is greater than a set order and the secondary association is greater than a set threshold are used as intermediate inference conclusions. These intermediate inference conclusions include: the naming of primary and secondary manifestations, the logical relationship between manifestations, the explanation of key points of divergence, and the citation of key evidence on which the inference process depends.

8. The feature analysis method for an eye image as described in claim 1, characterized in that, The multi-source ocular feature comprehensive processing report includes at least the following fields: summary of main manifestations, description of secondary manifestations, inference of relationships between manifestations, list of supporting evidence, discriminative analysis tips, and suggestions for observation directions.

9. A feature analysis system for eye images, characterized in that, include: The data acquisition module is used to acquire multimodal clinical input information from the electronic medical record system or entered by medical staff, and extract key eye features of the multimodal clinical input information through medical named entity recognition technology to obtain initial information text; The first-level retrieval module is used to identify the sub-specialty direction of the initial information text through the set prompt words in the large language model to obtain at least one ophthalmology specialty category label; and to perform the first knowledge retrieval in the sub-specialty exclusive vector knowledge base corresponding to each ophthalmology specialty category label to obtain the first medical knowledge. The first-level enhancement generation module is used to combine the initial information text and the first medical knowledge through a large language model to achieve independent feature analysis of different professional directions and obtain preliminary analysis results corresponding to each ophthalmology professional category label; The second-level retrieval module is used to summarize the initial information text and all preliminary analysis results through structured splicing to obtain a comprehensive reasoning context containing multi-dimensional analysis clues; based on the comprehensive reasoning context, a second knowledge retrieval is performed in the multi-factor interactive knowledge base to obtain second medical knowledge describing the potential correlation mechanism between different ocular features; The second-level enhancement generation module is used to perform comprehensive reasoning on the comprehensive reasoning context and the second medical knowledge through a large language model to complete cross-domain information fusion and consistent reasoning, and obtain intermediate reasoning conclusions; and to perform structured parsing and formatting processing on the intermediate reasoning conclusions to obtain a multi-source eye feature comprehensive processing report.

Images