Ophthalmology information extraction system based on electronic medical record

Abstract

The invention relates to the technical field of medical information, in particular to an ophthalmology information extraction system based on electronic medical records. The system comprises an electronic medical record data acquisition and cleaning module, a domain knowledge driven ophthalmology information extraction module, a multi-modal data fusion and labeling module and a structured result generation and display module, and original medical record data can be acquired; performing data cleaning on the original case data to generate a patient electronic case data set; therefore, through the whole-process modular design and multi-modal data fusion processing, the problems that in a traditional ophthalmic case information extraction mode, data processing is not coherent, information extraction is not accurate, and results are difficult to directly express are solved, and the data utilization rate and clinical application value are improved.

Description

Technical Field

[0001] The present invention relates to the field of medical information technology, and in particular to an ophthalmology information extraction system based on electronic medical records. Background Art

[0002] In terms of multimodal data fusion, existing technologies are difficult to effectively integrate ophthalmic electronic text cases and ophthalmic imaging cases. The lack of deep fusion of this multimodal data leads to data gaps and information loss in the information processing process. Existing information extraction technologies are not accurate enough for tracking key medical events, disease progression, and capturing changing trends in ophthalmic medical records. In the long span of ophthalmic cases, with the dynamic changes of patients' conditions, existing technologies are difficult to achieve accurate disease trend analysis and dynamic curve drawing. Current processing methods are mostly static data processing, and fail to effectively consider the time dimension and the long-term nature of disease changes, thereby limiting the comprehensive monitoring and evaluation of disease development. Furthermore, existing technologies often lack systematic modeling methods when processing complex data structures, especially when involving multiple clinical data and multiple time nodes, the temporal relationship and dynamic changes of medical record information cannot be fully captured and expressed. Although some advanced image annotation technologies have been applied to ophthalmic image analysis, the accuracy and consistency of image data labeling are still insufficient, especially for tasks such as regional annotation and thickness segmentation of lesions, which still face the challenges of low precision and high complexity. Summary of the invention

[0003] Based on this, it is necessary to provide an ophthalmic information extraction system based on electronic medical records to solve at least one of the above technical problems.

[0004] To achieve the above purpose, an ophthalmic information extraction system based on electronic medical records includes the following modules: The electronic medical record data acquisition and cleaning module is used to obtain the original case data; clean the original case data to generate the patient electronic case data set; The domain knowledge-driven ophthalmology information extraction module is used to obtain the ophthalmology professional database; initially create the knowledge graph of the ophthalmology professional database to generate the ophthalmology preliminary knowledge graph; construct the preliminary relationship information model of the ophthalmology preliminary knowledge graph to generate the ophthalmology preliminary information extraction model; The multimodal data fusion and annotation module is used to perform ophthalmic multimodal data fusion on ophthalmic electronic text cases and ophthalmic image cases to generate ophthalmic case multimodal fusion data; perform ophthalmic data annotation on ophthalmic case multimodal fusion data to generate ophthalmic case multimodal annotation data; perform model iteration on the ophthalmic preliminary information extraction model based on the ophthalmic case multimodal annotation data to generate an ophthalmic electronic case information extraction model; The structured result generation and display module is used to generate information extraction results from the ophthalmology electronic case information extraction model, generate ophthalmology electronic case information extraction results; generate structured reports based on the ophthalmology electronic case information extraction results, generate ophthalmology electronic case information structured reports; perform data visualization processing on the ophthalmology electronic case information structured reports, generate ophthalmology electronic case visualization charts, and thus complete the electronic case ophthalmology information extraction task.

[0005] The beneficial effect of the present invention is that the electronic medical record data acquisition and cleaning module standardizes the processing of raw data, provides a unified and accurate data basis for subsequent analysis, and avoids the problem of information loss caused by data redundancy, incompleteness or inconsistent format in traditional electronic medical records. Secondly, the domain knowledge-driven ophthalmic information extraction module constructs a preliminary information extraction model based on domain knowledge and uses professional databases for semantic relationship modeling, so that complex medical terms and related information can be efficiently parsed and extracted. This not only improves the accuracy of data extraction, but also provides support for the subsequent in-depth analysis of multimodal data. Thirdly, the multimodal data fusion and annotation module integrates electronic text cases and image cases to form multimodal fusion data, fully exploring the potential associations between different data types; at the same time, through annotation and model iteration, the information extraction model is further optimized, so that its recognition and processing capabilities for multimodal data are enhanced. Finally, the structured result generation and display module generates standardized reports based on the extraction results, and presents the dynamic characteristics and statistical trends of complex information through data visualization, which not only improves the efficiency of information expression, but also provides multi-dimensional data support for clinical decision-making. Therefore, the present invention solves the problems of incoherent data processing, inaccurate information extraction and difficult to intuitively express results in traditional ophthalmic case information extraction methods through full-process modular design and multimodal data fusion processing, thereby improving data utilization and clinical application value.

[0006] Preferably, the electronic medical record data acquisition and cleaning module includes the following functions: Obtain original case data; Unify the original case data into the HL7 standard information format to generate unified patient electronic case data; Perform preliminary metadata generation on the unified data of patient electronic medical records to generate electronic patient metadata case data; The data set is cleaned according to the patient's electronic metadata case data, and timestamp verification is performed to generate the patient's electronic case data set, which includes ophthalmology electronic text cases and ophthalmology imaging cases.

[0007] The present invention ensures the diversity and integrity of data sources through the acquisition of original case data, and provides a guarantee for comprehensively reflecting the patient's diagnosis and treatment information. Through the unified processing of the information format of the HL7 standard, not only the data integration barriers caused by format incompatibility between different data sources are solved, but also the standardization and consistency of data interaction are improved, thereby establishing a standardized bridge for data exchange between multiple systems. On this basis, metadata is generated for the patient's electronic case data, and by defining the attributes, structure and relevance of the data, an accurate semantic foundation is laid for further cleaning and analysis of the data. The introduction of metadata also improves the efficiency of data management, making the query, analysis and call of complex case data faster. Subsequently, through the timestamp verification and cleaning mechanism, redundant, repeated and outdated records are eliminated to ensure the timeliness and reliability of the data, and significantly reduce the impact of data noise. Finally, the generated patient electronic case data set covers two main data forms: ophthalmic electronic text cases and ophthalmic imaging cases, which not only realizes the unified management of multi-type data, but also provides complete data support for the subsequent analysis and fusion of multimodal data.

[0008] Preferably, the domain knowledge-driven ophthalmic information extraction module includes the following functions: Access to the ophthalmology specialty database; Initially create an ophthalmology knowledge graph for the ophthalmology professional database and generate a preliminary ophthalmology knowledge graph; The preset language model is used to extract semantic relationships from the preliminary ophthalmology knowledge graph, and the preset language model is optimized to generate a preliminary ophthalmology information extraction model.

[0009] The present invention ensures the authority and comprehensiveness of the data source by acquiring an ophthalmology professional database, covering a large number of standard terms, medical records and related research results in the field, and provides a rich basic data resource for subsequent knowledge construction. Secondly, the initial creation of the knowledge graph is carried out based on the ophthalmology professional database, and the scattered and unstructured professional information is converted into a graph form with semantic structure and relevance. This structured expression method effectively systematizes the complex ophthalmic knowledge and lays the foundation for information query, analysis and semantic reasoning. Subsequently, by using the preset BERT language model to extract semantic relationships from the preliminary knowledge graph, the ability to understand domain-specific terms and semantic relationships is further improved. The addition of the BERT model not only improves the semantic processing accuracy of unstructured data, but also can capture the implicit semantics and multi-level relationships in complex texts, ensuring the accuracy and depth of information extraction. In addition, the optimization process of the BERT language model combines the specific needs of the ophthalmology field, making it more suitable for the professional context and improving the specificity and efficiency of information extraction. The final generated ophthalmic preliminary information extraction model has powerful semantic analysis and relationship recognition capabilities, providing an accurate data foundation for subsequent clinical decision support, data mining and scientific research analysis.

[0010] Preferably, the initial creation of an ophthalmology knowledge graph for an ophthalmology professional database includes: Calibrate ophthalmic data features on the ophthalmic database to generate ophthalmic data structure features; define ophthalmic atlas nodes on the ophthalmic data structure features to generate professional ophthalmic atlas nodes; Map the node relationships according to the professional ophthalmology atlas nodes to generate ophthalmology data-atlas node data; Initially create an ophthalmic knowledge graph for ophthalmic data-graph node data to generate a preliminary ophthalmic knowledge graph.

[0011] The present invention converts complex and scattered raw data into structured data with clear semantic features by calibrating data features of ophthalmology professional database and extracting and classifying core attributes and structural characteristics in the data. This feature calibration not only improves the standardization of data, but also lays a clear data dimension foundation for the construction of knowledge graph. Secondly, by defining the graph nodes of data structure features, the classification standards and semantic attributes of nodes in the knowledge graph are refined, and the generated professional ophthalmology graph nodes effectively reflect the diversity and professionalism of entities and concepts in the field. Then, node relationship mapping is performed according to the defined graph nodes, and the implicit relationship in the ophthalmology data is converted into an explicit structure through logical mapping and semantic association, thereby generating ophthalmology data-graph node data. This process further enhances the relevance and interpretability of the data and solves the problem of insufficient relationship expression ability in traditional databases. Finally, a preliminary knowledge graph is created based on the graph node data, and the domain knowledge is systematically presented in the form of a graph, which not only realizes the fusion and structured expression of multi-source data, but also improves the efficiency and accuracy in application scenarios such as complex semantic query and knowledge reasoning.

[0012] Preferably, the method of extracting semantic relationships from the preliminary ophthalmology knowledge graph using a preset language model includes: Based on the preliminary ophthalmology knowledge graph, a change trend analysis is performed to generate the patient's ophthalmology case trend information; the patient's ophthalmology case trend information is analyzed for patient volatility to generate the patient's ophthalmology volatility information; the patient's ophthalmology case trend information and the patient's ophthalmology volatility information are used to generate a change trend graph to generate an ophthalmology patient change trend graph; Extract the semantic relationship between symptoms and intraocular pressure from the trend chart of ophthalmology patients, and generate the patient's symptom-intraocular pressure relationship data; extract the semantic relationship between complications and lesion points from the trend chart of ophthalmology patients, and generate the patient's complication-lesion point relationship data; The preset BERT language model is optimized based on the patient's symptom-intraocular pressure relationship data and the patient's complication-lesion point relationship data to generate a preliminary ophthalmic information extraction model.

[0013] The present invention generates ophthalmic case trend information by performing change trend analysis based on the preliminary ophthalmic knowledge graph and extracting dynamic change characteristics from patient case data. This process not only reveals the evolution law of the patient's condition in the time dimension, but also provides a comprehensive data basis for further analysis of volatility characteristics. Subsequently, the volatility analysis of the patient's ophthalmic case trend information is performed, and the patient's ophthalmic volatility information is generated by capturing abnormal changes and fluctuation patterns, further enriching the description dimension of the data. This analysis can effectively identify the key nodes and high-risk periods of the disease, providing a reference for precise intervention. Then, the trend information and volatility information are integrated to generate a change trend graph, which lays the foundation for the visualization analysis and multi-dimensional association exploration of the data. On the basis of the change trend graph, the patient's symptom-intraocular pressure relationship data and complication-lesion point relationship data are generated respectively through semantic relationship extraction. The extraction of this semantic relationship deepens the association expression between the data, can effectively reflect the potential connection between the symptom and intraocular pressure changes and between complications and lesions, and enhances the semantic application value of the data. Finally, the extracted semantic relationship data is used to optimize the preset BERT language model. By combining domain-specific relational semantics and data features, the model's semantic understanding and information extraction capabilities are further enhanced. The generated ophthalmological preliminary information extraction model has higher accuracy and applicability.

[0014] Preferably, the multimodal data fusion and annotation module includes the following functions: Perform ophthalmic multimodal data fusion on ophthalmic electronic text cases and ophthalmic imaging cases to generate ophthalmic case multimodal fusion data; Perform ophthalmic data annotation on multimodal fusion data of ophthalmic cases to generate multimodal annotation data of ophthalmic cases; Perform quantitative evaluation on the annotation results of multimodal annotation data of ophthalmic cases to generate quantitative evaluation data of multimodal annotation results of ophthalmic cases; iterate the ophthalmic preliminary information extraction model based on the quantitative evaluation data of multimodal annotation results of ophthalmic cases to generate an ophthalmic electronic case information extraction model.

[0015] The present invention effectively realizes the collaborative processing of structured and unstructured data by fusing ophthalmic electronic text cases with image cases through multimodal data, combining text semantic features with image lesion features through deep learning methods, and the generated multimodal fusion data has higher semantic consistency and representation ability. This fusion process overcomes the problem of information isolation in traditional single-modal data analysis, and provides rich associated information for subsequent annotation and mining. Ophthalmic data annotation is performed on multimodal fusion data, and high-quality annotation data sets are generated by combining the experience of domain experts, thereby improving the accuracy and semantic expression ability of data labels. Furthermore, the annotation results are quantitatively evaluated, and the quality of the annotation data is systematically measured by constructing quantitative indicators (such as accuracy, recall rate and consistency score), and the generated quantitative evaluation data provides a scientific basis for model optimization. Model iteration is performed based on quantitative evaluation data, and evaluation feedback is introduced into the model update process, so that the information extraction model can dynamically adapt to the changes in multimodal data features, improve its ability to parse complex data semantic relationships and adapt to new cases.

[0016] Preferably, the ophthalmic multimodal data fusion of ophthalmic electronic text cases and ophthalmic image cases includes: Classify the ophthalmology electronic case texts to generate ophthalmology electronic case classification data; Perform optical coherence tomography on ophthalmic imaging cases to generate ophthalmic imaging scan data; perform lesion area marking on ophthalmic imaging scan data to generate ophthalmic imaging lesion marking data; perform thickness segmentation masking on ophthalmic imaging lesion marking data to generate ophthalmic lesion imaging layer thickness data; The ophthalmic electronic medical record classification data and the ophthalmic lesion image layer thickness data are fused into ophthalmic multimodal data to generate ophthalmic case multimodal fusion data.

[0017] The present invention helps to organize and hierarchically process the case content according to dimensions such as medical history, diagnosis, and treatment plan by classifying ophthalmic electronic text cases. The generated classified data has a highly structured feature, which lays a clear logical framework for subsequent semantic analysis and data association. At the same time, the optical coherence tomography processing of ophthalmic imaging cases generates ophthalmic imaging scanning data through high-precision image acquisition, ensuring the spatial resolution of the image data and the clear presentation of tissue details. On this basis, through the lesion area marking, the disease-related lesion site is accurately locked, making the representation of the image data more concentrated and prominent, and the generated lesion marking data significantly improves the semantic relevance of the image information. Subsequently, through the technical processing of the thickness segmentation mask, the image thickness information of the ophthalmic lesions is extracted in layers, and the layered thickness data with high-dimensional anatomical significance is generated, which provides detailed quantitative support for disease typing and progression assessment.

[0018] Preferably, the ophthalmic data labeling of multimodal fusion data of ophthalmic cases includes: Perform preliminary annotation processing based on multimodal fusion data of ophthalmology cases to generate preliminary multimodal annotation results of ophthalmology; upload the preliminary multimodal annotation results of ophthalmology to the cloud platform for correction and verification, and generate preliminary ophthalmology annotation expert verification results; Compare the preliminary annotation parameters based on the ophthalmology preliminary annotation expert verification results and the ophthalmology multimodal preliminary annotation results, and transmit the parameters back to generate multimodal annotation comparison data of ophthalmology cases; The ophthalmic data is annotated for the multimodal annotated comparison data of ophthalmic cases to generate multimodal annotated data of ophthalmic cases.

[0019] The present invention performs preliminary annotation processing on the multimodal fusion data of ophthalmic cases, and the generated multimodal preliminary annotation results can provide a basis for subsequent data verification, and to a certain extent cover the core information characteristics of multimodal data. Subsequently, the preliminary annotation results are uploaded to the cloud platform for correction verification, and experts in professional fields are introduced to verify and correct the annotation content. The generated verification results not only integrate the accuracy of expert experience, but also achieve the efficiency based on cloud platform collaboration, and provide a reliable basis for the credibility of data annotation. Based on the preliminary annotation results and expert verification results, parameter comparison and feedback optimization are performed, which not only establishes a dynamic adjustment mechanism, but also effectively feedbacks the deviation or deficiency in the preliminary annotation, and provides an accurate optimization direction for the subsequent annotation steps. The multimodal annotation comparison data generated by parameter comparison can comprehensively reflect the difference between the preliminary annotation and the expert verification, and provide a quantitative basis for the further training of the annotation model. Finally, by performing ophthalmic data annotation on the multimodal annotation comparison data, the generated multimodal annotation data of ophthalmic cases has higher accuracy and reliability, and also lays a solid foundation for the continuous optimization of systematic annotation.

[0020] Preferably, the structured result generation and display module includes the following functions: Generate information extraction results from the ophthalmology electronic medical case information extraction model to generate ophthalmology electronic medical case information extraction results; perform field mapping on the ophthalmology electronic medical case information extraction results to generate ophthalmology electronic medical case field mapping data; sort the ophthalmology electronic medical case field mapping data in the time dimension to generate ophthalmology electronic medical case time dimension sorting data; Generate structured reports based on the time dimension sorting data of ophthalmic electronic medical records, and generate structured reports of ophthalmic electronic medical records information; Data visualization is performed on the structured report of ophthalmic electronic medical case information to generate visual charts of ophthalmic electronic medical cases, thereby completing the task of extracting ophthalmic information from electronic medical cases.

[0021] The present invention first generates the preliminary extraction results of electronic medical records through the information extraction model, which is the core link of data processing and ensures the accurate capture of key information in electronic medical records. Subsequently, the extraction results are normalized through the field mapping link, and the field data generated by mapping provides a standardized structural basis for subsequent analysis, ensuring the compatibility and consistency between different case data. Next, the field mapping data is sorted according to the time dimension to generate time dimension sorting data. This step introduces the organization mode of time series, so that the case information can reflect the time dynamics of the patient's diagnosis and treatment process, thereby supporting the tracking of the disease process and the analysis of the changing trend. Based on the time dimension sorting data, a structured report is further generated, so that the case information is displayed in a clear and organized manner, which is convenient for medical staff to quickly grasp the patient's condition and diagnosis and treatment history. Finally, by visualizing the data of the structured report, a visual chart is generated, and the complex electronic case information is converted into an intuitive graphical display form, such as a time axis, a trend curve, a statistical chart, etc., which greatly improves the efficiency of data interpretation. The overall process has achieved a gradual deepening from extraction to mapping, sorting, structured generation and visualization processing at the data level, which not only optimizes the expression of electronic medical record data, but also provides high-quality and easy-to-operate solutions for the application of medical data.

[0022] Preferably, the data visualization processing of the structured report of ophthalmic electronic medical information includes: Draw a timeline chart of the patient's full cycle view from the structured report of ophthalmology electronic medical records, mark key events, and generate a full cycle view of ophthalmology patients; Draw dynamic curves of examination results for structured reports of ophthalmic electronic medical records to generate dynamic curves of examination results for ophthalmic patients; The full cycle view of ophthalmological patients and the dynamic curve of ophthalmological patients' examination results are mapped into a medical history tree layered diagram to generate a visual chart of ophthalmic electronic medical records.

[0023] The present invention converts the time dimension information in the structured report into a visual expression with time continuity by drawing the time axis of the patient's full cycle view, so that the patient's diagnosis and treatment process, key inspection nodes and treatment adjustment plans are clearly presented, which is convenient for doctors to quickly grasp the overall picture of the patient's diagnosis and treatment. The introduction of key event markers further strengthens the hierarchy and emphasis of the data, helping clinical personnel to efficiently identify important diagnosis and treatment information in large amounts of data. Secondly, by drawing dynamic curves on the examination results, visualization results with strong continuity and clear trends are generated, such as dynamic curves of key parameters such as vision changes and intraocular pressure fluctuations, which not only improves the real-time monitoring ability of disease changes, but also provides a scientific basis for further trend prediction. Finally, by combining the full cycle view with the dynamic curve and presenting it in the form of a medical history tree-like hierarchical diagram, the generated chart comprehensively integrates the patient's longitudinal diagnosis and treatment trajectory and the horizontal examination result data, realizing the visual integrated display of multi-dimensional information. This processing method fundamentally optimizes the expression of complex medical data, overcomes the problems of scattered and non-intuitive traditional case data, and provides strong support for the rapidity and accuracy of medical decision-making.

[0024] The beneficial effect of the present invention is that the electronic medical record data acquisition and cleaning module standardizes the processing of raw data, provides a unified and accurate data basis for subsequent analysis, and avoids the problem of information loss caused by data redundancy, incompleteness or inconsistent format in traditional electronic medical records. Secondly, the domain knowledge-driven ophthalmic information extraction module constructs a preliminary information extraction model based on domain knowledge and uses professional databases for semantic relationship modeling, so that complex medical terms and related information can be efficiently parsed and extracted. This not only improves the accuracy of data extraction, but also provides support for the subsequent in-depth analysis of multimodal data. Thirdly, the multimodal data fusion and annotation module integrates electronic text cases and image cases to form multimodal fusion data, fully exploring the potential associations between different data types; at the same time, through annotation and model iteration, the information extraction model is further optimized, so that its recognition and processing capabilities for multimodal data are enhanced. Finally, the structured result generation and display module generates standardized reports based on the extraction results, and presents the dynamic characteristics and statistical trends of complex information through data visualization, which not only improves the efficiency of information expression, but also provides multi-dimensional data support for clinical decision-making. Therefore, the present invention solves the problems of incoherent data processing, inaccurate information extraction and difficult to intuitively express results in traditional ophthalmic case information extraction methods through full-process modular design and multimodal data fusion processing, thereby improving data utilization and clinical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a module diagram of the ophthalmic information extraction system based on electronic medical records; Figure 2 for Figure 1 Schematic diagram of the functional flow of the ophthalmic information extraction module driven by domain knowledge; Figure 3 for Figure 1 Schematic diagram of the functional flow of the structured result generation and display module.

[0026] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0027] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0028] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0029] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0030] To achieve this, please refer to Figures 1 to 3 , an ophthalmological information extraction system based on electronic medical records, the system comprises the following modules: The electronic medical record data acquisition and cleaning module is used to obtain the original case data; clean the original case data to generate the patient electronic case data set; The domain knowledge-driven ophthalmology information extraction module is used to obtain the ophthalmology professional database; initially create the knowledge graph of the ophthalmology professional database to generate the ophthalmology preliminary knowledge graph; construct the preliminary relationship information model of the ophthalmology preliminary knowledge graph to generate the ophthalmology preliminary information extraction model; The multimodal data fusion and annotation module is used to perform ophthalmic multimodal data fusion on ophthalmic electronic text cases and ophthalmic image cases to generate ophthalmic case multimodal fusion data; perform ophthalmic data annotation on ophthalmic case multimodal fusion data to generate ophthalmic case multimodal annotation data; perform model iteration on the ophthalmic preliminary information extraction model based on the ophthalmic case multimodal annotation data to generate an ophthalmic electronic case information extraction model; The structured result generation and display module is used to generate information extraction results from the ophthalmology electronic case information extraction model, generate ophthalmology electronic case information extraction results; generate structured reports based on the ophthalmology electronic case information extraction results, generate ophthalmology electronic case information structured reports; perform data visualization processing on the ophthalmology electronic case information structured reports, generate ophthalmology electronic case visualization charts, and thus complete the electronic case ophthalmology information extraction task.

[0031] The beneficial effect of the present invention is that the electronic medical record data acquisition and cleaning module standardizes the processing of raw data, provides a unified and accurate data basis for subsequent analysis, and avoids the problem of information loss caused by data redundancy, incompleteness or inconsistent format in traditional electronic medical records. Secondly, the domain knowledge-driven ophthalmic information extraction module constructs a preliminary information extraction model based on domain knowledge and uses professional databases for semantic relationship modeling, so that complex medical terms and related information can be efficiently parsed and extracted. This not only improves the accuracy of data extraction, but also provides support for the subsequent in-depth analysis of multimodal data. Thirdly, the multimodal data fusion and annotation module integrates electronic text cases and image cases to form multimodal fusion data, fully exploring the potential associations between different data types; at the same time, through annotation and model iteration, the information extraction model is further optimized, so that its recognition and processing capabilities for multimodal data are enhanced. Finally, the structured result generation and display module generates standardized reports based on the extraction results, and presents the dynamic characteristics and statistical trends of complex information through data visualization, which not only improves the efficiency of information expression, but also provides multi-dimensional data support for clinical decision-making. Therefore, the present invention solves the problems of incoherent data processing, inaccurate information extraction and difficult to intuitively express results in traditional ophthalmic case information extraction methods through full-process modular design and multimodal data fusion processing, thereby improving data utilization and clinical application value.

[0032] In the embodiment of the present invention, reference Figure 1 FIG. 1 is a schematic diagram of the steps of the ophthalmic information extraction system based on electronic medical records of the present invention. In this example, the ophthalmic information extraction system based on electronic medical records includes the following modules: Step S1: an electronic medical record data acquisition and cleaning module, used to acquire original case data; clean the original case data to generate a patient electronic case data set; In an embodiment of the present invention, by calling the medical information system (such as HIS, PACS) and the external database interface, the original case data is obtained using a standardized data transmission protocol (such as HL7, FHIR), thereby ensuring the compatibility and integrity of multi-source data. Subsequently, in the data cleaning phase, the text mining algorithm is used to perform semantic analysis and field mapping on the unstructured text part of the electronic case, and key diagnosis and treatment information (such as medical history description, examination results, diagnosis conclusions, etc.) is mapped to structured fields to eliminate text data redundancy and invalid information. For tabular and numerical data, an outlier detection algorithm (such as IQR detection or cluster-based outlier analysis) is applied to mark and correct abnormal data, and key indicator data (such as intraocular pressure, visual acuity, etc.) are verified according to the medical standard value range to eliminate misrecorded and missing values. At the same time, by verifying the timestamp of the data record, the time sequence consistency of the data is ensured, providing stable time dimension support for subsequent analysis. Finally, by merging the cleaned multi-dimensional data, a complete patient electronic medical record dataset is generated based on the patient's unique identifier. This dataset has the characteristics of high integrity, consistency and standardization, and can be directly used in subsequent information extraction and analysis processes.

[0033] Step S2: A domain knowledge-driven ophthalmology information extraction module is used to obtain an ophthalmology professional database; initially create a knowledge graph for the ophthalmology professional database to generate an ophthalmology preliminary knowledge graph; construct a preliminary relationship information model for the ophthalmology preliminary knowledge graph to generate an ophthalmology preliminary information extraction model; In an embodiment of the present invention, multiple professional databases in the field of ophthalmology (such as ophthalmology literature library, electronic medical record library, image library, etc.) are integrated, and data capture and extraction technologies (such as Web capture, API call, etc.) are used to obtain rich ophthalmology professional information from various data sources. Then, natural language processing (NLP) technology is used to pre-process these heterogeneous data, including data cleaning, noise removal and word segmentation, to ensure the effectiveness of subsequent processing. In order to construct a knowledge graph of ophthalmology, a graph database (such as Neo4j) is used in combination with the ontology of the ophthalmology field (such as ophthalmic diseases, treatment methods, drugs and other terminology systems) to create an initial knowledge graph. This process extracts entities from literature, guidelines and databases in the field of ophthalmology, and combines semantic relationships between entities (such as "cause", "treatment", "related to...") to construct nodes and edges in the graph to form a preliminary knowledge graph. Subsequently, based on the preliminary knowledge graph, machine learning and deep learning methods (such as BERT, Word2Vec, etc.) are used to semantically model the nodes and relationships in the knowledge graph to extract potential implicit relationships and structured information. Through iterative optimization of graph construction and information extraction, a preliminary information extraction model in the field of ophthalmology is generated. This model can not only identify entities such as diseases, symptoms, and drugs, but also accurately understand their semantic relationships, providing basic data support for subsequent case data analysis, decision support systems, and the formulation of personalized diagnosis and treatment plans.

[0034] Step S3: a multimodal data fusion and annotation module, which is used to perform ophthalmic multimodal data fusion on ophthalmic electronic text cases and ophthalmic image cases to generate ophthalmic case multimodal fusion data; perform ophthalmic data annotation on the ophthalmic case multimodal fusion data to generate ophthalmic case multimodal annotation data; perform model iteration on the ophthalmic preliminary information extraction model based on the ophthalmic case multimodal annotation data to generate an ophthalmic electronic case information extraction model; In an embodiment of the present invention, ophthalmic electronic text cases and ophthalmic imaging cases are integrated through multimodal data fusion technology. Ophthalmic electronic text cases include structured or semi-structured data such as patient medical records, diagnostic information, and treatment plans, while ophthalmic imaging cases include image data obtained by medical imaging equipment (such as OCT, fundus mirror, etc.). In order to achieve multimodal data fusion, the image and text joint embedding technology in deep learning (such as cross-modal learning, alignment learning, etc.) is adopted, and the image data features are extracted by using a multi-layer convolutional neural network (CNN), and the electronic text cases are extracted and understood by natural language processing (NLP) technology, and finally the two modal data are mapped and fused in the same vector space. This process needs to solve the heterogeneity problem of different data modalities, so that the fused data has a more comprehensive and rich expression ability. For the fused multimodal data of ophthalmic cases, data annotation technology is used to manually or automatically annotate the lesion area in the image data and the key information in the text data. This process can be trained through supervised learning methods in machine learning, and the annotation model can be trained using annotated samples (such as calibration of lesion areas, pathological analysis results, etc.). Through deep learning methods such as convolutional neural networks (CNN), local features in the image are extracted, and combined with text annotation information to generate high-quality multimodal annotated data of ophthalmic cases. These annotated data can effectively provide label data for subsequent information extraction and analysis. The ophthalmic preliminary information extraction model is iteratively optimized based on the multimodal annotated data of ophthalmic cases. In this stage, the incremental learning and transfer learning techniques are used to continuously update and optimize the model, so that the model can process newly input multimodal data more accurately. The model weights are adjusted through back-propagation algorithms and optimization algorithms (such as Adam, SGD, etc.) to generate a more efficient ophthalmic electronic case information extraction model, which can automatically extract key information such as diseases, treatment plans, and image features from fused data to provide high-quality clinical decision support.

[0035] Step S4: a structured result generation and display module is used to generate information extraction results from the ophthalmology electronic case information extraction model to generate ophthalmology electronic case information extraction results; to generate a structured report based on the ophthalmology electronic case information extraction results to generate an ophthalmology electronic case information structured report; to perform data visualization processing on the ophthalmology electronic case information structured report to generate an ophthalmology electronic case visualization chart, thereby completing the electronic case ophthalmology information extraction operation.

[0036] In an embodiment of the present invention, the ophthalmic electronic medical record information extraction model performs information extraction tasks by processing the input electronic medical record data. This process uses natural language processing (NLP) technology, such as named entity recognition (NER), relationship extraction, dependency syntax analysis, etc., to automatically extract key information from the case text, such as disease name, diagnosis results, treatment plan, drug use record, patient symptoms, examination results, etc. In order to further improve the accuracy of information extraction, pre-trained models (such as BERT, BioBERT, etc.) will be used for transfer learning. These models have been trained with a large amount of medical corpus and can understand the professional terms and context in the medical field, thereby achieving efficient information extraction at the data level. Next, a structured report is generated based on the key information extracted from the case. Structured report generation includes steps such as information classification, label mapping, and formatted output. By defining a standardized template or data model, the extracted medical information is classified and structured according to preset fields (such as basic patient information, diagnosis information, examination results, treatment plans, etc.). The key technical means of data structuring include data standardization and data formatting, which ensure that all extracted information meets the unified format requirements for subsequent storage and display. The generated structured report is visualized in the data visualization processing stage. Visualization technology uses graphics, tables, curves and other forms to show the changing trends and correlations of case data. For example, line charts, bar charts, pie charts and other forms can be used to show the patient's disease progression, drug use trends, changes in test results, etc. In order to achieve this goal, commonly used visualization tools and frameworks include D3.js, Plotly, ECharts, etc., which support complex data chart generation and can further enhance the user experience through interactive interfaces. In addition, the visualization of time series data can use dynamic visualization methods, such as data flow diagrams, trend charts, etc., to help doctors intuitively understand the patient's disease changes.

[0037] Preferably, the electronic medical record data acquisition and cleaning module includes the following functions: Obtain original case data; Unify the original case data into the HL7 standard information format to generate unified patient electronic case data; Perform preliminary metadata generation on the unified data of patient electronic medical records to generate electronic patient metadata case data; The data set is cleaned according to the patient's electronic metadata case data, and timestamp verification is performed to generate the patient's electronic case data set, which includes ophthalmology electronic text cases and ophthalmology imaging cases.

[0038] In an embodiment of the present invention, the process of obtaining the original case data is to extract data through a hospital information system (HIS) or an electronic medical record system (EMR). The original case data comes from a variety of formats, such as electronic medical records, imaging data, laboratory test reports, etc., covering the patient's basic information, medical history, diagnosis, treatment process, etc. These data are in different formats and lack a unified standard. In order to ensure the consistency and standardization of the data, the next step needs to be unified in the HL7 standard information format. HL7 (Health Level 7) is a data exchange standard widely used in the medical field. By formatting the original case data with the HL7 standard, the data can be converted into a unified standard format, so that it has good interoperability and interchangeability. This process maps the fields of the original data to the data elements of the HL7 standard and organizes them using the HL7 message structure (such as ORM, ADT, etc.), so that case data from different sources can be unified in data format, ensuring the standardization and availability of information. The unified data of the patient's electronic medical records will go through a preliminary metadata generation process. Metadata at this stage refers to additional information that describes the structure, attributes and source of the data. For example, by defining the fields, data types, data sources, update time and other meta-information of patient case data, the data is associated with its additional information to form a complete set of electronic metadata. These metadata provide a structured information framework for subsequent data processing, storage, retrieval and analysis. In the data set cleaning stage, the generated patient electronic metadata case data is first subjected to operations such as removing redundancy, supplementing missing values, and standardizing field formats to ensure the integrity and accuracy of the data. In addition, timestamp verification is also required, that is, verifying whether the time stamps in the data are consistent and accurate. Timestamp verification is crucial in medical data processing, especially when integrating multimodal data sources (such as images, texts, laboratory data, etc.), to ensure that records at different time nodes can be correctly aligned, thereby providing a basis for subsequent time series analysis.

[0039] As an example of the present invention, refer to Figure 2 As shown, in this example, the functions of the domain knowledge-driven ophthalmic information extraction module include: Step S21: Acquire an ophthalmology database; Step S22: Initially create an ophthalmology knowledge graph for the ophthalmology professional database to generate a preliminary ophthalmology knowledge graph; Step S23: Use the preset language model to extract semantic relationships from the preliminary ophthalmology knowledge graph, and optimize the preset language model to generate a preliminary ophthalmology information extraction model.

[0040] In an embodiment of the present invention, obtaining an ophthalmology professional database is the basis of the process, which comes from the hospital's electronic health record system, medical literature, professional databases (such as PubMed, Google Scholar, etc.) or special ophthalmology research data sets. These databases contain a large amount of text data, clinical records, medical history information, medical images, experimental results, etc. By accessing these data sources and extracting relevant information, building an ophthalmology professional database provides a data basis for the subsequent knowledge graph construction. Initial creation of the ophthalmology knowledge graph is performed. The construction of the knowledge graph depends on the fusion of structured data and unstructured data, and it is converted into a graph structure by defining domain-specific nodes (such as symptoms, treatment methods, drugs, equipment, experts, etc.) and their relationships (such as "treatment", "association", "influence"). In this process, a graph database (such as Neo4j, Amazon Neptune, etc.) is used to store and manage these nodes and relationships to ensure that the data can be represented in a graphical structure and can be quickly queried and reasoned. After the creation of the preliminary ophthalmology knowledge graph is completed, the data is presented as a multi-dimensional, semantically connected graph system. The BERT language model is used to extract semantic relationships from the knowledge graph. BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained language model based on the Transformer architecture. It is trained on a large corpus through unsupervised learning and can capture deep semantic information in text. In this process, the BERT model is used to analyze text data in the field of ophthalmology to identify and extract semantic relationships. For example, the model can identify the semantic relationship between symptoms and treatments, drugs and side effects, and diagnoses and symptoms. In this way, the model can extract potential medical information and associations and enrich the relationship hierarchy in the knowledge graph. The purpose of optimizing the preset BERT language model is to improve the performance of the model in a specific ophthalmology field. Since the application of the BERT model in the medical field requires targeted tuning, it can be retrained or fine-tuned through domain-customized annotated data to enable the model to more accurately understand and process specific terms and semantics in the field of ophthalmology. The optimized BERT model will be able to better perform information extraction tasks, such as automatically identifying important concepts in medical record texts, identifying the relationship between symptoms and treatments, and extracting key information from patient medical records, thereby generating a preliminary information extraction model for ophthalmology.

[0041] Preferably, the initial creation of the ophthalmological knowledge graph includes: Calibrate ophthalmic data features on the ophthalmic database to generate ophthalmic data structure features; define ophthalmic atlas nodes on the ophthalmic data structure features to generate professional ophthalmic atlas nodes; Map the node relationships according to the professional ophthalmology atlas nodes to generate ophthalmology data-atlas node data; Initially create an ophthalmic knowledge graph for ophthalmic data-graph node data to generate a preliminary ophthalmic knowledge graph.

[0042] In an embodiment of the present invention, the acquisition of ophthalmology professional databases provides basic data for subsequent analysis. These databases contain a large number of ophthalmology cases, diagnostic information, treatment plans, drug information and their interrelated clinical data, covering the clinical manifestations, causes, course of disease, treatment response, etc. of various ophthalmology diseases. On this basis, ophthalmology data feature calibration is the process of normalizing and labeling the original data, with the aim of providing structured and standardized feature information for subsequent graph construction. The data calibration process requires the application of natural language processing (NLP) technology to identify and extract key features in the text, such as disease names, symptoms, examination results, etc., and assign a unified standard identifier to these features. This process combines domain knowledge and medical terminology ontology, such as ICD (International Classification of Diseases) and SNOMED (Medical Terminology System) for data calibration to ensure that data features can be consistent across the entire field. The ophthalmology data structure features are used to define graph nodes, and the calibrated data features are converted into operable graph nodes. In this process, the nodes of the ophthalmology knowledge graph represent entities such as disease types, symptoms, treatment methods, and examination items, while the edges represent the relationships between them. For example, the node "glaucoma" is connected to the node "intraocular pressure" through an edge, indicating that the two are related. In this process, the definition of graph nodes needs to be combined with domain knowledge, and through expert experience and medical literature, determine which data features should be used as nodes and how the relationship between nodes should be established. Node relationship mapping is performed according to the nodes of the professional ophthalmology graph, that is, the specific semantic relationship between nodes is determined. The key technical means of this step is to represent these nodes and their relationships through graph construction algorithms (such as storage methods based on graph databases, such as Neo4j, ArangoDB, etc.). In the process of node relationship mapping, the semantic relationship between nodes will be converted into edges in the graph, such as the "treatment" edge connecting the treatment method and the disease, or the "influence" edge connecting a certain drug and its side effects. These relationships will constitute a rich semantic network in the knowledge graph, which is conducive to the structured expression of ophthalmic knowledge. Based on the above node relationship data, the initial creation of the ophthalmic knowledge graph is carried out to generate a preliminary ophthalmic knowledge graph. In this process, the final structure of the graph should not only accurately reflect the professional knowledge in the field of ophthalmology, but also have sufficient flexibility to facilitate subsequent expansion and knowledge updating.

[0043] Preferably, the semantic relationship extraction includes: Based on the preliminary ophthalmology knowledge graph, a change trend analysis is performed to generate the patient's ophthalmology case trend information; the patient's ophthalmology case trend information is analyzed for patient volatility to generate the patient's ophthalmology volatility information; the patient's ophthalmology case trend information and the patient's ophthalmology volatility information are used to generate a change trend graph to generate an ophthalmology patient change trend graph; Extract the semantic relationship between symptoms and intraocular pressure from the trend chart of ophthalmology patients, and generate the patient's symptom-intraocular pressure relationship data; extract the semantic relationship between complications and lesion points from the trend chart of ophthalmology patients, and generate the patient's complication-lesion point relationship data; The preset BERT language model is optimized based on the patient's symptom-intraocular pressure relationship data and the patient's complication-lesion point relationship data to generate a preliminary ophthalmic information extraction model.

[0044] In an embodiment of the present invention, the purpose of performing trend analysis based on the preliminary ophthalmology knowledge graph is to reveal the change patterns of different features (such as intraocular pressure, lesions, treatment responses, etc.) in the patient's ophthalmology cases over time. This process relies on time series analysis methods and statistical models, such as ARIMA models or long short-term memory (LSTM) networks, to identify and predict the change trends of key features in ophthalmology cases, and then generate trend information of patient ophthalmology cases. The ophthalmology information (such as intraocular pressure, visual acuity, fundus images, etc.) of each patient in the data set shows certain regularities over time, which can be revealed by trend analysis and converted into time series data. These trend data can be further compared with the patient's historical medical records and clinical records to help identify the evolution of the disease and the future development direction. In terms of patient volatility analysis, the main purpose is to evaluate the volatility of the data in the patient's case, especially focusing on the amplitude and frequency of changes in the condition. The commonly used technical means in this process are variance analysis and standard deviation calculation in statistics, which evaluate the volatility of ophthalmology data (such as intraocular pressure) at different time points. This volatility data can help clinicians understand the stability of the patient's disease and adjust the treatment plan in time. After that, the trend information and volatility information of the patient are combined to generate a trend chart. Through data visualization technology, such as Matplotlib, Plotly and other tools, a graphical view that can clearly show the changes of the patient's condition over time is generated, which helps clinical staff to intuitively analyze the development trend and fluctuation pattern of the disease. The next semantic relationship extraction process is based on the trend chart of the change of the patient's ophthalmological case to extract the relationship between symptoms and intraocular pressure and complications and lesions. This technology relies on the relationship extraction technology in natural language processing (NLP), and uses deep learning methods such as dependency syntactic analysis, named entity recognition (NER), and graph convolutional network (GCN) to extract the relationship between symptoms and intraocular pressure (such as the relationship between increased intraocular pressure and glaucoma) and the relationship between complications and lesions (such as retinal lesions caused by complications) from case data. These semantic relationship extraction technologies extract key medical semantic information by analyzing words, phrases and their relationships in the text. The extracted relationship data can be further converted into structured data forms, such as relationship triples (symptoms-intraocular pressure, complications-lesions) for subsequent analysis or reasoning. In order to improve the accuracy and efficiency of information extraction, the preset BERT language model is used to optimize the above semantic relationship extraction data. The BERT (Bidirectional Encoder Representations from Transformers) model is a deep learning model based on the Transformer architecture, which is particularly suitable for processing semantic understanding and reasoning tasks of long texts.In ophthalmology data, BERT can be optimized through a fine-tuning process combined with a domain-specific ophthalmology corpus, enabling the model to better understand specific terms and semantic relationships in ophthalmology cases, thereby generating a high-quality preliminary information extraction model.

[0045] Preferably, the multimodal data fusion and annotation module includes the following functions: Perform ophthalmic multimodal data fusion on ophthalmic electronic text cases and ophthalmic imaging cases to generate ophthalmic case multimodal fusion data; Perform ophthalmic data annotation on multimodal fusion data of ophthalmic cases to generate multimodal annotation data of ophthalmic cases; Perform quantitative evaluation on the annotation results of multimodal annotation data of ophthalmic cases to generate quantitative evaluation data of multimodal annotation results of ophthalmic cases; iterate the ophthalmic preliminary information extraction model based on the quantitative evaluation data of multimodal annotation results of ophthalmic cases to generate an ophthalmic electronic case information extraction model.

[0046] In an embodiment of the present invention, the fusion of ophthalmic electronic text cases and ophthalmic image cases is achieved through multimodal data fusion technology, which requires the combination of structured electronic case text data and unstructured image data (such as fundus images, retinal scans, etc.). The text data is extracted by natural language processing (NLP) technology, while the image data is extracted by image processing technology, such as convolutional neural network (CNN). The fusion method can adopt a multimodal deep learning model, especially a joint learning model, which can process text and image data at the same time to generate unified multimodal fusion data, providing comprehensive information support for subsequent analysis and annotation. The multimodal fusion data of ophthalmic cases is annotated to generate structured annotated data. This step relies on expert annotation, semi-supervised learning or active learning technology, and automatically generates annotation information through manual annotation or machine learning algorithms (such as deep learning models). Based on this annotated data, a quantitative evaluation method of the annotation results can be used to perform quality control and optimization to evaluate the accuracy and consistency of data annotation. Common quantitative evaluation methods include calculating the accuracy, recall, F1 value and other indicators of the annotation results, or using consistency analysis to check the consistency between the annotation results of different annotators or algorithms, thereby generating quantitative evaluation data for the annotation results. These evaluation results are crucial to improving data quality, and they provide a feedback mechanism for optimizing the annotation process. On the basis of data annotation and evaluation, the model iteration of the preliminary information extraction model of ophthalmology based on the quantitative evaluation data of the annotation results is to improve the accuracy and effect of information extraction. This process uses model training and verification techniques to train the existing information extraction model using the annotated data, and uses the quantitative evaluation results to adjust the model parameters, optimize the model structure, and gradually improve the performance of the model. The iterative process uses technologies such as reinforcement learning or transfer learning to enable the model to continuously improve based on the feedback from the annotated data, thereby generating a more accurate ophthalmic electronic medical record information extraction model.

[0047] Preferably, the ophthalmic multimodal data fusion includes: Classify the ophthalmology electronic case texts to generate ophthalmology electronic case classification data; Perform optical coherence tomography on ophthalmic imaging cases to generate ophthalmic imaging scan data; perform lesion area marking on ophthalmic imaging scan data to generate ophthalmic imaging lesion marking data; perform thickness segmentation masking on ophthalmic imaging lesion marking data to generate ophthalmic lesion imaging layer thickness data; The ophthalmic electronic medical record classification data and the ophthalmic lesion image layer thickness data are fused into ophthalmic multimodal data to generate ophthalmic case multimodal fusion data.

[0048] In an embodiment of the present invention, for the classification of ophthalmic electronic text cases, natural language processing (NLP) technology can be used to extract and classify text features using machine learning algorithms. Common technical means include text vectorization methods such as TF-IDF (term frequency-inverse document frequency) or Word2Vec, and then the text is automatically classified using a classification model (such as a support vector machine, a random forest, or a deep neural network) to generate classified data for ophthalmic electronic cases. These classified data can provide a structured information basis for subsequent case analysis. Secondly, for the processing of ophthalmic imaging cases, especially the processing of optical coherence tomography (OCT) images, commonly used technologies include image processing and deep learning methods. Through deep learning models such as convolutional neural networks (CNNs), key information in OCT images, such as the location and morphological characteristics of the lesion area, can be extracted, and the lesion area can be marked by an image segmentation algorithm to generate ophthalmic image lesion marking data. This process can effectively distinguish normal areas from abnormal areas in the image and provide information about the spatial distribution of lesions. Next, the ophthalmic image lesion marking data is subjected to thickness segmentation mask processing, and deep learning-based image segmentation methods such as U-Net or Mask R-CNN are used. These methods can generate accurate layered thickness data, and by modeling the thickness distribution of the lesion area, the detailed description of the characteristics of the lesion area is further improved. Finally, the ophthalmic electronic case classification data and the ophthalmic lesion image layered thickness data are multimodally fused. The common technical means is to fuse the two data types (text data and image data) into a unified representation through a multimodal deep learning network (such as a fused convolutional neural network, a Transformer model, etc.). This fusion method can comprehensively utilize the structured information of text data and the visual features of image data to generate a more comprehensive multimodal data set, further providing accurate support for the automated analysis and diagnosis of ophthalmic cases.

[0049] Preferably, the ophthalmic data annotation includes: Perform preliminary annotation processing based on multimodal fusion data of ophthalmology cases to generate preliminary multimodal annotation results of ophthalmology; upload the preliminary multimodal annotation results of ophthalmology to the cloud platform for correction and verification, and generate preliminary ophthalmology annotation expert verification results; Compare the preliminary annotation parameters based on the ophthalmology preliminary annotation expert verification results and the ophthalmology multimodal preliminary annotation results, and transmit the parameters back to generate multimodal annotation comparison data of ophthalmology cases; The ophthalmic data is annotated for the multimodal annotated comparison data of ophthalmic cases to generate multimodal annotated data of ophthalmic cases.

[0050] In an embodiment of the present invention, the technical means for preliminary annotation processing based on multimodal fusion data of ophthalmic cases can be completed by combining an automated annotation system with a deep learning method. Specifically, a convolutional neural network (CNN) or a multimodal learning model is used to process ophthalmic imaging data (such as OCT images) and text data (such as diagnostic reports), extract key information and annotate them. These annotations can be based on predefined tags, and the deep neural network is trained using the annotated data set to generate preliminary annotation results. These preliminary annotation results are then uploaded to the cloud platform. This process data storage and transmission technology uses the elastic storage and computing resources of the cloud computing platform to correct and verify the annotated data, and generates verification results using expert feedback. The verification process is generally to establish a verification mechanism for the annotated data, combined with the opinions of experts, and to correct and confirm the annotated data manually and automatically. After the verification data is compared with the preliminary annotated data, the technical means parameterizes the data comparison method, including data difference analysis and feedback processing. These methods can ensure data accuracy and optimize the annotation results through a feedback mechanism. Next, the multimodal annotation comparison data of ophthalmology cases is further annotated and processed, using an advanced data annotation platform, combined with automated annotation tools and manual review to further improve the annotation of multimodal data. This process uses reinforcement learning or semi-supervised learning models in machine learning to improve the accuracy and consistency of annotations through continuous iterative updates, thereby ultimately generating accurate multimodal annotation data for ophthalmology cases.

[0051] As an example of the present invention, refer to Figure 3 As shown, in this example, the functions of the structured result generation and display module include: Step S41: Generate information extraction results from the ophthalmology electronic medical case information extraction model to generate ophthalmology electronic medical case information extraction results; perform field mapping on the ophthalmology electronic medical case information extraction results to generate ophthalmology electronic medical case field mapping data; sort the ophthalmology electronic medical case field mapping data in the time dimension to generate ophthalmology electronic medical case time dimension sorting data; Step S42: generating a structured report based on the time dimension sorting data of the ophthalmology electronic medical records, and generating a structured report of the ophthalmology electronic medical records information; Step S43: Perform data visualization processing on the structured report of the ophthalmic electronic medical case information to generate an ophthalmic electronic medical case visualization chart, thereby completing the electronic medical case ophthalmic information extraction operation.

[0052] In an embodiment of the present invention, the design of the ophthalmic electronic case information extraction model mainly uses models such as convolutional neural networks (CNN) or long short-term memory networks (LSTM) in deep learning to process and extract information from ophthalmic case text data. At this stage, by performing operations such as word segmentation, named entity recognition (NER), and relationship extraction on the case text, key ophthalmological disease information, diagnosis results, examination data, and other information are extracted to generate preliminary electronic case information extraction results. Then, these information extraction results will be processed by field mapping technology. Field mapping is to match the extracted text fields with standard database fields through mapping rules, and use rule engines or machine learning methods (such as support vector machines SVM or decision tree algorithms) to implement this process, thereby generating field mapping data for ophthalmic electronic cases. Subsequently, based on the field mapping data, the time information in the case is further processed, timestamp correction and time series analysis are performed through time dimension sorting technology. This process can be implemented through a data processing framework based on a sorting algorithm (such as MapReduce) to generate standardized time dimension sorting data for ophthalmic electronic cases. On this basis, these data are further processed with the help of data structured report generation technology. The generation of structured reports can use template matching technology, combined with case data, to automatically generate standardized reports through preset templates, reduce manual intervention, and improve the accuracy and efficiency of reports. Finally, the generated ophthalmic electronic case information structured report will be processed through data visualization. Commonly used technologies include data visualization libraries (such as D3.js, Tableau, etc.), using data charts, trend charts, etc. to intuitively display case information, helping doctors quickly obtain key information during clinical decision-making.

[0053] Preferably, the ophthalmic data annotation includes: Draw a timeline chart of the patient's full cycle view from the structured report of ophthalmology electronic medical records, mark key events, and generate a full cycle view of ophthalmology patients; Draw dynamic curves of examination results for structured reports of ophthalmic electronic medical records to generate dynamic curves of examination results for ophthalmic patients; The full cycle view of ophthalmological patients and the dynamic curve of ophthalmological patients' examination results are mapped into a medical history tree layered diagram to generate a visual chart of ophthalmic electronic medical records.

[0054] In an embodiment of the present invention, the drawing of the timeline table of the patient's full cycle view depends on the accurate processing of the timestamp in the patient's electronic medical record data. At the data level, the diagnosis and treatment records of the case can be sorted in the time dimension by a time-based sorting algorithm (such as sorting tree, quick sort, etc.), and all key events (such as diagnosis, treatment, follow-up, etc.) can be marked and drawn on the timeline. This process uses a time series analysis method to map the events of each time node to the corresponding timestamp, and clearly displays the full cycle of the patient from the initial diagnosis to the final diagnosis and treatment through the timeline. In addition, key event marking can use data labeling technology to mark important events or changes related to the disease (such as worsening of the disease, treatment response, etc.) on the timeline to help doctors quickly obtain important information. In the drawing process of the dynamic curve, the changes in the examination results are processed by time series data. Examination data, such as intraocular pressure values ​​of ophthalmic examinations, retinal image analysis results, etc., can be smoothed based on statistical analysis methods (such as regression analysis, moving average, etc.) and drawn into a dynamic curve chart. Using data fitting algorithms, such as polynomial fitting and interpolation, the relationship between time and examination results is modeled to generate a dynamic trend chart of examination results, which can reflect the patient's condition changes and treatment effects. The generation of a medical history tree layered diagram is to represent the patient's disease course by constructing a multi-level tree diagram structure. At the data level, the construction of the tree diagram relies on graphical algorithms, especially hierarchical algorithms based on graph theory (such as hierarchical clustering algorithms, K-means algorithms, etc.), which layer and display the patient's medical history events according to categories and associations. These medical history events, including different diagnoses, treatment plans, examination results, etc., form nodes and are displayed hierarchically through a tree structure, allowing doctors to intuitively see the full picture of the patient's disease course.

[0055] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0056] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. An ophthalmic information extraction system based on electronic medical records, characterized in that: Includes the following modules: The electronic medical record data acquisition and cleaning module is used to obtain the original case data; clean the original case data to generate the patient electronic case data set; Domain knowledge-driven ophthalmology information extraction module, used to obtain ophthalmology professional database; Initially create a knowledge graph for the ophthalmology professional database to generate a preliminary ophthalmology knowledge graph; construct a preliminary relationship information model for the preliminary ophthalmology knowledge graph to generate a preliminary ophthalmology information extraction model; The multimodal data fusion and annotation module is used to perform ophthalmic multimodal data fusion on ophthalmic electronic text cases and ophthalmic image cases to generate ophthalmic case multimodal fusion data; perform ophthalmic data annotation on ophthalmic case multimodal fusion data to generate ophthalmic case multimodal annotation data; Based on the multimodal annotated data of ophthalmic cases, the ophthalmic preliminary information extraction model is iterated to generate an ophthalmic electronic case information extraction model; The structured result generation and display module is used to generate information extraction results from the ophthalmology electronic medical record information extraction model and generate ophthalmology electronic medical record information extraction results; Generate structured report based on the extraction results of ophthalmic electronic medical records information; Data visualization is performed on the structured report of ophthalmic electronic medical case information to generate visual charts of ophthalmic electronic medical cases, thereby completing the task of extracting ophthalmic information from electronic medical cases.

2. The ophthalmic information extraction system based on electronic medical records according to claim 1, characterized in that: The electronic medical record data acquisition and cleaning module includes the following functions: Obtain original case data; Unify the original case data into the HL7 standard information format to generate unified patient electronic case data; Perform preliminary metadata generation on the unified data of patient electronic medical records to generate electronic patient metadata case data; The data set is cleaned according to the patient's electronic metadata case data, and timestamp verification is performed to generate the patient's electronic case data set, which includes ophthalmology electronic text cases and ophthalmology imaging cases.

3. The ophthalmic information extraction system based on electronic medical records according to claim 1, characterized in that: The domain knowledge-driven ophthalmic information extraction module includes the following functions: Access to the ophthalmology specialty database; Initially create an ophthalmology knowledge graph for the ophthalmology professional database and generate a preliminary ophthalmology knowledge graph; The preset language model is used to extract semantic relationships from the preliminary ophthalmology knowledge graph, and the preset language model is optimized to generate a preliminary ophthalmology information extraction model.

4. The system for predicting postoperative liver transplantation according to claim 3, characterized in that: The initial creation of the ophthalmology knowledge graph includes: Calibrate ophthalmic data features on the ophthalmic database to generate ophthalmic data structure features; define ophthalmic atlas nodes on the ophthalmic data structure features to generate professional ophthalmic atlas nodes; Node relationship mapping is performed according to professional ophthalmology atlas nodes to generate ophthalmology data-atlas node data; ophthalmology knowledge graph is initially created for ophthalmology data-atlas node data to generate a preliminary ophthalmology knowledge graph.

5. The ophthalmic information extraction system based on electronic medical records according to claim 3, characterized in that: The semantic relationship extraction includes: Based on the preliminary ophthalmology knowledge graph, a change trend analysis is performed to generate the patient's ophthalmology case trend information; the patient's ophthalmology case trend information is analyzed for patient volatility to generate the patient's ophthalmology volatility information; the patient's ophthalmology case trend information and the patient's ophthalmology volatility information are used to generate a change trend graph to generate an ophthalmology patient change trend graph; Extract the semantic relationship between symptoms and intraocular pressure from the trend chart of ophthalmology patients, and generate the patient's symptom-intraocular pressure relationship data; extract the semantic relationship between complications and lesion points from the trend chart of ophthalmology patients, and generate the patient's complication-lesion point relationship data; The preset BERT language model is optimized based on the patient's symptom-intraocular pressure relationship data and the patient's complication-lesion point relationship data to generate a preliminary ophthalmic information extraction model.

6. The ophthalmic information extraction system based on electronic medical records according to claim 1, characterized in that: The multimodal data fusion and annotation module includes the following functions: Perform ophthalmic multimodal data fusion on ophthalmic electronic text cases and ophthalmic imaging cases to generate ophthalmic case multimodal fusion data; Perform ophthalmic data annotation on multimodal fusion data of ophthalmic cases to generate multimodal annotation data of ophthalmic cases; Perform quantitative evaluation on the annotation results of multimodal annotation data of ophthalmic cases to generate quantitative evaluation data of multimodal annotation results of ophthalmic cases; iterate the ophthalmic preliminary information extraction model based on the quantitative evaluation data of multimodal annotation results of ophthalmic cases to generate an ophthalmic electronic case information extraction model.

7. The ophthalmic information extraction system based on electronic medical records according to claim 6, characterized in that: The ophthalmic multimodal data fusion includes: Classify the ophthalmology electronic case texts to generate ophthalmology electronic case classification data; Perform optical coherence tomography on ophthalmic imaging cases to generate ophthalmic imaging scan data; perform lesion area marking on ophthalmic imaging scan data to generate ophthalmic imaging lesion marking data; perform thickness segmentation masking on ophthalmic imaging lesion marking data to generate ophthalmic lesion imaging layer thickness data; The ophthalmic electronic medical record classification data and the ophthalmic lesion image layer thickness data are fused into ophthalmic multimodal data to generate ophthalmic case multimodal fusion data.

8. The ophthalmic information extraction system based on electronic medical records according to claim 6, characterized in that: The ophthalmic data annotation includes: Perform preliminary annotation processing based on multimodal fusion data of ophthalmology cases to generate preliminary multimodal annotation results of ophthalmology; upload the preliminary multimodal annotation results of ophthalmology to the cloud platform for correction and verification, and generate preliminary ophthalmology annotation expert verification results; Compare the preliminary annotation parameters based on the ophthalmology preliminary annotation expert verification results and the ophthalmology multimodal preliminary annotation results, and transmit the parameters back to generate multimodal annotation comparison data of ophthalmology cases; The ophthalmic data is annotated for the multimodal annotated comparison data of ophthalmic cases to generate multimodal annotated data of ophthalmic cases.

9. The ophthalmic information extraction system based on electronic medical records according to claim 1, characterized in that: The structured result generation and display module includes the following functions: Generate information extraction results from the ophthalmology electronic medical case information extraction model to generate ophthalmology electronic medical case information extraction results; perform field mapping on the ophthalmology electronic medical case information extraction results to generate ophthalmology electronic medical case field mapping data; sort the ophthalmology electronic medical case field mapping data in the time dimension to generate ophthalmology electronic medical case time dimension sorting data; Generate structured reports based on the time dimension sorting data of ophthalmic electronic medical records, and generate structured reports of ophthalmic electronic medical records information; Data visualization is performed on the structured report of ophthalmic electronic medical case information to generate visual charts of ophthalmic electronic medical cases, thereby completing the task of extracting ophthalmic information from electronic medical cases.

10. The ophthalmic information extraction system based on electronic medical records according to claim 9, characterized in that: The data visualization processing includes: Draw a timeline chart of the patient's full cycle view from the structured report of ophthalmology electronic medical records, mark key events, and generate a full cycle view of ophthalmology patients; Draw dynamic curves of examination results for structured reports of ophthalmic electronic medical records to generate dynamic curves of examination results for ophthalmic patients; The full cycle view of ophthalmological patients and the dynamic curve of ophthalmological patients' examination results are mapped into a medical history tree layered diagram to generate a visual chart of ophthalmic electronic medical records.

Images