Intelligent treatment-based diagnosis and treatment data warehouse construction method and system

Abstract

The invention discloses a diagnosis and treatment data warehouse construction method and system based on intelligent governance, and the method comprises the steps: automatically converting an unstructured medical text into standardized data, integrating the standardized data with structured data, and automatically generating a data processing link based on an integrated data mode and a governance rule, when the data processing link is executed to construct the diagnosis and treatment data warehouse, data quality verification and safety control are synchronously implemented according to logic integrated on the link. The problem of data recycling failure caused by insufficient data understanding ability and absent governance mechanism of a medical data management system is solved, and the diagnosis and treatment data can be efficiently and reliably put into actual business by constructing the diagnosis and treatment data warehouse based on intelligent governance.

Description

Technical Field

[0001] This application relates to the field of data management technology, and in particular to a method and system for constructing a medical data warehouse based on intelligent governance. Background Technology

[0002] With the development of medical informatization, various medical institutions have accumulated a massive amount of clinical and management data. This data is scattered across different business systems such as hospital information systems, electronic medical record systems, laboratory information systems, and image archiving and communication systems. Due to system heterogeneity and inconsistent standards, a serious data silo phenomenon has been formed.

[0003] To integrate, manage, and fully utilize this medical data to support advanced applications such as clinical research, refined hospital management, and public health decision-making, related technologies acquire data from source systems, clean and process the acquired data to meet target requirements, and then load the transformed data into the target database.

[0004] However, this data integration solution based on preset rules and fixed processes exhibits shortcomings such as low automation and intelligence, rigid processes, and insufficient governance capabilities when faced with multi-source, heterogeneous, multimodal, and rapidly evolving business data in the medical field. It fails to meet the core requirements of modern medical systems for data real-time performance, accuracy, and security, and severely restricts the efficient release and in-depth utilization of medical data. Summary of the Invention

[0005] This application provides a method and system for constructing a medical data warehouse based on intelligent governance. This solves the problem of insufficient data understanding capabilities and lack of governance mechanisms in the medical data management system, which leads to the failure of data resource utilization. By constructing a medical data warehouse based on intelligent governance, medical data can be efficiently and reliably put into actual business.

[0006] This application provides a method for constructing a medical data warehouse based on intelligent governance, the method comprising: Acquire raw data from multiple heterogeneous data sources, including unstructured text data; The unstructured text data is subjected to medical entity recognition and standard terminology mapping to generate standardized data with standard coding. Based on the standardized data data pattern, the preset data warehouse target pattern, and the data governance rules associated with the data warehouse target pattern, a data processing chain is generated; The data processing chain is executed to organize and store the integrated data according to the target data warehouse model. During the execution process, data governance is implemented synchronously according to the logic on the data processing chain to complete the construction of the diagnosis and treatment data warehouse.

[0007] Optionally, the step of performing medical entity recognition and standard term mapping on the unstructured text data includes: The unstructured text data is used to perform medical entity recognition through an entity recognition and mapping model, and the recognition results are converted into semantic vectors. Calculate the semantic similarity between the semantic vector and the term vectors in the standard term knowledge graph, and generate at least one candidate standard code and its corresponding confidence score; If the confidence level is higher than a preset threshold, then the corresponding candidate standard code is adopted; If the confidence level is lower than the preset threshold, the medical entity and the candidate standard code are pushed to the verification platform, and the verification result is fed back to the entity recognition and mapping model for iterative optimization of the model.

[0008] Optionally, the step of integrating the structured data and the standardized data, and generating a data processing chain based on the data pattern of the integrated data, a preset data warehouse target pattern, and data governance rules associated with the data warehouse target pattern, includes: The data pattern of the integrated data and the preset data warehouse target pattern are parsed to obtain the field mapping relationship and data transformation logic from the standardized data to the data defined by the data warehouse target pattern; Data quality verification rules and data security control strategies are obtained from the data governance rules associated with the target schema of the data warehouse. Based on the field mapping relationship, the data conversion logic, the data quality verification rules, and the data security control strategy, a task sequence required to realize data conversion, verification, and control, as well as the execution dependencies between the task sequences, are generated. Based on the task sequence and the execution dependencies, a directed acyclic graph consisting of executable tasks is generated as the data processing link.

[0009] Optionally, the step of parsing the data pattern of the integrated data and the preset data warehouse target pattern to obtain the field mapping relationship and data transformation logic from the standardized data to the data defined by the data warehouse target pattern includes: The structure defined by the data pattern of the integrated data is used as the source structure for data processing; The structure defined by the target schema of the data warehouse is used as the target structure for data processing; Based on the source structure and the target structure, establish a field mapping relationship from the source structure to the target structure, and define the data transformation logic required to implement the field mapping relationship.

[0010] Optionally, the step of generating the task sequence required to implement data conversion, verification, and control, and the execution dependencies between the task sequences, based on the field mapping relationship, the data conversion logic, the data quality verification rules, and the data security control strategy, includes: The field mapping relationship and the data transformation logic are transformed into a data transformation task; the data quality verification rules are transformed into a data quality verification task; and the data security control strategy is transformed into a data security control task. The data transformation task, the data quality verification task, and the data security control task are taken as a task set, and the execution order of the tasks is determined based on the data input and output relationship between the tasks in the task set, thereby generating the task sequence and the execution dependency relationship.

[0011] Optionally, the steps of executing the data processing link, organizing and storing the integrated data according to the data warehouse target model, and synchronously implementing data governance according to the logic on the data processing link during execution to complete the construction of the diagnosis and treatment data warehouse include: According to the data processing link, data transformation tasks are scheduled and executed to organize and store the integrated data according to the data warehouse target pattern; When scheduling and executing the data transformation task, data quality verification and data security control tasks are simultaneously scheduled and executed according to the logic of the data processing link; wherein, Executing the data quality verification task includes: invoking the data quality verification rules bound to the data quality verification task to verify the data in transit; Performing the data security control task includes: invoking the data security control policy bound to the data security control task, and performing dynamic desensitization or encryption on sensitive fields defined by the data security control policy.

[0012] Furthermore, to achieve the above objectives, embodiments of the present invention also provide a system for constructing a medical data warehouse based on intelligent governance, the system comprising: The data access and adaptation module is used to connect to various heterogeneous data sources and to perform protocol parsing and format conversion on data from different data sources with different protocols or formats, and output raw data with a unified specification. The raw data includes structured data and unstructured text data. The data standardization processing module is communicatively connected to the data access and adaptation module, and is used to perform medical entity recognition and standard term mapping on the unstructured text data to generate standardized data with standard codes. The link construction and generation module is communicatively connected to the data standardization processing module. It is used to integrate the structured data and the standardized data, and generate a data processing link based on the data pattern of the integrated data, the preset data warehouse target pattern, and the data governance rules associated with the data warehouse target pattern. The data construction and governance execution module is communicatively connected to the link construction and generation module. It is used to execute the data processing link, organize and store the integrated data according to the data warehouse target pattern, and synchronously implement data governance according to the logic integrated on the data processing link during the execution process to complete the construction of the diagnosis and treatment data warehouse.

[0013] Optionally, the data standardization processing module includes: The natural language processing unit is used to perform medical named entity recognition on the unstructured text data; The semantic mapping unit is used to convert the identified medical entities into vectors and generate candidate standard codes and their confidence scores by calculating the semantic similarity with the standard medical terminology knowledge graph. The adjudication unit is used to push the medical entity and the candidate standard code to the verification platform for adjudication when the confidence level is lower than a preset threshold.

[0014] Optionally, the link construction and generation module includes: The pattern parsing unit is used to parse the data pattern of the integrated data and the preset data warehouse target pattern to obtain the field mapping relationship and data transformation logic from the integrated data to the data defined by the data warehouse target pattern; The rule acquisition unit is used to acquire data quality verification rules and data security control strategies from the data governance rules associated with the target schema of the data warehouse; The task derivation unit is used to derive, based on the field mapping relationship, the data conversion logic, the data quality verification rules, and the data security control strategy, the task sequence required to realize data conversion, verification, and control, and the execution dependency relationship between the task sequences. The link generation unit is used to generate a directed acyclic graph consisting of executable tasks based on the task sequence and the execution dependencies, as the data processing link.

[0015] Optionally, the data construction and governance execution module includes a governance logic executor, which is used to dynamically call the data quality verification rules bound to the data quality verification task to perform data verification when executing the data processing link, and to call the data security control policy bound to the data security control task to perform data desensitization or encryption.

[0016] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. A unified access mechanism breaks down data silos, centralizing structured and unstructured data scattered across various business systems and resolving the issue of physical data dispersion. Furthermore, natural language processing technology is used to parse unstructured text data and map it to standard terms, addressing the problems of logical heterogeneity and semantic ambiguity. This significantly improves data usability, transforming descriptive clinical records into quantifiable and analyzable structured information, providing data raw materials for subsequent automated processing and in-depth governance.

[0017] 2. By analyzing data models and governance rules, a complete data processing chain is automatically derived and constructed, transforming data warehouse construction from relying on manual coding to automated design. This ensures that every governance requirement is solidified into specific execution tasks during the process design phase, guaranteeing the standardization and reliability of data processing.

[0018] 3. While executing the data processing chain, data quality verification and security control tasks are triggered synchronously according to the logic integrated on the chain, ensuring that governance actions and data processing are integrated. This mechanism, which transforms governance rules into built-in processes, changes the lag of ex-post governance, ensuring data quality and security in real time during data flow, and ultimately building a truly reliable medical data warehouse. Attached Figure Description

[0019] Figure 1 This is a schematic diagram illustrating the framework of the intelligent governance-based diagnostic and treatment data warehouse construction method of this application; Figure 2 This is a flowchart illustrating the method for constructing a medical data warehouse based on intelligent governance as described in this application. Figure 3 This is a comparison diagram of the medical data table structure before and after processing in the method for constructing a medical data warehouse based on intelligent governance in this application. Figure 4 A system architecture diagram for the intelligent governance-based diagnosis and treatment data warehouse of this application is constructed; Figure 5 This is the architecture diagram of the data access and adaptation module for this application; Figure 6 This is a diagram of the data standardization processing module architecture for this application; Figure 7 This is a diagram illustrating the module architecture for the link construction and generation in this application. Figure 8 This is a diagram of the metadata management and monitoring platform architecture for this application; Figure 9 This is a schematic diagram of the terminal structure of the hardware operating environment involved in one embodiment of this application. Detailed Implementation

[0020] This application addresses the data warehouse governance failures caused by the inability of medical data integration solutions to intelligently handle unstructured text, rigid processes, and disconnected governance links. It proposes a method for constructing a medical data warehouse based on intelligent governance. First, unstructured medical text is automatically converted into standardized data using natural language processing technology. Then, a data processing chain is automatically generated based on data models and governance rules. Finally, while constructing the data warehouse using this chain, data quality verification and security control are implemented simultaneously and automatically. This achieves the integration and automation of data governance and data processing processes, resulting in the technical effects of ensuring data quality and security from the source, significantly improving construction efficiency, and deeply activating the value of medical data.

[0021] To better understand the above technical solutions, exemplary embodiments of this application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.

[0022] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0023] Example 1 In this embodiment, a method for constructing a diagnosis and treatment data warehouse based on intelligent governance is provided.

[0024] Reference Figure 1 and Figure 2 The method for constructing a diagnosis and treatment data warehouse based on intelligent governance in this embodiment includes the following steps: Step S100: Obtain raw data from multiple heterogeneous data sources, the raw data including structured data and unstructured text data; In this embodiment, the diagnostic and treatment data is distributed across different business systems, such as hospital information systems, electronic medical record systems, laboratory information systems, and image archiving and communication systems. The data sources are heterogeneous, the formats are diverse, and the standards are inconsistent; therefore, it is necessary to uniformly receive and preprocess this data.

[0025] Optionally, the raw data may include not only structured data and structured text data, but also semi-structured data such as XML and JSON logs.

[0026] As an optional implementation method, data can be acquired in two modes: timed batch acquisition and real-time streaming acquisition. Timed batch acquisition is suitable for scenarios with low real-time requirements but large data volume and complex calculations, while real-time streaming acquisition is suitable for scenarios with high real-time requirements.

[0027] As another optional implementation, after acquiring the raw data, preliminary processing and in-depth analysis are performed. First, basic data cleaning is completed through operations such as deduplication, character encoding removal, and format standardization. For unstructured text data that cannot be standardized through simple data cleaning, such as medical records, diagnostic descriptions, and imaging reports, the data is fed into an intelligent engine integrating natural language processing technology.

[0028] Step S200: Perform medical entity recognition and standard term mapping on the unstructured text data to generate standardized data with standard coding; In this embodiment, an intelligent engine integrating natural language processing technology is used to perform medical entity recognition and standard terminology mapping on unstructured text data. Medical entities refer to independent conceptual units with specific clinical significance in medical texts, including key medical concepts such as diseases, symptoms, drugs, examinations and tests, surgical procedures, and body parts.

[0029] Standard terminology mapping refers to the process of converting non-standardized medical terms extracted during the medical entity recognition process into unified and standardized codes and terms by intelligently matching them with authoritative medical knowledge bases using semantic matching algorithms. Standard terminology mapping can eliminate terminology differences (e.g., unifying different expressions such as "acute myocardial infarction" and "acute myocardial infarction" into standard diagnostic codes), support data aggregation, and improve interactivity.

[0030] As an optional implementation, medical entity recognition is performed on unstructured text data, and the recognition results are converted into semantic vectors through a pre-trained natural language model, such as the BERT model. The semantic vectors are then compared with term vectors in the built-in standard term knowledge graph (including standards such as ICD-10, SNOMED CT, and ATC) to calculate the cosine similarity, i.e., semantic similarity, and generate at least one candidate standard code and its corresponding confidence score.

[0031] For example, assuming the unstructured text data is an image report describing "a mass-like high-density shadow with unclear boundaries is seen in the upper lobe of the right lung, suggesting a possible malignant tumor," key entities such as "upper lobe of the right lung" (body part), "mass-like high-density shadow" (imaging manifestation), and "malignant tumor" (disease) are first extracted through medical entity recognition. Then, "malignant tumor" is converted into a semantic vector, and its semantic similarity with standard terms such as "C34.1 malignant tumor of the upper lobe of the lung" (confidence 92%), "R91.0 solitary shadow of the lung" (confidence 68%), and "C34.9 unspecified malignant tumor of the lung" (confidence 45%) is calculated in the standard terminology knowledge graph. Finally, a candidate standard code list is generated in descending order of confidence, providing a decision-making basis for subsequent automatic adjudication or manual verification.

[0032] As an alternative implementation, after generating candidate standard codes and their corresponding confidence levels, the decision to adopt a candidate standard code is made based on the confidence level. When multiple high-confidence candidate standard codes appear, a conflict resolution mechanism based on context rules, departmental information, etc., is activated to select the optimal match. Automatic data routing through confidence thresholds significantly improves data processing efficiency.

[0033] For example, when the confidence level is higher than a preset threshold, the candidate standard code corresponding to that confidence level is adopted. For instance, when the unstructured text data is "A patient was admitted to the hospital due to acute chest pain, and the electrocardiogram showed ST segment elevation and elevated myocardial enzyme levels," candidate standard codes "I21.0 Acute anterior wall myocardial infarction" (confidence level 91%), "I21.1 Acute inferior wall myocardial infarction" (confidence level 87%), and "R07.9 Chest pain" (confidence level 42%) are generated through semantic calculation. Assuming the preset confidence threshold is 85%, the conflict resolution mechanism is activated. First, the context rule base is called, and it is found that the specific infarct location is not described in the text. Then, the patient's department information is obtained as "CCU (Cardiovascular Medicine)". Finally, combined with the statistical weight of "anterior wall myocardial infarction is more common" in the clinical knowledge base, "I21.0 Acute anterior wall myocardial infarction" is automatically selected as the optimal code, and the resolution basis is recorded in the system for traceability.

[0034] Optionally, if the highest confidence level among the candidate codes is only 75%, the case and all candidate codes are pushed to the verification platform. The verification results are then fed back to the model performing medical entity recognition and standardized term mapping to iteratively optimize the natural language processing model performing medical entity recognition and standardized term mapping.

[0035] Optionally, the verification platform can be a human-machine collaborative verification platform, where medical experts or medical record coders ultimately determine the correct coding, feeding back the verification results and iteratively optimizing the natural language processing model, thereby improving the semantic understanding model's accuracy in recognizing similar terms. For example, when pushed to the verification platform, the platform interface displays the original text, data source, patient context information, recommended candidate standard codes, and confidence levels. Medical experts or medical record coders can click the "Confirm" button to accept the matching result, or correct it through methods such as "Remapping" or "Manual Coding." This confirmation or correction operation will be submitted as high-quality labeled data for feedback, used to optimize and update the natural language processing model and mapping rules, achieving a closed loop of knowledge updates.

[0036] It should be noted that standardized data, after undergoing high-confidence automatic processing or verification by a validation platform, will be loaded in layers through subsequent data processing steps. (Refer to...) Figure 3 For example, the original diagnostic records, as well as the newly generated diagnostic standard code and diagnostic standard name fields, will be loaded into the diagnostic details table in the detailed data layer for data traceability.

[0037] Step S300: Integrate the structured data and the standardized data, and generate a data processing link based on the data pattern of the integrated data, the preset data warehouse target pattern, and the data governance rules associated with the data warehouse target pattern; In this embodiment, the data schema of the integrated data clarifies the final structure of the data source, the data warehouse target schema clarifies the final target form of the data, and the data governance rules clarify the quality and security specifications that the data must adhere to during the transformation process. Based on these inputs, a directly executable data processing chain that integrates data transformation, quality verification, and security control logic can be automatically derived and generated.

[0038] As an optional implementation, the data schema of the integrated data and the preset data warehouse target schema are first analyzed to obtain the field mapping relationship and data transformation logic from the integrated data to the data defined in the data warehouse target schema. The field mapping relationship explains where each field in the target table comes from, which is a static correspondence. The data transformation logic explains what operations need to be performed and what business rules need to be followed to complete the transformation from the source (the data schema of the integrated data) to the target (the data warehouse target schema), which is a dynamic processing logic.

[0039] Optionally, the result defined by the data schema of the integrated data can be used as the source structure for data processing; the structure defined by the target schema of the data warehouse can be used as the target structure for data processing; based on the source structure and the target structure, a field mapping relationship from the source structure to the target structure can be established, and the data transformation logic required to implement the field mapping relationship can be defined.

[0040] For example, when constructing the patient visit wide table, the source structure is the integrated data, including a standardized patient information table, a standardized diagnosis fact table, etc. The target structure is the patient visit fact wide table defined by the data warehouse target schema. Based on this, a field mapping relationship is established: the "primary diagnosis code" field in the patient visit fact wide table needs to be mapped to the "diagnosis_std_code" field in the standardized diagnosis fact table of the original structure. Data transformation logic: It is necessary to associate the standardized patient information table and the standardized diagnosis fact table in the source structure through "patient identifier" and filter the records with the diagnosis type "primary diagnosis" to complete the data extraction.

[0041] As another optional implementation, while generating the complete transformation logic from source to target, data quality verification rules and data security control strategies are obtained from the data governance rules associated with the preset data warehouse target schema. This achieves deep integration of governance rules and data processing flow, transforming the originally independent, post-event data control actions into synchronized, automated processes that are executed concurrently with the data transformation process, ensuring the quality and security of incoming data from the source.

[0042] For example, during the data processing chain for generating the "Patient Visit Fact Table," quality rules and security policies are simultaneously retrieved from the governance rule base. The quality rule states: "The master diagnosis code field cannot be empty and must conform to the ICD-10 format"; the security policy states: "The patient name field requires dynamic anonymization." Subsequently, two nodes are inserted into the data processing chain: before data loading, the validity of the diagnosis code is validated, automatically intercepting and alerting against illegal data; before data output, the anonymization component is invoked to convert the plaintext name to the "Li*" format. In this way, data quality control and privacy protection are completed simultaneously with the data conversion, eliminating the need for additional governance processes.

[0043] As another optional implementation, based on the field mapping relationship, the data transformation logic, the data quality verification rules, and the data security control strategy, a task sequence required to achieve data transformation, verification, and control, and the execution dependencies between the task sequences are generated. Then, based on the task sequence and the execution dependencies, a directed acyclic graph (DAG) composed of executable tasks is generated as the data processing link. By automatically orchestrating the data transformation, quality verification, and security control tasks and their dependencies into a DAG, precise scheduling and closed-loop governance of the data processing pipeline are achieved, ensuring that data is executed in an orderly manner according to preset logic during the flow process, avoiding data quality and security risks caused by task sequence errors or missing governance links.

[0044] Optionally, when generating the task sequence and execution dependencies, the field mapping relationship and the data transformation logic are transformed into data transformation tasks; the data quality verification rules are transformed into data quality verification tasks; the data security control strategy is transformed into data security control tasks; then, the data transformation tasks, the data quality verification tasks, and the data security control tasks are used as a task set, and the execution order of the tasks is determined based on the data input and output relationships between the tasks in the task set, thereby generating the task sequence and the execution dependencies.

[0045] For example, when constructing a wide table of patient visits, the established field mapping relationships (such as mapping the "diagnostic standard code" field in the target table to the corresponding field in the source structure) and data transformation logic (such as associating tables and filtering "primary diagnoses") are first transformed into a "table association and diagnosis filtering" data transformation task; the data quality verification rules (diagnostic codes must comply with the ICD-10 standard) are transformed into a "diagnostic code compliance verification" task; and the data security control strategy (patient names must be anonymized) is transformed into a "patient name anonymization" task. Subsequently, the execution order is determined based on the data dependencies between tasks: the "table association and diagnosis filtering" task must be completed first to provide the data to be verified and anonymized, and then the "diagnostic code compliance verification" and "patient name anonymization" tasks can be executed in parallel.

[0046] The final generated task sequence and its execution dependencies are as follows: first, the table join and diagnostic screening tasks are executed; after completion, the diagnostic code verification and patient name desensitization tasks are started in parallel; finally, the data loading task is executed. These tasks are constructed into a directed acyclic graph according to their execution dependencies, forming a complete data processing chain.

[0047] It should be noted that execution dependency refers to the sequential execution constraints between tasks in a data processing flow. It defines the logical relationship in a task sequence where a task must wait for one or more of its dependent tasks to complete before it can start.

[0048] Step S400: Execute the data processing link, organize and store the integrated data according to the target data warehouse model, and implement data governance synchronously according to the logic on the data processing link during the execution process to complete the construction of the diagnosis and treatment data warehouse.

[0049] In this embodiment, standardized data is physically stored according to the data warehouse target model by executing the generated data processing pipeline, and the integrated quality verification and security control logic on the pipeline is triggered synchronously during the data loading process. This process realizes the embedding and synchronous execution of data construction and data governance processes. By transforming governance actions from independent external processes into inherent links in the data processing pipeline, quality auditing and security protection are completed at the same time as data is written to storage, ensuring the credibility and compliance of the data entering the warehouse.

[0050] As an optional implementation, data transformation tasks are scheduled and executed according to the data processing chain to organize and store the integrated data according to the data warehouse target model. During the scheduling and execution of data transformation tasks, data quality verification tasks and data security control tasks are simultaneously scheduled and executed according to the logic in the data processing chain. Specifically, executing data quality verification tasks includes invoking data quality verification rules bound to the data quality verification task to verify the data in transit; executing data security control tasks includes invoking data security control policies bound to the data security control task to dynamically de-identify or encrypt sensitive fields defined by the data security control policies.

[0051] For example, when constructing a wide table of patient visit facts, a data transformation task is scheduled and executed. By associating a standardized patient information table with a standardized diagnostic fact table, the integrated data is organized into a wide table structure according to the target pattern. Simultaneously, a data quality verification task is scheduled and executed according to the link logic. This task invokes bound rules (such as diagnostic codes must conform to the ICD-10 standard) to perform real-time verification of diagnostic codes in transit, intercepting illegal encoding and issuing alerts. At the same time, a data security control task is scheduled and executed, invoking bound policies (such as "patient names must be dynamically anonymized") to convert the patient name field in the wide table to anonymized forms such as "Li*" in real time. Through task-level synchronous scheduling, atomic collaboration of data storage, quality assurance, and security control is achieved, ensuring that data entering the warehouse meets governance requirements from the outset.

[0052] Optionally, after the construction of the diagnosis and treatment data warehouse is completed, the complete lineage of the data can be continuously recorded and tracked through the metadata management platform to achieve unified management and dynamic application of data models, quality rules and security strategies, and to monitor and provide intelligent early warning of the entire chain operation status in real time, forming a closed-loop management system of "governance-monitoring-optimization", and finally building a traceable, auditable and sustainably evolving data asset system.

[0053] In this embodiment, an intelligent governance system integrating natural language processing and automated orchestration technologies is constructed to achieve the integrated transformation of medical data from its original, multi-source, heterogeneous form into high-quality, computable data assets. The system deeply integrates intelligent data standardization and parsing capabilities, automatic generation of data processing chains, and embedded execution capabilities of governance rules. This ensures that diagnostic and treatment data undergoes quality verification and security protection simultaneously upon entry into the database, resolving the data resource failure issues caused by lagging governance and rigid processes. This provides efficient and reliable data support for the refined management of diagnostic and treatment data.

[0054] Based on the same inventive concept, this application also provides a system corresponding to the method in Embodiment 1, as shown in Embodiment 2.

[0055] Example 2 Reference Figure 4 The intelligent governance-based diagnostic and treatment data warehouse construction system of this embodiment includes: The data access and adaptation module is used to connect to various heterogeneous data sources and to perform protocol parsing and format conversion on data from different data sources with different protocols or formats, and output raw data with a unified specification. The raw data includes structured data and unstructured text data. In this embodiment, refer to Figure 5 The data access and adaptation module is responsible for connecting with various medical information systems, supporting multiple data protocols and formats, and realizing the unified collection and initial adaptation of heterogeneous data.

[0056] Optionally, the data access and adaptation module includes a data preprocessing unit for standardizing data formats, handling missing values, and detecting and repairing outliers.

[0057] Optionally, the data access and adaptation module also includes a multi-protocol adapter that supports medical standard protocols such as HL7, DICOM, and FHIR, enabling seamless integration of multi-source systems.

[0058] Optionally, the data access and adaptation module also includes a data extraction engine that extracts full and incremental data from hospital information systems, electronic medical record systems, laboratory information systems, and image archiving and communication systems on a regular / real-time basis.

[0059] Optionally, the data access and adaptation module also includes a preliminary adaptation and buffering unit, which performs preliminary data format conversion, establishes a buffering mechanism, and ensures the stability of data access.

[0060] The data access and adaptation module cleans the structured data and generates standard structured data, which can then be combined with the processed unstructured text data to form integrated data.

[0061] The data standardization processing module is communicatively connected to the data access and adaptation module, and is used to perform medical entity recognition and standard term mapping on the unstructured text data to generate standardized data with standard codes. In this embodiment, refer to Figure 6 The data standardization processing module performs deep analysis, medical entity recognition, and data standardization on unstructured text data, and performs cleaning tasks such as missing value imputation and error value correction.

[0062] Optionally, the data standardization processing module includes a natural language processing unit for medical named entity recognition of unstructured text data.

[0063] Optionally, the data standardization processing module includes a semantic mapping unit, which is used to convert the identified medical entities into vectors and generate candidate standard codes and their confidence scores by calculating the semantic similarity with the standard medical terminology knowledge graph. Optionally, the semantic mapping unit includes an NLP engine and a multi-standard mapping core unit. The NLP engine is used for medical entity recognition, relation extraction, and clinical text standardization processing. The multi-standard mapping core unit performs mapping and conversion of medical terminology standards such as ICD-10, SNOMEDCT, and LOINC.

[0064] Optionally, the data standardization processing module includes an adjudication unit, used to push the medical entity and the candidate standard code to the verification platform for adjudication when the confidence level is lower than a preset threshold. Specifically, low-confidence tasks are automatically routed to the verification platform for auxiliary verification. Changes involving key data such as diagnostic codes and drug codes require independent review and confirmation by two verification parties before taking effect, ensuring the authority and accuracy of the data standardization process. Simultaneously, the natural language processing model is continuously optimized through feedback from the verification results.

[0065] Optionally, the data standardization processing module also includes a data cleaning rule engine, which can be configured with a cleaning rule library and supports custom rules and machine learning-assisted cleaning.

[0066] The link construction and generation module is communicatively connected to the data standardization processing module. It is used to integrate the structured data and the standardized data, and generate a data processing link based on the data pattern of the integrated data, the preset data warehouse target pattern, and the data governance rules associated with the data warehouse target pattern. In this embodiment, refer to Figure 7 The link building and generation module integrates structured and standardized data, and automatically compiles them into an executable data processing pipeline with the preset data warehouse model and governance rules.

[0067] Optionally, the link construction and generation module includes a pattern parsing unit, which is used to parse the data pattern of the integrated data and the preset data warehouse target pattern to obtain the field mapping relationship and data transformation logic from the integrated data to the data defined by the data warehouse target pattern; Optionally, the link construction and generation module includes a rule acquisition unit, used to acquire data quality verification rules and data security control strategies from data governance rules associated with the target schema of the data warehouse; Optionally, the link construction and generation module includes a task derivation unit, which is used to derive the task sequence required to realize data conversion, verification and control and the execution dependency relationship between the task sequences based on the field mapping relationship, the data conversion logic, the data quality verification rules and the data security control strategy. Optionally, the link construction and generation module includes a link generation unit, used to generate a directed acyclic graph composed of executable tasks based on the task sequence and the execution dependencies, as the data processing link.

[0068] Optionally, the link building and generation module provides a visual data modeling tool that supports various link model designs such as star and snowflake schemas. It can also automatically generate data processing and loading links from data access to various data warehouse levels based on the data model and preset rules.

[0069] Optionally, the pre-defined data warehouse model can be designed using a drag-and-drop data model interface through a visual modeling tool, supporting medical-specific data model templates. The data warehouse model has a layered architecture, including ODS (Operational Data Layer), DWD (Detailed Data Layer), DWS (Summary Data Layer), and ADS (Application Data Layer).

[0070] The data construction and governance execution module is communicatively connected to the link construction and generation module. It is used to execute the data processing link, organize and store the integrated data according to the data warehouse target pattern, and synchronously implement data governance according to the logic integrated on the data processing link during the execution process to complete the construction of the diagnosis and treatment data warehouse.

[0071] In this embodiment, the data construction and governance execution module integrates functions such as data encryption, dynamic desensitization, fine-grained access control, data quality rule management, and real-time quality monitoring and evaluation, forming an integrated data governance framework.

[0072] Optionally, the data construction and governance execution module includes a governance logic executor, used to dynamically invoke the data quality verification rules bound to the data quality verification task to perform data verification during the execution of the data processing link, and to invoke the data security control policy bound to the data security control task to perform data de-identification or encryption. Data encryption includes transmission encryption and storage encryption.

[0073] In addition, refer to Figure 8 The intelligent governance-based diagnostic data warehouse construction system also includes a metadata management and monitoring platform. This platform is used to uniformly manage technical metadata (such as table structure and field definitions), business metadata (such as business terminology), management metadata (such as data quality rules and security policies), and operational metadata (such as data lineage and task scheduling logs), and provides comprehensive monitoring and alerts for system operation status. Through multi-dimensional quality monitoring of integrity, accuracy, consistency, and timeliness, visual reports can be generated.

[0074] Optionally, the metadata management and monitoring platform includes a unified metadata management unit for the unified collection and management of technical metadata, business metadata, and management metadata.

[0075] Optionally, the metadata management and monitoring platform also includes a data lineage analysis unit for end-to-end data lineage tracing, impact analysis, change auditing, and version management.

[0076] Optionally, the metadata management and monitoring platform also includes a system operation monitoring and alarm unit, which is used to monitor the system operation status in real time, detect anomalies and issue intelligent alarms, and perform performance analysis.

[0077] In addition, the intelligent governance-based medical data warehouse construction system can also set up a fine-grained access control mechanism. By defining basic permissions for user roles (such as doctors, nurses, and researchers) through RBAC, and combining it with ABAC based on dynamic attributes (such as department affiliation, patient disease type, and data sensitivity level) to achieve multi-dimensional permission determination, it can ensure that data access is accurately authorized while meeting the principle of least privilege, effectively guaranteeing the safe and compliant use of medical data.

[0078] Since the system described in Embodiment 2 of this application is a system used to implement the method of Embodiment 1 of this application, those skilled in the art can understand the specific structure and variations of the system based on the method described in Embodiment 1 of this application, and therefore will not be described again here. All systems used in the method of Embodiment 1 of this application fall within the scope of protection of this application.

[0079] Example 3 In this application embodiment, a diagnostic and treatment data warehouse construction device based on intelligent governance is proposed.

[0080] Reference Figure 9 , Figure 9 This is a schematic diagram of the terminal structure of the hardware operating environment involved in one embodiment of this application.

[0081] like Figure 9 As shown, the control terminal may include: a processor 1001, such as a CPU, a network interface 1003, a memory 1004, and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The network interface 1003 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1004 may be high-speed RAM or stable non-volatile memory, such as disk storage. Alternatively, the memory 1004 may be a storage device independent of the aforementioned processor 1001.

[0082] Those skilled in the art will understand that Figure 9 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0083] like Figure 9 As shown, the memory 1004, which serves as a computer storage medium, may include an operating system, a network communication module, and a program for constructing a medical data warehouse based on intelligent governance.

[0084] exist Figure 9 In the hardware structure of the intelligent governance-based medical data warehouse construction device shown, the processor 1001 can call the intelligent governance-based medical data warehouse construction program stored in the memory 1004 and perform the following operations: Acquire raw data from multiple heterogeneous data sources, including structured data and unstructured text data; The unstructured text data is subjected to medical entity recognition and standard terminology mapping to generate standardized data with standard coding. The structured data and the standardized data are integrated, and a data processing chain is generated based on the data pattern of the integrated data, the preset data warehouse target pattern, and the data governance rules associated with the data warehouse target pattern. The data processing chain is executed to organize and store the integrated data according to the target data warehouse model. During the execution process, data governance is implemented synchronously according to the logic on the data processing chain to complete the construction of the diagnosis and treatment data warehouse.

[0085] Optionally, the processor 1001 may invoke the intelligent governance-based diagnosis and treatment data warehouse construction program stored in the memory 1004, and further perform the following operations: The unstructured text data is used to perform medical entity recognition through an entity recognition and mapping model, and the recognition results are converted into semantic vectors. Calculate the semantic similarity between the semantic vector and the term vectors in the standard term knowledge graph, and generate at least one candidate standard code and its corresponding confidence score; If the confidence level is higher than a preset threshold, then the corresponding candidate standard code is adopted; If the confidence level is lower than the preset threshold, the medical entity and the candidate standard code are pushed to the verification platform, and the verification result is fed back to the entity recognition and mapping model for iterative optimization of the model.

[0086] Optionally, the processor 1001 may invoke the intelligent governance-based diagnosis and treatment data warehouse construction program stored in the memory 1004, and further perform the following operations: The data pattern of the integrated data and the preset data warehouse target pattern are parsed to obtain the field mapping relationship and data transformation logic from the standardized data to the data defined by the data warehouse target pattern; Data quality verification rules and data security control strategies are obtained from the data governance rules associated with the target schema of the data warehouse. Based on the field mapping relationship, the data conversion logic, the data quality verification rules, and the data security control strategy, a task sequence required to realize data conversion, verification, and control, as well as the execution dependencies between the task sequences, are generated. Based on the task sequence and the execution dependencies, a directed acyclic graph consisting of executable tasks is generated as the data processing link.

[0087] Optionally, the processor 1001 may invoke the intelligent governance-based diagnosis and treatment data warehouse construction program stored in the memory 1004, and further perform the following operations: The structure defined by the data pattern of the integrated data is used as the source structure for data processing; The structure defined by the target schema of the data warehouse is used as the target structure for data processing; Based on the source structure and the target structure, establish a field mapping relationship from the source structure to the target structure, and define the data transformation logic required to implement the field mapping relationship.

[0088] Optionally, the processor 1001 may invoke the intelligent governance-based diagnosis and treatment data warehouse construction program stored in the memory 1004, and further perform the following operations: The field mapping relationship and the data transformation logic are transformed into a data transformation task; the data quality verification rules are transformed into a data quality verification task; and the data security control strategy is transformed into a data security control task. The data transformation task, the data quality verification task, and the data security control task are taken as a task set, and the execution order of the tasks is determined based on the data input and output relationship between the tasks in the task set, thereby generating the task sequence and the execution dependency relationship.

[0089] Optionally, the processor 1001 may invoke the intelligent governance-based diagnosis and treatment data warehouse construction program stored in the memory 1004, and further perform the following operations: According to the data processing link, data transformation tasks are scheduled and executed to organize and store the integrated data according to the data warehouse target pattern; When scheduling and executing the data transformation task, data quality verification and data security control tasks are simultaneously scheduled and executed according to the logic of the data processing link; wherein, Executing the data quality verification task includes: invoking the data quality verification rules bound to the data quality verification task to verify the data in transit; Performing the data security control task includes: invoking the data security control policy bound to the data security control task, and performing dynamic desensitization or encryption on sensitive fields defined by the data security control policy.

[0090] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0091] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0092] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0093] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0094] It should be noted that any reference signs placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. This application can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, third, etc., does not indicate any order. These words can be interpreted as names.

[0095] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0096] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of the invention. Therefore, if these modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. A method for constructing a medical data warehouse based on intelligent governance, characterized in that, The method includes: Acquire raw data from multiple heterogeneous data sources, including structured data and unstructured text data; The unstructured text data is subjected to medical entity recognition and standard terminology mapping to generate standardized data with standard coding. The structured data and the standardized data are integrated, and a data processing chain is generated based on the data pattern of the integrated data, the preset data warehouse target pattern, and the data governance rules associated with the data warehouse target pattern. The data processing chain is executed to organize and store the integrated data according to the target data warehouse model. During the execution process, data governance is implemented synchronously according to the logic on the data processing chain to complete the construction of the diagnosis and treatment data warehouse.

2. The method as described in claim 1, characterized in that, The steps for performing medical entity recognition and standard terminology mapping on the unstructured text data include: The unstructured text data is used to perform medical entity recognition through an entity recognition and mapping model, and the recognition results are converted into semantic vectors. Calculate the semantic similarity between the semantic vector and the term vectors in the standard term knowledge graph, and generate at least one candidate standard code and its corresponding confidence score; If the confidence level is higher than a preset threshold, then the corresponding candidate standard code is adopted; If the confidence level is lower than the preset threshold, the medical entity and the candidate standard code are pushed to the verification platform, and the verification result is fed back to the entity recognition and mapping model for iterative optimization of the model.

3. The method as described in claim 1, characterized in that, The step of integrating the structured data and the standardized data, and generating a data processing chain based on the data pattern of the integrated data, a preset data warehouse target pattern, and data governance rules associated with the data warehouse target pattern, includes: The data pattern of the integrated data and the preset data warehouse target pattern are parsed to obtain the field mapping relationship and data transformation logic from the standardized data to the data defined by the data warehouse target pattern; Data quality verification rules and data security control strategies are obtained from the data governance rules associated with the target schema of the data warehouse. Based on the field mapping relationship, the data conversion logic, the data quality verification rules, and the data security control strategy, a task sequence required to realize data conversion, verification, and control, as well as the execution dependencies between the task sequences, are generated. Based on the task sequence and the execution dependencies, a directed acyclic graph consisting of executable tasks is generated as the data processing link.

4. The method as described in claim 3, characterized in that, The step of parsing the data pattern of the integrated data and the preset data warehouse target pattern to obtain the field mapping relationship and data transformation logic from the standardized data to the data defined by the data warehouse target pattern includes: The structure defined by the data pattern of the integrated data is used as the source structure for data processing; The structure defined by the target schema of the data warehouse is used as the target structure for data processing; Based on the source structure and the target structure, establish a field mapping relationship from the source structure to the target structure, and define the data transformation logic required to implement the field mapping relationship.

5. The method as described in claim 3, characterized in that, The step of generating the task sequence required to implement data conversion, verification, and control, and the execution dependencies between the task sequences, based on the field mapping relationship, the data conversion logic, the data quality verification rules, and the data security control strategy, includes: The field mapping relationship and the data transformation logic are transformed into a data transformation task; the data quality verification rules are transformed into a data quality verification task; and the data security control strategy is transformed into a data security control task. The data transformation task, the data quality verification task, and the data security control task are taken as a task set, and the execution order of the tasks is determined based on the data input and output relationship between the tasks in the task set, thereby generating the task sequence and the execution dependency relationship.

6. The method as described in claim 1, characterized in that, The steps of executing the data processing link, organizing and storing the integrated data according to the data warehouse target model, and synchronously implementing data governance according to the logic on the data processing link during execution to complete the construction of the diagnosis and treatment data warehouse include: According to the data processing link, data transformation tasks are scheduled and executed to organize and store the integrated data according to the target data warehouse model; When scheduling and executing the data transformation task, data quality verification and data security control tasks are simultaneously scheduled and executed according to the logic of the data processing link; wherein, Executing the data quality verification task includes: invoking the data quality verification rules bound to the data quality verification task to verify the data in transit; Performing the data security control task includes: invoking the data security control policy bound to the data security control task, and performing dynamic desensitization or encryption on sensitive fields defined by the data security control policy.

7. A system for constructing a medical data warehouse based on intelligent governance, characterized in that, The system includes: The data access and adaptation module is used to connect to various heterogeneous data sources and to perform protocol parsing and format conversion on data from different data sources with different protocols or formats, and output raw data with a unified specification. The raw data includes structured data and unstructured text data. The data standardization processing module is communicatively connected to the data access and adaptation module, and is used to perform medical entity recognition and standard term mapping on the unstructured text data to generate standardized data with standard codes. The link construction and generation module is communicatively connected to the data standardization processing module. It is used to integrate the structured data and the standardized data, and generate a data processing link based on the data pattern of the integrated data, the preset data warehouse target pattern, and the data governance rules associated with the data warehouse target pattern. The data construction and governance execution module is communicatively connected to the link construction and generation module. It is used to execute the data processing link, organize and store the integrated data according to the data warehouse target pattern, and synchronously implement data governance according to the logic integrated on the data processing link during the execution process to complete the construction of the diagnosis and treatment data warehouse.

8. The system as described in claim 7, characterized in that, The data standardization processing module includes: The natural language processing unit is used to perform medical named entity recognition on the unstructured text data; The semantic mapping unit is used to convert the identified medical entities into vectors and generate candidate standard codes and their confidence scores by calculating the semantic similarity with the standard medical terminology knowledge graph. The adjudication unit is used to push the medical entity and the candidate standard code to the verification platform for adjudication when the confidence level is lower than a preset threshold.

9. The system as described in claim 7, characterized in that, The link construction and generation module includes: The pattern parsing unit is used to parse the data pattern of the integrated data and the preset data warehouse target pattern to obtain the field mapping relationship and data transformation logic from the integrated data to the data defined by the data warehouse target pattern; The rule acquisition unit is used to acquire data quality verification rules and data security control strategies from the data governance rules associated with the target schema of the data warehouse; The task derivation unit is used to derive, based on the field mapping relationship, the data conversion logic, the data quality verification rules, and the data security control strategy, the task sequence required to realize data conversion, verification, and control, and the execution dependency relationship between the task sequences. The link generation unit is used to generate a directed acyclic graph consisting of executable tasks based on the task sequence and the execution dependencies, as the data processing link.

10. The system as described in claim 7, characterized in that, The data construction and governance execution module includes a governance logic executor, which is used to dynamically call the data quality verification rules bound to the data quality verification task to perform data verification when executing the data processing link, and to call the data security control policy bound to the data security control task to perform data desensitization or encryption.

Images