A literature evidence-driven clinical data governance rule generation method and system

Abstract

The application relates to the technical field of data processing, and discloses a literature evidence-driven clinical data management rule generation method and system, which comprises the following steps: constructing a target text data set according to an original clinical data set and a research field identifier, retrieving a literature database and extracting field identifiers and constraint descriptions to form an initial rule set; performing credibility evaluation on the initial rule, and obtaining a standard rule set after screening and merging; mapping the field identifiers in the rule and the data set fields to generate data management constraint items; comparing the data set and the constraint items item by item, accurately positioning the abnormal field positions and distinguishing the difference types; combining the abnormal positions and the difference categories, and outputting data completion suggestions and complete data cleaning schemes in a targeted manner to realize clinical data standardization management; and the application can improve the efficiency of literature evidence-driven clinical data management rule generation.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for generating clinical data governance rules based on literature evidence. Background Technology

[0002] Currently, clinical data governance rules rely heavily on manual sorting and experience-based formulation, resulting in low rule generation efficiency, insufficient standardization, and a lack of full utilization of high-quality literature evidence to form objective constraints. This leads to poor rule accuracy, consistency, and reusability, making it difficult to adapt to the governance needs of multiple scenarios and types of clinical data. Furthermore, the credibility of the rules lacks a quantitative assessment mechanism, and the results of abnormal data location, cleaning, and repair are inadequate.

[0003] For example, when extracting constraint rules for clinical data, existing technologies cannot automatically screen highly credible evidence based on literature citation networks and publication sources, easily incorporating invalid constraints from low-quality, non-peer-reviewed literature, leading to logical conflicts in the rules; field mapping only supports simple character matching and cannot achieve precise association at the synonym or semantic level, easily resulting in mismatches between governance constraints and data fields; abnormal data detection can only identify basic missing and formatting issues, and cannot accurately determine types such as value range out-of-bounds and constraint missing, and data completion and cleaning schemes lack standardized basis.

[0004] Therefore, existing clinical data governance rule generation technologies suffer from core defects such as low automation, uncontrollable rule reliability, insufficient mapping and matching accuracy, and limited anomaly handling capabilities. These shortcomings make it difficult to meet the needs of efficient, accurate, and standardized clinical data governance, thus hindering the high-quality utilization of clinical data and the application of artificial intelligence technology in the medical field. Summary of the Invention

[0005] This invention provides a method and system for generating clinical data governance rules based on literature evidence to solve the problems mentioned in the background.

[0006] To achieve the above objectives, this invention provides a method for generating clinical data governance rules based on literature evidence, comprising: S1. Receive the original clinical dataset and research domain identifier of the target text to obtain the dataset of the target text; S2. Obtain a literature set by retrieving a preset literature database through the data set, and extract field identifiers and constraint descriptions from the literature set to form an initial rule set for the target text; S3. Evaluate the source credibility of each rule in the initial rule set, and filter and merge them according to the evaluation results to obtain the standard rule set of the target text; S4. Map the field identifiers in the standard rule set to the data fields in the data set to generate the governance constraint terms of the target text; S5. Compare the data set with the governance constraints item by item to locate the abnormal field positions in the original clinical dataset and determine the type of difference in the abnormal field positions; S6. Based on the location of the abnormal field and the type of difference, generate data completion suggestions and data cleaning schemes for the target text.

[0007] In a preferred embodiment, the original clinical dataset and research domain identifier for receiving the target text include: Obtain the original data records of the target text to obtain the original clinical dataset of the target text; The original clinical dataset is parsed to obtain the field types and field hierarchy of the target text; Using the field types as nodes and the field hierarchy as connecting edges, construct the field topology of the original clinical dataset; The research field identifier of the target text is received, and the research field identifier is associated with the field topology to obtain the data set of the target text.

[0008] In a preferred embodiment, the step of retrieving a pre-defined literature database through the data set to obtain a literature set, and extracting field identifiers and constraint descriptions from the literature set to form an initial rule set for the target text, includes: Using the field names in the data set as anchors, documents that simultaneously contain the field names and synonyms are located in the preset literature database to form an initial set of literature. Using the references in the initial screening set as inbound nodes and the cited documents in the initial screening set as outbound nodes, a citation network for the initial screening set is constructed. The core set of documents in the document database is then selected based on the node centrality in the citation network. Syntactic dependency parsing is performed on each document in the core document set, and the parsed limiting phrases that have dependency relationships with the field name are type-labeled to generate candidate constraint pairs for the target text. The candidate constraint pairs are subjected to conflict detection and consistency merging to obtain the constraint description of the target text. The field names corresponding to the constraint descriptions are used as field identifiers and summarized into the initial rule set of the target text.

[0009] In a preferred embodiment, the step of filtering the core literature set from the literature database based on the node centrality in the citation network includes: The frequency of citations of the references by other documents in the reference database is aggregated and analyzed to determine the in-chain centrality and out-chain centrality of the citation network. The outgoing chain centrality of the citation network is obtained by measuring the breadth and diffusion of the cited documents in relation to the other documents. The inbound centrality and outbound centrality are weighted and fused to obtain the comprehensive influence score of the referenced network; Based on the comprehensive influence score, all documents in the initial screening document set are sorted in descending order to obtain the target text's undetermined core document set; The core document set to be determined is deduplicated to obtain the core document set of the target text.

[0010] In a preferred embodiment, the step of evaluating the source credibility of each rule in the initial rule set and filtering and merging them based on the evaluation results to obtain the standard rule set for the target text includes: Extract the publication source identifier and peer review status identifier of the documents from which the rules in the initial rule set originate; Based on the publication source identifier, the literature collection is divided into core publication sources, extended publication sources, and general publication sources; and based on the peer review status identifier, the literature collection is divided into those that have passed peer review and those that have not been peer reviewed. Rules belonging to the core publication source and having passed peer review are marked as high-confidence rules; rules belonging to the extended publication source and having passed peer review are marked as medium-confidence rules; and rules belonging to the general publication source or not yet peer-reviewed are marked as low-confidence rules. The low-confidence rules are removed, and the high-confidence rules and the medium-confidence rules are combined to form the initial rule set of the target text; The initial rule set is merged and integrated to obtain the standard rule set for the target text.

[0011] In a preferred embodiment, the step of merging and integrating the initial rule set to obtain the standard rule set for the target text includes: The initial rule set is grouped using the field identifiers to obtain the field rule groups of the target text; The rule priority sequence of the field rule group is determined based on the publication source identifier and the peer review status identifier; By applying the rule priority sequence to the field rule group, consistency rules are applied, and redundant fields and logical conflicts in the merged results are eliminated to obtain the standard rule set for the target text.

[0012] In a preferred embodiment, the step of mapping field identifiers from the standard rule set to data fields in the dataset to generate governance constraints for the target text includes: Extract field identifiers from the standard rule set, construct a field identifier queue, and construct a data field queue from the data field names in the data set; Perform precise character matching between the field identifier queue and the data field queue, and output the precise mapping relationship of the target text; Perform synonym expansion matching on the field identifier queue and the data field queue to obtain the synonym mapping relationship of the target text; Semantic inclusion matching is performed on the field identifier queue and the data field queue to generate a semantic mapping relationship for the target text; The precise mapping relationship, the synonym mapping relationship, and the semantic mapping relationship are summarized into a field mapping table for the target text; By using the field mapping table, the constraint descriptions corresponding to the field identifiers in the standard rule set are associated with the matching data fields in the data set to generate the governance constraint items for the target text.

[0013] In a preferred embodiment, the step of comparing the dataset with each governance constraint item to locate the abnormal field in the original clinical dataset and determining the type of difference in the abnormal field location includes: Traverse the data fields in the dataset and perform a step-by-step difference comparison of the constraint descriptions corresponding to the data fields in the governance constraint items; If the data field has no corresponding constraint description in the governance constraint item, then the position of the data field is marked as a missing constraint anomaly. If the data field has a corresponding constraint description but the current value of the data field is empty, then the position of the data field is marked as a missing value anomaly. If the current value of the data field does not meet the value range requirement in the constraint description, then the position of the data field is marked as a value range out-of-bounds exception; If the current value of the data field does not meet the format requirements in the constraint description, then the position of the data field is marked as a format violation exception; The missing constraint anomalies, missing value anomalies, value range out-of-bounds anomalies, and format violation anomalies are summarized and labeled to obtain the location and difference type of the abnormal fields in the original clinical dataset.

[0014] In a preferred embodiment, generating data completion suggestions and data cleaning schemes for the target text based on the location of the abnormal field and the type of difference includes: The abnormal field location and the difference type are mapped to a preset correction strategy table, and the completion strategy and cleaning strategy of the abnormal field location and the difference type are matched. The completion strategy is applied to the abnormal field position, and the strategy is instantiated for the difference type to obtain the completion suggestion for the target text; The cleaning strategy is applied to the location of the abnormal field, and the operation is bound to the difference type to obtain the cleaning operation of the target text; The completion suggestions and the cleaning operations are correlated and summarized to obtain the data completion suggestions and data cleaning scheme for the target text.

[0015] To address the aforementioned issues, this invention also provides a clinical data governance rule generation system based on literature evidence, the system comprising: The data acquisition module is used to receive the original clinical dataset and research area identifier of the target text, and obtain the data set of the target text; The retrieval and extraction module is used to retrieve a set of documents from a preset literature database through the data set, and extract field identifiers and constraint descriptions from the literature set to form an initial rule set for the target text. The standard judgment module is used to evaluate the source credibility of each rule in the initial rule set, and filter and merge them according to the evaluation results to obtain the standard rule set of the target text; The data mapping module is used to map the field identifiers in the standard rule set to the data fields in the data set, thereby generating the governance constraint items of the target text; An anomaly localization module is used to compare the data set with the governance constraints item by item to locate the abnormal field in the original clinical dataset and determine the type of difference in the abnormal field location. The solution output module is used to generate data completion suggestions and data cleaning solutions for the target text based on the location of the abnormal field and the type of difference.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention automatically generates clinical data governance rules through literature evidence, which can significantly improve the efficiency and standardization of rule generation. Based on the process of literature retrieval and extraction, credibility assessment and rule merging, it forms an objective, unified and highly credible set of clinical data governance standard rules, ensuring that the rules are authoritative in origin, logically consistent and can be efficiently reused.

[0017] 2. This invention can accurately complete multi-level mapping between rule fields and clinical data fields, realize the precise location of abnormal fields and automatic determination of difference types, and automatically generate data completion suggestions and cleaning schemes based on abnormal information, effectively improving the integrity, standardization and accuracy of clinical data governance and reducing manual governance costs. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a method for generating clinical data governance rules based on literature evidence, as provided in an embodiment of the present invention. Figure 2 A functional block diagram of a clinical data governance rule generation system based on literature evidence-driven approach provided in an embodiment of the present invention; The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0020] This application provides a method for generating clinical data governance rules based on literature evidence. The executing entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, this method can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cluster of cloud servers. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0021] Reference Figure 1 As shown in Example 1, this is a flowchart illustrating a method for generating clinical data governance rules based on literature evidence, according to an embodiment of the present invention. In this embodiment, the method for generating clinical data governance rules based on literature evidence includes: S1. Receive the original clinical dataset and research domain identifier of the target text to obtain the dataset of the target text; In this embodiment of the invention, the original clinical dataset and research domain identifier for receiving the target text include: Obtain the original data records of the target text to obtain the original clinical dataset of the target text; The original clinical dataset is parsed to obtain the field types and field hierarchy of the target text; Using the field types as nodes and the field hierarchy as connecting edges, construct the field topology of the original clinical dataset; The research field identifier of the target text is received, and the research field identifier is associated with the field topology to obtain the data set of the target text.

[0022] Specifically, all original data records corresponding to the target text are obtained, all original data records are integrated according to a unified data storage format, and all integrated data content is summarized to form the original clinical dataset of the target text.

[0023] Each data item in the original clinical dataset is analyzed field by field to identify the data category to which each data item belongs, determine all field types contained in the target text, and sort out the attribution and association relationships between different fields to obtain the field hierarchy of the target text.

[0024] All parsed field types are set as independent nodes. Connection edges are established between directly related nodes according to the determined field hierarchy. The field topology structure of the original clinical dataset is fully constructed according to the combination of nodes and connection edges.

[0025] The system receives the research domain identifier of the target text from external input, binds the research domain identifier to the corresponding field topology structure and completes the annotation, so that the field topology structure carries the research domain identifier information, and obtains the data set of the target text.

[0026] In summary, by acquiring raw data records to form a complete raw clinical dataset, conducting field structure analysis to clarify field types and hierarchical relationships, constructing field topology structures and associating them with research domain identifiers, the overall structure and domain attributes of clinical data can be clearly presented. This provides a stable data structure foundation for subsequent literature retrieval, rule extraction, and credibility assessment, and improves the pertinence and accuracy of clinical data governance rule generation.

[0027] S2. Obtain a literature set by retrieving a preset literature database through the data set, and extract field identifiers and constraint descriptions from the literature set to form an initial rule set for the target text; In this embodiment of the invention, the step of retrieving a pre-defined literature database through the data set to obtain a literature set, and extracting field identifiers and constraint descriptions from the literature set to form an initial rule set for the target text, includes: Using the field names in the data set as anchors, documents that simultaneously contain the field names and synonyms are located in the preset literature database to form an initial set of literature. Using the references in the initial screening set as inbound nodes and the cited documents in the initial screening set as outbound nodes, a citation network for the initial screening set is constructed. The core set of documents in the document database is then selected based on the node centrality in the citation network. Syntactic dependency parsing is performed on each document in the core document set, and the parsed limiting phrases that have dependency relationships with the field name are type-labeled to generate candidate constraint pairs for the target text. The candidate constraint pairs are subjected to conflict detection and consistency merging to obtain the constraint description of the target text. The field names corresponding to the constraint descriptions are used as field identifiers and summarized into the initial rule set of the target text.

[0028] The process of filtering the core literature set from the literature database based on the node centrality in the citation network includes: The frequency of citations of the references by other documents in the reference database is aggregated and analyzed to determine the in-chain centrality and out-chain centrality of the citation network. The outgoing chain centrality of the citation network is obtained by measuring the breadth and diffusion of the cited documents in relation to the other documents. The inbound centrality and outbound centrality are weighted and fused to obtain the comprehensive influence score of the referenced network; Based on the comprehensive influence score, all documents in the initial screening document set are sorted in descending order to obtain the target text's undetermined core document set; The core document set to be determined is deduplicated to obtain the core document set of the target text.

[0029] Specifically, all field names in the dataset are retrieved, and each field name is used as a search anchor. A reverse search operation is performed in the preset literature database, traversing all literature content in the literature database, locating all literature that records both the field name and synonyms with the same meaning as the field name, and summarizing and combining all the located literature to form the initial set of literature.

[0030] The references of each document in the initial screening set are extracted, and each reference is taken as an inbound node. Cited documents formed by citations of each document in the initial screening set are extracted, and each cited document is taken as an outbound node. According to the actual citation correspondence between references and cited documents, all inbound and outbound nodes are connected and arranged to build a complete citation network for the initial screening set. The total number of times each node in the citation network is connected to other nodes is counted. The documents are sorted from high to low according to the number of nodes connected, and the documents with the highest number of connections are selected to obtain the core document set from the document database.

[0031] For each full-text article in the core literature collection, the sentence structure is divided sentence by sentence. The modification, limitation, and subordination relationships between words in the sentences are analyzed to complete syntactic dependency parsing. From the parsing results, word combinations that have a direct limitation relationship with the field name are extracted. These word combinations are identified as limitation phrases. Each limitation phrase is labeled with a corresponding type according to the constraint meaning expressed by the limitation phrase. The field name is paired with the limitation phrases with the completed type labels to generate candidate constraint pairs for the target text.

[0032] It should be noted that the syntactic dependency parsing method identifies subject-predicate, verb-object, attributive, and prepositional relationships between words on a sentence-by-sentence basis. Words or phrases with any of the following dependency relationships with field names are extracted as qualifying phrases: adjectives where the field name is a noun and it has an attributive relationship, such as "shrinkage" in "systolic blood pressure"; verb phrases where the field name is an object and it has a verb-object relationship, such as "must not exceed" in "must not exceed 140 mmHg"; numerical unit phrases following the field name, such as "mmHg" or "mg / dL"; and numerical range phrases guided by keywords such as "range," "interval," or "between."

[0033] Furthermore, the type labeling includes the following four constraint categories: the first category is the value range constraint, which is used to limit the value range of the field, such as "0 to 100" or "greater than or equal to 18"; the second category is the format constraint, which is used to limit the data format of the field, such as "YYYY-MM-DD" or a specific regular expression; the third category is the existence constraint, which is used to limit whether the field is required, such as "cannot be empty" or "optional"; the fourth category is the enumeration constraint, which is used to limit the field to be selected only from a specific value set, such as "male or female" or "Level I, Level II, Level III".

[0034] Specifically, during annotation, by matching a predefined keyword library, such as "range", "must", "only", "format as", etc., and phrase structure, the limiting phrases are classified into one of the above four categories.

[0035] The content of all candidate constraint pairs is compared one by one. Candidate constraint pairs with contradictory content are identified and eliminated. Candidate constraint pairs with the same or similar meanings are merged and integrated to obtain the constraint description of the target text. The field names corresponding to the constraint descriptions are directly determined as field identifiers. All field identifiers and their corresponding constraint descriptions are collected and organized to form the initial rule set of the target text.

[0036] Specifically, the total number of times other documents in the statistical literature database cite the reference is counted, and all citation counts corresponding to the same reference are aggregated and analyzed to determine the in-chain centrality of the citation network.

[0037] The number of references to other documents in a cited document database is counted to measure the coverage of related documents of the cited document, thus completing the diffusion measure of citation breadth and obtaining the out-chain centrality of the citation network.

[0038] By integrating the information on the degree of citation of documents represented by the centrality of the inbound chain and the information on the scope of citation diffusion of documents represented by the centrality of the outbound chain, the two types of document influence information are combined into a unified evaluation result, resulting in a comprehensive influence score of the citation network.

[0039] It should be noted that the in-chain centrality is calculated using the following formula: the in-chain centrality of document v equals the total number of times all other documents in the citation network cite document v, i.e., the sum of the number of edges pointing from all other documents to document v. The out-chain centrality is calculated using the following formula: the out-chain centrality of document v equals the total number of other documents cited by document v, i.e., the sum of the number of edges pointing from document v to other documents. The in-chain centrality and out-chain centrality are weighted and fused to obtain the comprehensive influence score, calculated as follows: the comprehensive influence score equals the first weight coefficient multiplied by the in-chain centrality plus the second weight coefficient multiplied by the out-chain centrality. The first weight coefficient ranges from 0.6 to 0.8, the second weight coefficient ranges from 0.2 to 0.4, and the sum of the two coefficients is 1.

[0040] In a preferred implementation, the first weighting coefficient is 0.7 and the second weighting coefficient is 0.3. All documents in the initial screening set are sorted in descending order based on the comprehensive influence score, and the top N documents are selected as the core document set, where N is a preset positive integer ranging from 5 to 50. In a preferred implementation, N is set to 20.

[0041] Based on the comprehensive impact score from high to low, all documents in the initial screening document set are arranged and organized to form the core document set to be determined for the target text.

[0042] The document content of each document in the pending core document set is reviewed one by one. Duplicate documents with completely identical content are identified. One document is kept and the rest are deleted. The deduplication process of the pending core document set is completed, and the core document set of the target text is obtained.

[0043] In summary, this approach enables targeted literature retrieval using field names as anchors, ensures the quality of information sources by filtering high-value literature through citation networks, directly extracts constraint information from literature through syntactic dependency parsing, and eliminates rule contradictions through conflict detection and consistency merging. It can quickly construct an initial rule set with standardized sources and unified content, shorten the rule generation cycle, improve the completeness and stability of rule extraction, and provide complete and usable original rule materials for subsequent rule credibility assessment and standardization processing.

[0044] By conducting quantitative analysis of citation relationships, the in-chain centrality and out-chain centrality are determined separately, and a comprehensive literature influence score is formed by merging them. Based on the score, the literature is sorted and deduplicated. This can reliably select core literature with outstanding influence and unique content from the initial screening of literature, ensuring that the source of rule extraction has high credibility and improving the stability and authority of the subsequent rule generation process.

[0045] S3. Evaluate the source credibility of each rule in the initial rule set, and filter and merge them according to the evaluation results to obtain the standard rule set of the target text; In this embodiment of the invention, the step of evaluating the source credibility of each rule in the initial rule set and filtering and merging them according to the evaluation results to obtain the standard rule set of the target text includes: Extract the publication source identifier and peer review status identifier of the documents from which the rules in the initial rule set originate; Based on the publication source identifier, the literature collection is divided into core publication sources, extended publication sources, and general publication sources; and based on the peer review status identifier, the literature collection is divided into those that have passed peer review and those that have not been peer reviewed. Rules belonging to the core publication source and having passed peer review are marked as high-confidence rules; rules belonging to the extended publication source and having passed peer review are marked as medium-confidence rules; and rules belonging to the general publication source or not yet peer-reviewed are marked as low-confidence rules. The low-confidence rules are removed, and the high-confidence rules and the medium-confidence rules are combined to form the initial rule set of the target text; The initial rule set is merged and integrated to obtain the standard rule set for the target text.

[0046] The process of merging and integrating the initial rule set to obtain the standard rule set for the target text includes: The initial rule set is grouped using the field identifiers to obtain the field rule groups of the target text; The rule priority sequence of the field rule group is determined based on the publication source identifier and the peer review status identifier; By applying the rule priority sequence to the field rule group, consistency rules are applied, and redundant fields and logical conflicts in the merged results are eliminated to obtain the standard rule set for the target text.

[0047] Specifically, from the source literature corresponding to each rule in the initial rule set, the publication source identifier used to indicate the publication channel of the literature is extracted, and from the source literature corresponding to each rule, the peer review status identifier used to indicate the review status of the literature is extracted.

[0048] Based on the publication channel level represented by the extracted publication source identifiers, the literature collection is divided into three categories: core publication source, extended publication source, and general publication source. Based on the review results represented by the extracted peer review status identifiers, the literature collection is divided into two categories: peer-reviewed and non-peer-reviewed.

[0049] Rules that simultaneously belong to the core publication source and the peer-reviewed category are marked as high-credibility rules; rules that simultaneously belong to the extended publication source and the peer-reviewed category are marked as medium-credibility rules; and rules that belong to the general publication source category or the unreviewed category are marked as low-credibility rules.

[0050] Remove all rules marked as low confidence from the rule set, retain all rules marked as high confidence and medium confidence, and recombine the retained rules to form the initial rule set for the target text.

[0051] The rules within the newly formed initial rule set are sorted out and integrated. Constraints corresponding to the same fields are merged, and duplicate expressions and logical conflicts in the rules are cleaned up to form a standard rule set for the target text.

[0052] Specifically, based on the field identifier carried by each rule in the initial rule set, all rules carrying the same field identifier are grouped into the same group to form the field rule group of the target text.

[0053] Based on the publication source identifier level and peer review status identifier corresponding to each rule in the field rule group, rules with higher publication source levels and those that have passed peer review are arranged in the first position, and rules with lower publication source levels or those that have not passed peer review are arranged in the last position, forming a rule priority sequence for the field rule group.

[0054] It should be noted that the core publication sources refer to academic journals included in the Science Citation Index or Social Science Citation Index and whose latest Journal Citation Report is classified as Q1 or Q2, as well as journals included in the Chinese Science Citation Database and included in its core database; the extended publication sources refer to SCI or SSCI journals whose Journal Citation Report is classified as Q3 or Q4, as well as journals included in the Chinese Science and Technology Paper Statistics Source Journals; and the general publication sources refer to academic journals, conference proceedings, and dissertations that are not included in the aforementioned scope but have a formal publication number (ISSN or CN).

[0055] It should be noted that the peer review status identifier is obtained by extracting the "peer-reviewed" tag from a field in a literature database (such as PubMed) or parsing the peer review mark in the metadata of the literature DOI; if the literature clearly indicates that it has been peer reviewed, it is marked as "peer reviewed", otherwise it is marked as "not peer reviewed".

[0056] According to the order of rule priority sequence, the execution content of the rules in the field rule group is uniformly processed, the rule content with higher priority is retained, the duplicate rule content with lower priority is removed, the redundant field items in the merged content are deleted, and the logical contradictions between the rules are eliminated to obtain the standard rule set of the target text.

[0057] In summary, by classifying the credibility of rules through the sources of literature publication and peer review status, eliminating low-credibility rules, and merging and integrating rules, a standard rule set with reliable sources, standardized content, and unified logic can be constructed, thereby improving the overall quality of clinical data governance rules and providing stable and standardized rule support for subsequent field mapping and governance constraint generation.

[0058] By grouping rules using field identifiers and determining rule priorities based on publication source identifiers and peer review status identifiers, and then processing and cleaning up redundancies according to priorities, rule contradictions and duplicate content can be eliminated, forming a set of standard rules with unified logic and standardized content, thereby improving the overall standardization and usability of the rules.

[0059] S4. Map the field identifiers in the standard rule set to the data fields in the data set to generate the governance constraint terms of the target text; In this embodiment of the invention, the step of mapping the field identifiers in the standard rule set to the data fields of the data set to generate the governance constraint terms of the target text includes: Extract field identifiers from the standard rule set, construct a field identifier queue, and construct a data field queue from the data field names in the data set; Perform precise character matching between the field identifier queue and the data field queue, and output the precise mapping relationship of the target text; Perform synonym expansion matching on the field identifier queue and the data field queue to obtain the synonym mapping relationship of the target text; Semantic inclusion matching is performed on the field identifier queue and the data field queue to generate a semantic mapping relationship for the target text; The precise mapping relationship, the synonym mapping relationship, and the semantic mapping relationship are summarized into a field mapping table for the target text; By using the field mapping table, the constraint descriptions corresponding to the field identifiers in the standard rule set are associated with the matching data fields in the data set to generate the governance constraint items for the target text.

[0060] Specifically, extract all field identifiers from the standard rule set, arrange all field identifiers in order to form a field identifier queue, extract all data field names from the dataset, arrange all data field names in order to form a data field queue.

[0061] The text content in the field identifier queue and the data field queue is compared bit by bit. A correspondence is established between the field identifier and the data field where the text content is completely consistent, and the precise mapping relationship of the target text is output.

[0062] The meanings of the contents in the field identifier queue and the data field queue are compared. A correspondence is established between the field identifiers and data fields that have different words but the same meaning, so as to obtain the synonym mapping relationship of the target text.

[0063] A range comparison is performed on the content in the field identifier queue and the data field queue. A correspondence is established between the field identifiers and data fields whose content is completely covered by the content of the other party, and the semantic inclusion relationship of the target text is generated.

[0064] It should be noted that the specific implementation method for extended synonym matching is as follows: A thesaurus for clinical medicine is pre-constructed, which is derived from the Unified Medical Language System, the Medical Subject Headings, and the Chinese Clinical Terminology Set (e.g., ICD-10); for each field identifier in the field identifier queue, its core stem is extracted, for example, by removing suffixes such as "value" and "code", and then all its synonyms and near-synonyms are retrieved in the thesaurus to generate a thesaurus set; each word in the thesaurus set is matched with the field name in the data field queue, ignoring capitalization and underscores during the matching process. If the match is successful, a synonym mapping relationship is established.

[0065] To further explain the specific implementation of semantic inclusion matching: a deep learning-based semantic similarity model, such as the BERT pre-trained model, is used. Field identifiers from the field identifier queue and field names from the data field queue are input into the model, and their cosine similarity is calculated. The similarity threshold is set to 0.75. If the similarity is greater than or equal to 0.75, a semantic inclusion relationship is determined. For example, the similarity between "patient age" and "age (years)" can reach 0.92. For identifying the direction of the inclusion relationship, word vectors are used to determine whether a word is a hypernym or hyponym of another word. For example, "blood pressure" includes "systolic pressure," thus establishing an inclusion mapping.

[0066] The precise mapping relationships, synonym mapping relationships, and semantic inclusion relationships are recorded and integrated in a centralized manner to form a field mapping table for the target text.

[0067] Based on the correspondence established in the field mapping table, the constraint descriptions corresponding to the field identifiers in the standard rule set are bound to the matching data fields in the data set to generate the governance constraint items of the target text.

[0068] In summary, by constructing a field identifier queue and a data field queue to achieve multi-level matching and form a comprehensive field mapping table, the constraint descriptions in standard rules can be stably associated with the corresponding data fields, automatically generating governance constraint items that fit the clinical data structure, improving the adaptation effect between rules and data fields, and providing a unified and usable constraint basis for subsequent abnormal data location.

[0069] S5. Compare the data set with the governance constraints item by item to locate the abnormal field positions in the original clinical dataset and determine the type of difference in the abnormal field positions; In this embodiment of the invention, the step of comparing the data set with the governance constraints item by item to locate the abnormal field positions in the original clinical dataset and determining the type of difference in the abnormal field positions includes: Traverse the data fields in the dataset and perform a step-by-step difference comparison of the constraint descriptions corresponding to the data fields in the governance constraint items; If the data field has no corresponding constraint description in the governance constraint item, then the position of the data field is marked as a missing constraint anomaly. If the data field has a corresponding constraint description but the current value of the data field is empty, then the position of the data field is marked as a missing value anomaly. If the current value of the data field does not meet the value range requirement in the constraint description, then the position of the data field is marked as a value range out-of-bounds exception; If the current value of the data field does not meet the format requirements in the constraint description, then the position of the data field is marked as a format violation exception; The missing constraint anomalies, missing value anomalies, value range out-of-bounds anomalies, and format violation anomalies are summarized and labeled to obtain the location and difference type of the abnormal fields in the original clinical dataset.

[0070] Specifically, examine each data field in the dataset in turn, and compare the content differences between each data field and the corresponding constraint description in the governance constraint item.

[0071] When a data field does not have a corresponding constraint description in the governance constraint item, the location of this data field is marked as a missing constraint anomaly.

[0072] If a data field has a corresponding constraint description, and the currently filled content in the data field is blank, then the position of this data field will be marked as a missing value anomaly.

[0073] When the content currently entered in a data field exceeds the numerical range specified by the constraint description, the location of this data field will be marked as a value range out-of-bounds exception.

[0074] It should be noted that the value range in the constraint description must be expressed in any of the following forms: First, a numerical interval, represented as a closed or open interval from the lower limit to the upper limit, for example, [18, 100] represents greater than or equal to 18 and less than or equal to 100, and [0, 200) represents greater than or equal to 0 and less than 200. Second, a discrete enumeration, represented as a list of enumerated values ​​within curly braces, for example, {male, female}. Third, a comparison expression, represented as greater than X, less than X, greater than or equal to X, less than or equal to X, or not equal to X, for example, greater than 18.

[0075] When the content currently entered in a data field does not conform to the format specifications defined by the constraint description, the location of this data field will be marked as a format violation exception.

[0076] It should be noted that the format requirements are constrained by regular expressions, specifically, the corresponding matching rules are described in natural language. For example, the requirements for date format are: start with four digits, followed by a hyphen, then two digits, then another hyphen, and finally end with two digits; the requirements for phone number format are: start with the number 1, the second digit is a number between 3 and 9, followed by nine digits. The email address format requirements are as follows: a string consisting of letters, numbers, or underscores, which may contain periods or hyphens, followed by an at sign, then the domain name (consisting of letters, numbers, or underscores, which may contain periods or hyphens), and ending with two or three letters connected by periods; the alphanumeric format requirements are: containing only uppercase and lowercase English letters and numbers, without any other symbols; during item-by-item comparison, the current value of the data field is parsed against the corresponding constraint description: if the constraint is a numerical range, it is determined whether the current value is within the range; if it is an enumeration, it is determined whether the current value belongs to the set; if it is a format constraint, it is matched and determined according to the rules described in the above natural language; if the corresponding conditions are not met, it is determined as an anomaly of the corresponding type.

[0077] The already labeled exceptions, including those with missing constraints, missing values, out-of-bounds values, and format violations, are organized and annotated to obtain the location and type of difference of the abnormal fields in the original clinical dataset.

[0078] In summary, by comparing the differences between data content and constraint descriptions field by field, various anomalies in clinical data can be automatically identified, the location of abnormal fields can be accurately located and the type of difference can be clarified, and the standardized classification and labeling of abnormal data can be completed. This provides a clear basis for subsequent data completion and cleaning operations, and improves the accuracy and efficiency of clinical data governance.

[0079] S6. Based on the location of the abnormal field and the type of difference, generate data completion suggestions and data cleaning schemes for the target text.

[0080] In this embodiment of the invention, generating data completion suggestions and data cleaning schemes for the target text based on the location of the abnormal field and the type of difference includes: The abnormal field location and the difference type are mapped to a preset correction strategy table, and the completion strategy and cleaning strategy of the abnormal field location and the difference type are matched. The completion strategy is applied to the abnormal field position, and the strategy is instantiated for the difference type to obtain the completion suggestion for the target text; The cleaning strategy is applied to the location of the abnormal field, and the operation is bound to the difference type to obtain the cleaning operation of the target text; The completion suggestions and the cleaning operations are correlated and summarized to obtain the data completion suggestions and data cleaning scheme for the target text.

[0081] Specifically, the location of the abnormal field and its corresponding difference type are input into a preset correction strategy table. The table is then searched and matched according to the correspondence between the abnormal type and the records in the correction strategy table to determine the completion strategy and cleaning strategy corresponding to the location of the abnormal field and the difference type.

[0082] The matching completion strategy is applied to the corresponding abnormal field position. Combined with the specific behavior of the difference type, the general completion strategy is transformed into executable processing content adapted to the current abnormal field, and the completion suggestions of the target text are obtained.

[0083] The matched cleaning strategy is applied to the corresponding abnormal field position. According to the specific characteristics of the difference type, the cleaning strategy is fixedly bound to the corresponding executable processing action to obtain the cleaning operation of the target text.

[0084] The completion suggestions and cleaning operations corresponding to the same abnormal field position are combined accordingly. All the combined processing content is then organized to form a complete data completion suggestion and data cleaning plan for the target text.

[0085] It should be noted that the structure of the preset correction strategy table includes at least the following fields: exception type, data type, recommended completion strategy, and recommended cleanup strategy. The correction strategy table can be set as follows: For missing value exceptions with a numeric data type, the completion strategy is median filling, and the cleanup strategy is to retain null values ​​and mark them; for missing value exceptions with a fractional data type, the completion strategy is mode filling, and the cleanup strategy is to fill with "unknown"; for missing value exceptions with a date data type, the completion strategy is default value filling (e.g., 1970-01-01), and the cleanup strategy is to retain null values ​​and mark them. For value range out-of-bounds exceptions with a numeric data type, the completion strategy is boundary value truncation (set to the nearest boundary), and the cleanup strategy is to delete the record and mark it; for value range out-of-bounds exceptions with a fractional data type, the completion strategy is mapping to the closest enumeration item (e.g., mapping "male" to "male"), and the cleanup strategy is to delete and mark it. For format violation anomalies where the data type is string, the completion strategy is to extract the valid part (e.g., remove spaces), and the cleaning strategy is to delete and mark it. For missing constraint anomalies (i.e., the data field has no corresponding constraint description in the governance constraint items), regardless of the data type, no completion is performed, only a prompt is given, and the cleaning strategy is not to clean, but to prompt that the constraint needs to be manually defined.

[0086] The method for instantiating the strategy involves replacing the parameters (such as median, mode, and boundary values) in the general completion strategy with specific values ​​calculated in real time from the dataset or obtained from a historical knowledge base. For example, for missing value anomalies in the "age" field, the completion strategy is median imputation. It automatically calculates the median of all non-null values ​​in the age field in the dataset, such as 45, and then generates a completion suggestion: "Missing values ​​in the age field are suggested to be filled with the median 45".

[0087] The operation binding is achieved by associating a cleansing strategy with a specific executable operation (such as an SQL UPDATE statement or a Python pandas code snippet) and generating specific execution instructions with anomaly location information. For example, for a cleansing strategy of "delete the record and mark it", the generated instruction is: delete the corresponding row from the clinical data table and insert the location, difference type, and processing time of the anomaly field into the cleansing log table.

[0088] In summary, by matching abnormal information with a preset correction strategy table, the system automatically obtains suitable completion and cleaning strategies, transforms general strategies into specific executable processing content, and completes the association and summary of completion suggestions and cleaning operations. This enables the formation of standardized and directly implementable clinical data governance solutions, improving the efficiency and standardization of abnormal data processing.

[0089] like Figure 2 As shown in Example 2, this is a functional block diagram of a clinical data governance rule generation system based on literature evidence-driven according to an embodiment of the present invention.

[0090] The clinical data governance rule generation system 100 based on literature evidence-driven approach described in this invention can be installed in an electronic device. Depending on the functions implemented, the clinical data governance rule generation system 100 may include a data acquisition module 101, a retrieval and extraction module 102, a standard judgment module 103, a data mapping module 104, an anomaly localization module 105, and a protocol output module 106. The modules described in this invention can also be referred to as units, which are a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, stored in the memory of the electronic device.

[0091] In this embodiment, the functions of each module / unit are as follows: The data acquisition module 101 is used to receive the original clinical dataset and research field identifier of the target text, and obtain the data set of the target text; The retrieval and extraction module 102 is used to retrieve a set of documents from a preset literature database through the data set, and extract field identifiers and constraint descriptions from the literature set to form an initial rule set for the target text. The standard judgment module 103 is used to evaluate the source credibility of each rule in the initial rule set, and filter and merge them according to the evaluation results to obtain the standard rule set of the target text. The data mapping module 104 is used to map the field identifiers in the standard rule set to the data fields in the data set, thereby generating the governance constraint items of the target text; The anomaly localization module 105 is used to compare the data set with the governance constraints item by item to locate the abnormal field position in the original clinical dataset and determine the type of difference in the abnormal field position. The solution output module 106 is used to generate data completion suggestions and data cleaning solutions for the target text based on the location of the abnormal field and the type of difference.

[0092] In the several embodiments provided by this invention, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0093] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0094] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0095] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0096] This application embodiment can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for generating clinical data governance rules based on literature evidence, characterized in that, The method includes: S1. Receive the original clinical dataset and research domain identifier of the target text to obtain the dataset of the target text; S2. Obtain a literature set by retrieving a preset literature database through the data set, and extract field identifiers and constraint descriptions from the literature set to form an initial rule set for the target text; S3. Evaluate the source credibility of each rule in the initial rule set, and filter and merge them according to the evaluation results to obtain the standard rule set of the target text; S4. Map the field identifiers in the standard rule set to the data fields in the data set to generate the governance constraint terms of the target text; S5. Compare the data set with the governance constraints item by item to locate the abnormal field positions in the original clinical dataset and determine the type of difference in the abnormal field positions; S6. Based on the location of the abnormal field and the type of difference, generate data completion suggestions and data cleaning schemes for the target text.

2. The method for generating clinical data governance rules based on literature evidence as described in claim 1, characterized in that, The original clinical dataset and research domain identifiers for receiving the target text include: Obtain the original data records of the target text to obtain the original clinical dataset of the target text; The original clinical dataset is parsed to obtain the field types and field hierarchy of the target text; Using the field types as nodes and the field hierarchy as connecting edges, construct the field topology of the original clinical dataset; The research field identifier of the target text is received, and the research field identifier is associated with the field topology to obtain the data set of the target text.

3. The method for generating clinical data governance rules based on literature evidence as described in claim 1, characterized in that, The step of retrieving a pre-defined literature database through the data set to obtain a literature set, and extracting field identifiers and constraint descriptions from the literature set to form an initial rule set for the target text, includes: Using the field names in the data set as anchors, documents that simultaneously contain the field names and synonyms are located in the preset literature database to form an initial set of literature. Using the references in the initial screening set as inbound nodes and the cited documents in the initial screening set as outbound nodes, a citation network for the initial screening set is constructed. The core set of documents in the document database is then selected based on the node centrality in the citation network. Syntactic dependency parsing is performed on each document in the core document set, and the parsed limiting phrases that have dependency relationships with the field name are type-labeled to generate candidate constraint pairs for the target text. The candidate constraint pairs are subjected to conflict detection and consistency merging to obtain the constraint description of the target text. The field names corresponding to the constraint descriptions are used as field identifiers and summarized into the initial rule set of the target text.

4. The method for generating clinical data governance rules based on literature evidence as described in claim 3, characterized in that, The process of filtering the core literature set from the literature database based on the node centrality in the citation network includes: The frequency of citations of the references by other documents in the reference database is aggregated and analyzed to determine the in-chain centrality and out-chain centrality of the citation network. The outgoing chain centrality of the citation network is obtained by measuring the breadth and diffusion of the cited documents in relation to the other documents. The inbound centrality and outbound centrality are weighted and fused to obtain the comprehensive influence score of the referenced network; Based on the comprehensive influence score, all documents in the initial screening document set are sorted in descending order to obtain the target text's undetermined core document set; The core document set to be determined is deduplicated to obtain the core document set of the target text.

5. The method for generating clinical data governance rules based on literature evidence as described in claim 1, characterized in that, The process involves evaluating the source credibility of each rule in the initial rule set, and filtering and merging them based on the evaluation results to obtain a standard rule set for the target text, including: Extract the publication source identifier and peer review status identifier of the documents from which the rules in the initial rule set originate; Based on the publication source identifier, the literature collection is divided into core publication sources, extended publication sources, and general publication sources; and based on the peer review status identifier, the literature collection is divided into those that have passed peer review and those that have not been peer reviewed. Rules belonging to the core publication source and having passed peer review are marked as high-confidence rules; rules belonging to the extended publication source and having passed peer review are marked as medium-confidence rules; and rules belonging to the general publication source or not yet peer-reviewed are marked as low-confidence rules. The low-confidence rules are removed, and the high-confidence rules and the medium-confidence rules are combined to form the initial rule set of the target text; The initial rule set is merged and integrated to obtain the standard rule set for the target text.

6. The method for generating clinical data governance rules based on literature evidence as described in claim 5, characterized in that, The process of merging and integrating the initial rule set to obtain the standard rule set for the target text includes: The initial rule set is grouped using the field identifiers to obtain the field rule groups of the target text; The rule priority sequence of the field rule group is determined based on the publication source identifier and the peer review status identifier; By applying the rule priority sequence to the field rule group, consistency rules are applied, and redundant fields and logical conflicts in the merged results are eliminated to obtain the standard rule set for the target text.

7. The method for generating clinical data governance rules based on literature evidence as described in claim 1, characterized in that, The step of mapping field identifiers from the standard rule set to data fields in the dataset to generate governance constraints for the target text includes: Extract field identifiers from the standard rule set, construct a field identifier queue, and construct a data field queue from the data field names in the data set; Perform precise character matching between the field identifier queue and the data field queue, and output the precise mapping relationship of the target text; Perform synonym expansion matching on the field identifier queue and the data field queue to obtain the synonym mapping relationship of the target text; Semantic inclusion matching is performed on the field identifier queue and the data field queue to generate a semantic mapping relationship for the target text; The precise mapping relationship, the synonym mapping relationship, and the semantic mapping relationship are summarized into a field mapping table for the target text; By using the field mapping table, the constraint descriptions corresponding to the field identifiers in the standard rule set are associated with the matching data fields in the data set to generate the governance constraint items for the target text.

8. The method for generating clinical data governance rules based on literature evidence as described in claim 1, characterized in that, The step of comparing the data set with the governance constraints item by item to locate the abnormal fields in the original clinical dataset and determine the type of difference in the abnormal field locations includes: Traverse the data fields in the dataset and perform a step-by-step difference comparison of the constraint descriptions corresponding to the data fields in the governance constraint items; If the data field has no corresponding constraint description in the governance constraint item, then the position of the data field is marked as a missing constraint anomaly. If the data field has a corresponding constraint description but the current value of the data field is empty, then the position of the data field is marked as a missing value anomaly. If the current value of the data field does not meet the value range requirement in the constraint description, then the position of the data field is marked as a value range out-of-bounds exception; If the current value of the data field does not meet the format requirements in the constraint description, then the position of the data field is marked as a format violation exception; The missing constraint anomalies, missing value anomalies, value range out-of-bounds anomalies, and format violation anomalies are summarized and labeled to obtain the location and difference type of the abnormal fields in the original clinical dataset.

9. The method for generating clinical data governance rules based on literature evidence as described in claim 1, characterized in that, The step of generating data completion suggestions and data cleaning solutions for the target text based on the location of the abnormal field and the type of difference includes: The abnormal field location and the difference type are mapped to a preset correction strategy table, and the completion strategy and cleaning strategy of the abnormal field location and the difference type are matched. The completion strategy is applied to the abnormal field position, and the strategy is instantiated for the difference type to obtain the completion suggestion for the target text; The cleaning strategy is applied to the location of the abnormal field, and the operation is bound to the difference type to obtain the cleaning operation of the target text; The completion suggestions and the cleaning operations are correlated and summarized to obtain the data completion suggestions and data cleaning scheme for the target text.

10. A clinical data governance rule generation system based on literature evidence, characterized in that, The system for implementing the clinical data governance rule generation method based on literature evidence as described in claim 1 includes: The data acquisition module is used to receive the original clinical dataset and research area identifier of the target text, and obtain the data set of the target text; The retrieval and extraction module is used to retrieve a set of documents from a preset literature database through the data set, and extract field identifiers and constraint descriptions from the literature set to form an initial rule set for the target text. The standard judgment module is used to evaluate the source credibility of each rule in the initial rule set, and filter and merge them according to the evaluation results to obtain the standard rule set of the target text; The data mapping module is used to map the field identifiers in the standard rule set to the data fields in the data set, thereby generating the governance constraint items of the target text; An anomaly localization module is used to compare the data set with the governance constraints item by item to locate the abnormal field in the original clinical dataset and determine the type of difference in the abnormal field location. The solution output module is used to generate data completion suggestions and data cleaning solutions for the target text based on the location of the abnormal field and the type of difference.

Images