A clinical research information batch extraction system and extraction method

Abstract

The application discloses a clinical research information batch extraction system and method, belongs to the field of clinical research literature data processing, and solves the technical problems of low efficiency and poor accuracy of the existing extraction method, and lack of field logic constraints and no checking mechanism of the existing tool. The system comprises sequentially coordinated preprocessing, data acquisition, a core processing engine of a double-layer semantic checking architecture, and a summary output module. The method comprises four steps of identifier analysis and cleaning, metadata concurrent retrieval, double-layer semantic parallel extraction, and structured summary output. Identifier standardization is realized through adaptive regular analysis, data is preliminarily screened based on a PICO element semantic constraint template, logical checking and correction are completed by using a medical logic mutual exclusion rule library, and finally, a structured data set with a standard table header is generated and CSV / Excel export is supported. The application greatly reduces the data extraction error rate, shortens the processing period to the minute level, and enables the extracted data to be directly used for statistical analysis without secondary cleaning, thereby meeting the scientific research needs of evidence-based medical research.

Description

Technical Field

[0001] This invention belongs to the field of clinical research literature data processing, specifically relating to a system and method for batch extraction of clinical research information. Background Technology

[0002] In evidence-based medicine research, systematic reviews, and meta-analyses, accurately extracting structured clinical research information such as PICO elements, study type, and sample size from massive amounts of literature is a core step that directly determines the scientific validity and accuracy of subsequent research.

[0003] Currently, the extraction of clinical research information mainly relies on manual operation, which has significant technical drawbacks: First, the processing efficiency is extremely low. When faced with large-scale literature databases, manually reading, extracting, and recording data from each article is too time-consuming and cannot meet the timeliness requirements of systematic reviews. Second, the accuracy and standardization of data are insufficient. Manual extraction is easily affected by subjective factors, leading to errors and omissions in key data such as sample size and grouping details. Furthermore, different researchers have different understandings and recording standards for the same data field, making it difficult to unify and integrate the extracted data. Third, manual processing is costly, requiring a large investment of professional research personnel, resulting in a waste of research resources.

[0004] Existing technologies include some tools to assist in extracting literature data, such as conventional literature management software and text analysis tools based on general large models. However, these tools still cannot solve the core pain points of clinical research information extraction and have obvious technical limitations: First, they lack logical constraints specific to the clinical field. General models can only achieve keyword matching and extraction of surface information and cannot understand the mutually exclusive logic and dependencies unique to clinical research, easily leading to logically contradictory extraction results such as "extracting specific sample sizes from review articles." Second, relying on a single model for data extraction lacks effective logical self-verification and error correction mechanisms. The model is prone to generating erroneous data, and it cannot self-correct after generating errors, resulting in low reliability of the extraction results. Third, their unstructured text processing capabilities are weak. They cannot accurately convert complex intervention descriptions, population characteristics, and other free text in literature abstracts into standardized structured fields that meet the requirements of evidence-based medicine and statistics. The extraction results still require a large amount of manual secondary cleaning.

[0005] In summary, existing technologies lack an automated batch extraction technology for clinical research information that can adapt to the characteristics of the clinical research field and balance processing efficiency, data accuracy, and standardization. There is an urgent need for a new technical solution to address these issues and achieve efficient, accurate, and standardized batch extraction of clinical research information. Summary of the Invention

[0006] To address the technical problems of low efficiency and poor accuracy in manual processing of existing clinical research information extraction methods, and the lack of domain logic constraints, verification and correction mechanisms, and weak unstructured data processing capabilities in existing automated tools, this application designs a batch extraction system and method for clinical research information. By constructing a two-layer semantic verification architecture and combining adaptive regular expression parsing, high-concurrency retrieval, and medical logical mutual exclusion rule bases, the system achieves automated batch extraction of clinical research information. While significantly improving processing efficiency, it ensures the accuracy, logical consistency, and standardization of the extracted data, meeting the research needs of evidence-based medicine research, systematic reviews, and meta-analysis.

[0007] A system for batch extraction of clinical research information includes a preprocessing module, a data acquisition module, a core processing engine, and a summary output module that work in sequence and in collaboration. The preprocessing module is used to receive an unstructured text stream containing document identifiers (PMID (PubMed Unique Identifier, i.e., the unique numerical number of the PubMed document database), and cleans, segments, and removes duplicates from the text stream using an adaptive regular expression parsing algorithm to extract a string sequence consisting of pure numbers without any special characters or delimiters, thus generating a standardized unique target document identifier sequence. The data acquisition module is used to call the literature database interface (PubMed open interface or Helicopter Medical Literature Database interface) for batch concurrent retrieval based on the target literature identifier sequence to obtain the metadata object of the target literature. The required fields of the metadata object are pmid_id, title, publication year, doi, authors, journal name, and abstract. Optional fields include pmic, keywords, author affiliation information, NLM unique identifier, country, and publication type. The core processing engine is a two-layer semantic verification architecture, which includes cascaded initial screening units and verification units. It is used to extract metadata objects in parallel and perform logical verification and correction to generate the final structured record of a single document. The specific task scheduling algorithm for concurrent iteration is an asynchronous multi-threaded distribution mechanism, which maps the parsed multiple document data into independent task queues, configures a parallel iterative scheduling algorithm with a parallelism of 10, and introduces an abnormal termination protection mechanism to prevent a single dirty data from causing global pipeline blockage, thereby improving the system's throughput and stability. The summary output module is used to collect all final structured records, automatically generate standardized structured datasets and support format export. The dataset storage format supports direct export of two types: CSV and XLSX. The export trigger conditions include: (1) all concurrent iteration tasks are completed and automatically triggered after passing the format verification; (2) the user manually triggers the export on the interface.

[0008] Preferably, the adaptive regular expression parsing algorithm of the preprocessing module performs the following operations: A character set containing valid delimiters is constructed, and delimiters in the text stream are uniformly replaced with standard delimiters. The text is then segmented using the standard delimiters to obtain a candidate identifier list. Empty strings and noisy data with illegal formats are removed. Illegal format determination employs both morphological and logical checks. Morphologically, strings containing English letters, special symbols, or empty strings after regular expression cleaning are considered illegal. Logically, suspected garbled strings exceeding the threshold for regular identifiers (such as pure numeric sequences that are not 1 to 8 digits) are also intercepted and removed. Through these mechanisms, it is ensured that the data entering the downstream fully conforms to the target interface specification. The hash set is used to remove duplicates from the cleaned identifiers, generating a sequence of target document identifiers; The legal separators include English commas, Chinese commas, English semicolons, Chinese semicolons, tabs, and newlines.

[0009] Preferably, the initial screening unit is a high-throughput semantic model. The basic architecture of the high-throughput semantic model is a lightweight large language model based on Transformer (such as gpt-4.1-mini), which has contextual understanding capabilities. It uses system-level expert-level prompt word templates to map literature metadata as context and extract structured information. It has built-in semantic constraint templates containing PICO element definitions, and sets fine-grained extraction boundaries for each PICO element. For example, the extracted content of the study population includes the target disease type, age range, gender restriction, diagnostic criteria, inclusion criteria, and exclusion criteria; the extracted content of the intervention measures includes the specific intervention name, dosage, route of administration, frequency, course of treatment, and combined intervention regimen; the extracted content of the control method includes the control type (such as placebo control, positive control, blank control), control intervention name and parameters; the extracted content of the study results includes the primary endpoint, secondary endpoint, effect size, confidence interval, significance level, follow-up time point, etc. Utilizing the internal attention mechanism and multilayer perceptron of a large language model, the input document metadata is automatically mapped and feature extracted into high-dimensional semantic vectors through natural language processing. Key information fragments are located based on templates, and primary structured key-value pair data is forcibly output using JSON Schema. Key-value pair naming follows a snake-case naming convention, with each field name corresponding one-to-one with the output fields. These fields include article type (article_type), sample size (sample_size), study population (study_population), intervention measures (intervention), control methods (control), and research results (outcome), all of which are string types. The primary structured key-value pair data strictly adheres to the aforementioned JSON Schema constraints, ensuring the completeness of the output data's fields, type consistency, and parsability by downstream systems.

[0010] Preferably, the verification unit is a high-precision inference semantic model, employing a higher-order model with a larger number of parameters (such as GPT-4.1, Claude-Sonnet-3.7, etc.), which has stronger logical error correction and verification capabilities. The high-precision inference semantic model and the high-throughput semantic model form a cascaded architecture: the high-throughput semantic model is used for initial screening and structured extraction of batch documents, outputting primary structured key-value pair data; the high-precision inference semantic model, as the back-end verification module, receives the primary structured key-value pair data and combines it with the original document metadata for deep inference and consistency verification. Its model has a larger number of parameters and a higher inference latency than the high-throughput semantic model, in exchange for a higher verification accuracy. It incorporates a medical logical mutual exclusion rule base built based on PICO element causal and dependency relationships, containing no fewer than 20 mutual exclusion rules, such as type-sample size dependency rules and grouping-intervention integrity rules. Each rule is constructed based on clinical medical knowledge and literature corpora, with the selection criteria being the accuracy and coverage of logical conflict detection in the training and validation sets. The system compares the primary structured key-value pair data with the original literature metadata and performs logical consistency checks. If logical conflicts are found, an automatic correction strategy is triggered, with the correction priorities as follows: First priority, field coverage is performed based on the explicit statements in the original literature metadata; second priority, field correction is performed based on the mutual exclusion rule with the highest confidence in the rule base; third priority, conflicting fields are marked and handed over to high-precision inference tools. The semantic model performs secondary reasoning to complete the data; if data integrity is missing, a secondary backtracking search is triggered, with the trigger threshold being the absence of any core subfield of a PICO element (including the disease type of the study population, the name of the intervention, the type of control method, and the main endpoint of the study results); using the aforementioned dual literature database interface, the abstract of the literature and the full text paragraphs of the PMC original (if any) are retrieved, and the search terminates when three rounds of incremental retrieval are completed and no supplementary information is found, or all missing fields have been completed; the structured dataset generated by the summary output module includes PMID, title, publication year, journal name, abstract, DOI, article type, study population, sample size, intervention, and study results header, and supports export in CSV / Excel format.

[0011] Based on the aforementioned clinical research information batch extraction system, this application also designs a clinical research information batch extraction method, including the following steps: Step S1, Multi-source identifier parsing and cleaning: Receive an unstructured text stream containing document identifiers, and use an adaptive regular expression parsing algorithm to complete format normalization, word segmentation and noise reduction, and hash deduplication to generate a standardized and unique target document identifier sequence. If the input text stream is empty, or there are no usable legal identifiers after segmentation and cleaning, return an empty result (result: ) and terminate all subsequent execution logic. Step S2, Automated Metadata Acquisition: Based on the target document identifier sequence, call the document database interface for batch concurrent retrieval to obtain the target document metadata object containing title, publication year, DOI, and abstract; the retrieval has a built-in retry mechanism, automatically retrying when a single document identifier fails to be retrieved, with a maximum of 3 retries and an exponential backoff strategy for the retry interval, with an initial interval of 1 second; if it is still unsuccessful after 3 retries, the document identifier is marked as "no document found", the reason for failure and timestamp are recorded, and a retrieval failure report is generated. The retrieval failure report includes a list of failed document identifiers, a classification of failure reasons (such as interface timeout, empty result return, restricted permissions, and incorrect format), and suggested handling measures; Step S3: Parallel extraction based on a two-layer semantic architecture: The metadata object of a single document is processed independently using a concurrent iterative mode. The JSON data containing multiple documents is parsed and reorganized into a string array. Then, the iteration node is triggered, and a parallel independent task branch with a parallel count of 10 is started to extract the large model. First, the primary structured key-value pair data is extracted through the initial screening unit, and then the logical verification and correction are performed through the verification unit to generate the final structured record. If the verification unit fails the verification, the corresponding document is marked as "verification anomaly", and the anomaly type (including uncorrected logical conflicts, missing data integrity, and rule base matching failure) is recorded and output. Step S4, Structured Summary and Output: Collect all final structured records from concurrent processing, automatically generate a standardized structured dataset, and support direct copying or export to a specified format; data exchange between steps is conducted via JSON strings and string arrays, and the final summary supports direct export of both CSV and XLSX formats; CSV follows the RFC4180 standard, UTF-8 (BOM) encoding, uses commas as delimiters, encloses fields containing delimiters in double quotes, and uses CRLF for line breaks; the XLSX workbook is named "Batch Extraction of Clinical Research Information_YYYYMMDD_HHMMSS.xlsx", the first worksheet is named "Extracted", and the column order is consistent with the standard table header.

[0012] Preferably, in step S1, format normalization specifically involves constructing a set of legal delimiter characters S={',',',',';',';','\t','\n',''}, and uniformly replacing all characters belonging to S in the text stream with English commas; word segmentation and noise removal specifically involves segmenting the text with English commas as the boundary to obtain a list of candidate identifiers, traversing and removing empty strings and noise data containing illegal special characters. The illegal special characters include, but are not limited to, the following typical characters: @, #, $, %, ^, &, *, (, ), +, =, [, ], {,}, |, \, :, ", ', <, >, ?, / , `, ~, as well as Unicode control characters (such as \u0000-\u001F, \u007F) and full-width spaces.

[0013] Preferably, in step S3, the specific operations of the initial screening unit in extracting primary structured key-value pair data are as follows: loading a constraint template containing the semantic boundary definition of PICO elements, mapping the document abstract into a high-dimensional semantic vector, locating key information fragments according to the template; forcibly outputting primary structured key-value pair data in the form of structured key-value pairs through JSONSchema; the key information fragments include: article type, sample size, and intervention measures.

[0014] Preferably, in step S3, the medical logical mutual exclusion rule base of the verification unit includes at least type-sample size dependency rules, grouping-intervention integrity rules, population-disease consistency rules, intervention-outcome correlation rules, article type inference rules, and anti-hallucination rules. For example, the type-sample size dependency rule is that if the article type is a review, case report, or commentary, the sample size field is left empty. If there is a logical conflict between type and sample size in the primary structured key-value pair data, the article type is given priority and the sample size field is corrected. If the article type is a randomized controlled trial or cohort study, the total sample size value in the sample size field must be ≥1, and the sum of the intervention and control sample sizes must be equal to the total sample size value. The grouping-intervention integrity rule is that if the article type is a randomized controlled trial, the intervention field must include at least two group descriptions: experimental group and control group. Each group description must include the intervention name and at least two of the following: dosage, frequency, duration, and route of administration.

[0015] Preferably, in step S3, the specific operations of logical verification and correction are as follows: the primary structured key-value pair data is compared with the original literature metadata, and a medical logical mutual exclusion rule check is performed; if a logical conflict is detected, an automatic correction strategy is triggered to complete the data correction; if the completeness of the intervention field of the randomized controlled trial is missing, a secondary backtracking search is triggered to scan the abstract for text fragments containing opposing semantic expressions for the relationship between the experimental group and the control group to supplement the data; the opposing semantic expressions include "vs", "compared to", and "placebo", which do not rely on a hard-coded opposing semantic keyword library, but utilize the contextual semantic understanding capability of a high-order large language model, combined with user instructions, to review and complete the extracted intervention measures, thus avoiding the coverage limitations of the keyword library.

[0016] Preferably, in step S2, the metadata object further includes author, journal name, publicly available full text, keywords, author affiliation information, NLM unique identifier, country, and document type; in step S4, the header of the standardized structured dataset is PMID, title, publication year, journal name, abstract, DOI, article type, study population, sample size, intervention measures, research results, and conclusion, with the specified export format being CSV / Excel. The filling specifications for each header field are as follows: PMID: Unique identifier of the document, formatted as a pure numeric string; Title: Original title and Chinese translation of the document; Publication year: Four-digit format, such as "2020"; Journal name: Use the official full name or standard abbreviation of the journal; Abstract: Original abstract text and Chinese translation of the document, maintaining the original format; DOI: Unique identifier of the digital object, formatted as "10.xxxx / xxxxx"; Article type: Enumerated value, including randomized controlled trials, cohort studies, case-control studies, systematic reviews, meta-analyses, and cross-sectional studies. Case reports and reviews; Study population: including disease type, age range, gender, inclusion criteria, exclusion criteria, etc.; Sample size: including total sample size, experimental group sample size, and control group sample size. If the article type is a review or case report, leave it blank; Intervention: including intervention name, dosage, frequency, duration, route of administration, etc.; Study results: distinguish between quantitative and qualitative descriptions: quantitative descriptions include effect size, confidence interval, and p-value; qualitative descriptions include standardized conclusions such as "significant improvement," "no significant difference," and "not reported"; Conclusion: original text of the literature conclusions or structured inductive conclusions.

[0017] The advantages and effects of this application are as follows: Significantly improves the accuracy and logical consistency of data extraction: The technical features include a two-layer semantic verification architecture and a medical logical mutual exclusion rule base based on PICO elements. Through a cascaded processing method of high-throughput model initial screening and high-precision inference model verification, the extracted data undergoes a clinically specific logical consistency check. Automatic correction is triggered for logical conflicts, and secondary backtracking retrieval is triggered for missing data. This effectively eliminates erroneous data caused by model illusions and solves the problem of logical contradictions in the extraction results of existing tools. Compared with single model extraction, the error rate of key fields such as sample size and intervention measures is reduced by 40%-60%, and the extracted data meets the quality standards of the preliminary system evaluation.

[0018] Achieving efficient batch extraction of clinical research information: The technical features include automated preprocessing with adaptive regular expression parsing, batch concurrent literature metadata retrieval, and concurrent iterative single-article processing mode. This transforms the manual, sequential work of reading and extracting literature into parallel machine processing, eliminating the need for manual formatting of literature identifiers. This solves the problem of low efficiency in manual processing, reducing the original weeks-long work of literature screening and data extraction to minutes, significantly improving the efficiency of scientific research and accelerating the initiation and advancement of evidence-based medicine research.

[0019] Ensuring the standardization and reusability of extracted data: The technical feature is that the model is forced to output structured key-value pair data through JSONSchema, and the summary output module generates a structured dataset with fixed standard headers, eliminating the format differences caused by manual input, solving the problem of low standardization of existing extracted data, and the output structured data does not require secondary cleaning. It can be directly copied or exported as CSV / Excel format and directly imported into statistical software for subsequent analysis, reducing the secondary processing costs for researchers.

[0020] Enhancing the system's adaptability and usability: The technical features include an adaptive regular expression parsing algorithm in the preprocessing module that is compatible with unstructured text streams using various delimiters. It can automatically normalize and deduplicate identifier formats, solving the tedious problem of users manually formatting identifiers and ensuring the compliance of input for subsequent searches. At the same time, the system is compatible with mainstream literature database interfaces, enabling the acquisition of multi-dimensional literature metadata to meet the needs of extracting clinical research information in different research scenarios.

[0021] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0022] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0024] Figure 1 This is a flowchart of the method for batch extraction of clinical research information according to the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0026] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0027] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0028] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0029] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0030] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0031] Example 1: This example addresses the technical pain points of existing clinical research information extraction by constructing a closed-loop processing workflow of preprocessing, data acquisition, two-layer semantic extraction, and summary output. Through the sequential collaborative work of the preprocessing module, data acquisition module, core processing engine, and summary output module, combined with standardized batch extraction methods, it simulates the double-blind verification mode of clinical research experts, achieving efficient, accurate, and standardized batch extraction of clinical research information. The specific implementation of the technical solution of this invention is described in detail below with reference to the accompanying drawings.

[0032] Specific implementation of system modules This system adopts a linear processing architecture of "data preprocessing - batch retrieval - core extraction - summary output". The core processing engine is a two-layer semantic verification architecture. Each module is designed specifically based on the extraction needs of the clinical research field. The specific implementation principle and working process are as follows: The preprocessing module, acting as the system's input processing unit, receives unstructured text streams containing document identifiers from the user (such as messy text containing PMIDs, multi-line identifier lists, etc.). Its core function is to execute an adaptive regular expression parsing algorithm. First, it constructs a set of valid delimiter characters S={',',',',';',';','\t','\n',''}, uniformly replacing all non-standard delimiters in the text stream with English commas to normalize the format. Then, it segments the text using commas as boundaries to obtain a candidate document identifier list. This list is then traversed to remove empty strings, illegal format data containing special symbols, and other noisy data, completing word segmentation and noise reduction. Finally, utilizing the uniqueness of hash sets, it deduplicates the cleaned identifiers, generating a standardized, unique sequence of target document identifiers. This provides compliant input data for subsequent batch searches, solving the tedious problem of users manually formatting identifiers.

[0033] Data Acquisition Module: As the system's document data retrieval unit, based on the target document identifier sequence output by the preprocessing module, it calls the open interfaces of mainstream literature databases such as PubMed and uses a batch concurrent retrieval method to synchronously acquire the metadata objects of the target documents. The metadata objects at least include the title, publication year, DOI, author, journal name, and abstract. It can also retrieve the freely available full text (PMCArticle). Compared to serial retrieval, the concurrent retrieval mode significantly shortens the metadata acquisition time and improves the overall processing efficiency of the system.

[0034] Core processing engine: The core computing unit of the system, it is the key to achieving accurate extraction of clinical research information. It adopts a two-layer semantic verification architecture, which includes cascaded initial screening units and verification units. It uses a concurrent iterative mode to process the metadata object of each document independently, ensuring the efficiency of parallel extraction of multiple documents.

[0035] The initial screening unit is a high-throughput semantic model with a built-in semantic constraint template containing definitions of PICO elements (study population, intervention, control method, and study results). The template clearly defines the semantic boundaries of core fields such as article type, sample size, and intervention. For example, it defines that "research" articles must contain specific subject data. This unit maps literature abstracts to high-dimensional semantic vectors, locates key information fragments based on the semantic constraint template, and forces the model to output primary structured key-value pair data in structured key-value pair form through JSONSchema, avoiding the data non-standardization problem caused by the model outputting free text. The verification unit is a high-precision reasoning semantic model with a built-in medical logical mutual exclusion rule base constructed based on the causal and dependency relationships of PICO elements. Its core includes clinical research-specific logical rules such as type-sample size dependency rules and grouping-intervention integrity rules. This unit compares the initial structured key-value pair data output by the initial screening unit with the original literature metadata field by field, performing a logical consistency check: if a logical conflict is detected (e.g., a review article extracts a sample size of 120), an automatic correction strategy is triggered, prioritizing the article type and setting the sample size field to empty; if only a single-group description is extracted from the intervention field of a randomized controlled trial, it is marked as incomplete, triggering a secondary backtracking search, focusing on scanning text fragments in the abstract containing opposing semantic expressions for the control group relationship to supplement the data; these opposing semantic expressions include "vs", "compared to", and "placebo", ultimately generating a logically consistent and data-complete final structured record of a single literature article.

[0036] Verification Unit Medical Logic Mutual Exclusion Rule Base Rule ID: R001; Rule Category: Type-Sample Size Dependency; Rule Name: Empty Sample Size for Review / Case Reports; Rule Content: If the article type is review, case report, commentary, or editorial, the sample size field should be empty; Execution Logic: When the above types are detected and the sample size field is not empty, automatic correction is triggered, prioritizing the article type and emptying the sample size field.

[0037] Rule Number: R002; Rule Category: Type-Sample Size Dependence; Rule Name: RCT / Cohort Sample Size Integrity; Rule Content: If the article type is a randomized controlled trial or cohort study, the sample size must satisfy: total = intervention_group + control_group, and the sample size of each group ≥ 1; Execution Logic: When a discrepancy is detected between total and the sum of the group sample sizes, automatic correction is triggered, adjusting the group sample sizes proportionally based on the total value. If the proportion is not evenly divisible, a conflict is marked. Rule Number: R003; Rule Category: Type-Sample Size Dependence; Rule Name: Case-Control Sample Size Integrity; Rule Content: If the article type is a case-control study, the sample size must satisfy: total = cases_group + controls_group, and the sample size of each group ≥ 1; Execution Logic: When a discrepancy is detected between total and the sum of cases / controls, automatic correction is triggered, and the sample size of each group is adjusted proportionally based on the total value. Rule Number: R004; Rule Category: Grouping - Intervention Completeness; Rule Name: RCT Grouping Completeness; Rule Content: If the article type is a randomized controlled trial, the intervention field must contain descriptions of at least the experimental group and the control group; Execution Logic: When the intervention description is missing, it is marked as incomplete, triggering a secondary backtracking search, focusing on scanning supplementary data in the abstract containing text fragments that express the opposing semantics of the experimental group and the control group; the opposing semantics include "vs", "compared to", and "placebo"; Rule Number: R005; Rule Category: Grouping - Intervention Completeness; Rule Name: Grouping Description Element Completeness; Rule Content: If the article type is a randomized controlled trial or cohort study, each intervention description must include the intervention name and at least two of the following: dose, frequency, duration, and route of administration; Execution Logic: When any missing key element is detected in a group description, automatic correction is triggered, prioritizing retrieval and supplementation from the abstract or methods section. If the missing element is still present, it is marked as incomplete. Rule Number: R006; Rule Category: Population-Disease Consistency; Rule Name: Intervention-Disease Indication Association; Rule Content: The disease type in the study population should have a known clinical indication association with the intervention; Execution Logic: When it is detected that the disease type and the intervention name have no clinical relevance or have a contraindication association, automatic correction is triggered, prioritizing the correction of the intervention field based on the disease type, or marking a conflict. Rule Number: R007; Rule Category: Population-Disease Consistency; Rule Name: Population Age Reasonableness; Rule Content: If the age range of the study population does not match the known age distribution characteristics of the disease type (e.g., the age range of childhood diseases is >65 years old), then a check is triggered; Execution Logic: When an age range is detected to be inconsistent with the typical age of onset of the disease, it is marked as a logical conflict, triggering automatic correction, and age information is extracted a second time from the inclusion and exclusion criteria of the original method paragraph. Rule Number: R008; Rule Category: Intervention-Outcome Relevance; Rule Name: Outcome-Intervention Efficacy Association; Rule Content: The primary endpoint in the study results should have a known clinical efficacy association with the intervention; Execution Logic: When a primary endpoint is detected to have no clinical relevance to the expected efficacy of the intervention, automatic correction is triggered, prioritizing the correction of the primary endpoint field based on the intervention, or marking a conflict. Rule Number: R009; Rule Category: Intervention - Outcome Relevance; Rule Name: Reasonableness of Outcome Indicator Type; Rule Content: If the article type is a randomized controlled trial, the primary endpoint should be a quantitative indicator (such as survival rate, efficacy rate, effect size), and should not be a qualitative description; Execution Logic: When a purely qualitative description (such as "good effect") is detected as the primary endpoint, it is marked as a formatting error, triggering a secondary backtracking search to extract quantitative data from the results paragraph; Rule Number: R010; Rule Category: Data Integrity; Rule Name: Core Field Integrity; Rule Content: Completion is triggered when any core subfield of a PICO element is missing (disease type of the study population, name of the intervention, type of control method, or primary endpoint of the study outcome); Execution Logic: When a core field is detected to be missing, a secondary backtracking search is triggered, with the search scope including the abstract and the full text of the original article. The search terminates when three rounds of incremental search are completed or all missing fields have been completed. Rule Number: R011; Rule Category: Data Integrity; Rule Name: Sample Size Outlier Detection; Rule Content: Anomaly detection is triggered when the total sample size exceeds 10,000 or is less than 1; Execution Logic: When the sample size exceeds a reasonable threshold, it is marked as a suspected anomaly, triggering automatic correction, and the sample size is extracted a second time from the "Participants" or "Study Subjects" section of the Method paragraph. Rule Number: R012; Rule Category: Format Validation; Rule Name: Year Format Validation; Rule Content: The publication year must be a four-digit number, ranging from 1900 to the current year; Execution Logic: When an abnormal year format is detected (such as "2020" or "20"), it will be automatically corrected to a four-digit number format; if the year is outside the reasonable range, it will be marked as suspicious and submitted for manual review. Rule ID: R013; Rule Category: Format Validation; Rule Name: DOI Format Validation; Rule Content: DOI fields must conform to the format "10.xxxx / xxxxx"; Execution Logic: When a DOI format does not conform to the standard, it is automatically extracted from the original metadata; if it still fails, it is marked as a format exception. Rule Number: R014; Rule Category: Logical Consistency; Rule Name: Intervention-Sample Size Logic; Rule Content: If several pre-measure fields have multiple sets of descriptions, the sum of the grouped sample sizes in the sample size field should be consistent with the total sample size; Execution Logic: When the sum of the grouped sample sizes is detected to be not equal to the total sample size, automatic correction is triggered, with the total sample size taking precedence. Rule Number: R015; Rule Category: Logical Consistency; Rule Name: Population-Sample Size Logic; Rule Content: If the study population includes "inclusion criteria" and "exclusion criteria", the sample size should not be 0; Execution Logic: When an automatic correction is triggered when the sample size is 0 but inclusion and exclusion criteria descriptions are present, the sample size is extracted a second time from the method paragraph. Rule ID: R016; Rule Category: Article Type Inference; Rule Name: Abstract Feature Inference; Rule Content: If the abstract contains keywords such as "randomized", "randomly assigned", or "double-blind", the article type should be inferred to be a randomized controlled trial; Execution Logic: When an inconsistency is detected between the article type and the abstract features of the primary structured key-value pair data, automatic correction is triggered, and the abstract feature inference result is given priority. Rule ID: R017; Rule Category: Article Type Inference; Rule Name: Research Method Inference; Rule Content: If the method section contains keywords such as "cohort", "prospective", or "follow-up", the article type should be inferred as a cohort study; Execution Logic: When an inconsistency is detected between the article type and research method characteristics of primary structured key-value pair data, automatic correction is triggered, prioritizing the inference results from the method section; Rule Number: R018; Rule Category: Anti-Hallucination Rule; Rule Name: Numerical Consistency Check; Rule Content: The sample size value extracted from the abstract should be consistent with the value extracted from the method section of the full text, with an error not exceeding 5%; Execution Logic: When a numerical inconsistency is detected and the error exceeds 5%, automatic correction is triggered, prioritizing the adoption of the value in the method section, and recording the inconsistency marker. Rule Number: R019; Rule Category: Anti-Hallucination Rule; Rule Name: Entity Existence Verification; Rule Content: The name of the intervention extracted from the primary structured key-value pair data should appear at least once in the original abstract or method paragraph; Execution Logic: When an intervention name is detected not appearing in the original text, automatic correction is triggered, it is marked as a suspected hallucination, and a secondary backtracking search is triggered to re-extract the intervention from the abstract or full text. Rule Number: R020; Rule Category: Anti-Hallucination Rule; Rule Name: PICO Element Relevance Validation; Rule Content: The four elements—study population, intervention, control method, and study results—should form a coherent semantic context in the original text and should not be combined unrelatedly across paragraphs; Execution Logic: When it is detected that the four elements do not appear in a paragraph in the original text, automatic correction is triggered, key information fragments are relocated and extracted, and if there is still no correlation, it is marked as logically incoherent.

[0038] The summary output module, serving as the system's result output unit, collects the final structured records from all concurrent tasks within the core processing engine. It integrates the data according to preset standard headers, automatically generating a standardized structured dataset containing PMID, title, publication year, journal name, abstract, DOI, article type, study population, sample size, intervention measures, research results, and conclusions. This module allows users to directly copy the dataset or export it to CSV / Excel format. The exported data can be directly imported into statistical software such as SPSS and R for subsequent analysis without requiring manual secondary cleaning.

[0039] Specific implementation of extraction method like Figure 1 The diagram shown is a flowchart of the batch extraction method for clinical research information according to the present invention. This flowchart illustrates the entire process from unstructured text stream input to standardized structured dataset output, corresponding one-to-one with the S1-S4 extraction steps of the present invention. Each step is highly compatible with the system modules and seamlessly integrated. The specific implementation steps and operational details are detailed in conjunction with the attached diagram below: Step S1, Multi-source identifier parsing and cleaning: Corresponding Figure 1 In the "preprocessing module" stage, the unstructured text stream containing document identifiers input by the user is received and processed in three levels using an adaptive regular expression parsing algorithm: First, format normalization is performed, replacing all kinds of delimiters in the text stream with English commas; then, word segmentation and noise removal are performed, splitting the text and removing noisy data; finally, hash deduplication is performed to generate a standardized and unique sequence of target document identifiers. If all the parsed identifiers are in an illegal format, the processing flow is terminated and the user is notified of the input error.

[0040] Step S2, Automated Metadata Acquisition: Corresponding to Figure 1In the "Data Acquisition Module" section, based on the target document identifier sequence generated in step S1, the document database interface is called to perform batch concurrent retrieval, and the target document metadata object corresponding to each identifier is obtained synchronously. The metadata object contains core information such as title, publication year, DOI, author, journal name, and abstract. After the retrieval is completed, a set of document metadata objects is generated, providing a data foundation for subsequent information extraction.

[0041] Step S3, Parallel extraction based on a two-layer semantic architecture: corresponding to Figure 1 The "core processing engine - concurrent iteration" step is the core step of this invention. It uses a concurrent iteration mode to independently process the metadata of a single document in the document metadata object set, and is divided into two sub-steps: initial screening and verification. Sub-step 3.1: The initial screening unit loads the semantic constraint template, maps the literature abstract into a high-dimensional semantic vector, locates and extracts key information such as article type, sample size, and intervention measures, and outputs primary structured key-value pair data through JSONSchema; Sub-step 3.2: The verification unit calls the medical logic mutual exclusion rule base, compares the primary structured key-value pair data with the original metadata to perform a logical consistency check. If a logical conflict is detected, an automatic correction strategy is triggered. If a data integrity deficiency is detected, a secondary backtracking search is triggered, and finally, the final structured record of a single document is generated.

[0042] Step S4, Structured Summary and Output: Corresponding Figure 1 In the "Summary Output Module" section, all the final structured records generated by concurrent processing in step S3 are collected and automatically integrated into a standardized structured dataset with standard headers. Users can directly copy the dataset or export it to CSV / Excel format as needed, and the extraction work is completed.

[0043] The specific implementation of this invention, through the high degree of adaptation between system modules and extraction methods, combined with a two-layer semantic verification architecture and a medical logic mutual exclusion rule base, forms a complete automated batch extraction solution for clinical research information. It specifically solves all the core pain points of existing technologies, realizes efficient, accurate and standardized extraction of clinical research information, provides efficient research tools for evidence-based medicine research, systematic reviews and meta-analyses, and significantly reduces the literature data processing costs for researchers.

[0044] The above description is merely a preferred embodiment of the present invention and does not limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter alterations to these embodiments within the spirit and principles of the present invention, achieved through conventional substitutions or by achieving the same function without departing from the principles and spirit of the present invention, fall within the scope of protection of the present invention.

Claims

1. A system for batch extraction of clinical research information, characterized in that, It includes a preprocessing module, a data acquisition module, a core processing engine, and a summary output module that work in sequence. The preprocessing module is used to receive an unstructured text stream containing document identifiers, and to clean, segment, and deduplicate the text stream using an adaptive regular expression parsing algorithm to generate a standardized sequence of unique target document identifiers. The data acquisition module is used to call the document database interface to perform batch concurrent retrieval based on the target document identifier sequence, and obtain the metadata object of the target document; The core processing engine is a two-layer semantic verification architecture, which includes cascaded initial screening units and verification units, used to extract metadata objects in parallel and perform logical verification and correction to generate the final structured record of a single document. The summary output module is used to collect all final structured records, automatically generate standardized structured datasets, and support format export. The initial screening unit is a high-throughput semantic model with a built-in semantic constraint template containing PICO element definitions. It maps document metadata into high-dimensional semantic vectors, locates key information fragments based on the template, and forces the output of primary structured key-value pair data through JSONSchema. The PICO elements include the research population, intervention measures, control methods, and research results. The primary structured key-value pair data includes the fields of article type, sample size, intervention measures, and research population.

2. The clinical research information batch extraction system according to claim 1, characterized in that, The adaptive regular expression parsing algorithm of the preprocessing module performs the following operations: Construct a character set containing valid delimiters and uniformly replace the delimiters in the text stream with standard delimiters; divide the text by the standard delimiters to obtain a list of candidate identifiers, and remove empty strings and noisy data with illegal formats; The hash set is used to remove duplicates from the cleaned identifiers, generating a sequence of target document identifiers; The legal separators include English commas, Chinese commas, English semicolons, Chinese semicolons, tabs, and newlines.

3. The clinical research information batch extraction system according to claim 1, characterized in that, The verification unit is a high-precision reasoning semantic model with a built-in medical logical mutual exclusion rule base constructed based on the causal and dependency relationships of PICO elements. It compares the primary structured key-value pair data with the original literature metadata and performs a logical consistency check. If a logical conflict exists, an automatic correction strategy is triggered. If data integrity is missing, a secondary backtracking retrieval is triggered. The structured dataset generated by the summary output module includes PMID, title, publication year, journal name, abstract, DOI, article type, study population, sample size, intervention measures, and study results header, and supports export in CSV / Excel format.

4. A method for batch extraction of clinical research information, applied to the batch extraction system for clinical research information as described in any one of claims 1-3, characterized in that, Includes the following steps: Step S1, Multi-source identifier parsing and cleaning: Receive an unstructured text stream containing document identifiers, and use an adaptive regular expression parsing algorithm to complete format normalization, word segmentation and noise reduction, and hash deduplication to generate a standardized and unique target document identifier sequence; Step S2, Automated Metadata Acquisition: Based on the target document identifier sequence, call the document database interface to perform batch concurrent retrieval and obtain the target document metadata object containing title, publication year, DOI, and abstract; Step S3: Parallel extraction based on a two-layer semantic architecture: The metadata object of a single document is processed independently using a concurrent iterative mode. First, the primary structured key-value pair data is extracted through the initial screening unit, and then the logical verification and correction are performed through the verification unit to generate the final structured record. Step S4, Structured Summary and Output: Collect all the final structured records of concurrent processing, automatically generate a standardized structured dataset, and support direct copying or export to a specified format; In step S3, the specific operations of the initial screening unit to extract primary structured key-value pair data are as follows: loading a constraint template containing the semantic boundary definition of PICO elements, mapping the document abstract into a high-dimensional semantic vector, locating key information fragments according to the template, and forcibly outputting primary structured key-value pair data in the form of structured key-value pairs through JSONSchema; The key information segments include: article type, sample size, and intervention measures.

5. The method for batch extraction of clinical research information according to claim 4, characterized in that, In step S1, format normalization specifically involves constructing a set of legal delimiter characters S={',',',',';',';','\t','\n',''}, and uniformly replacing all characters belonging to S in the text stream with English commas; word segmentation and noise removal specifically involves dividing the text with English commas as the boundary to obtain a list of candidate identifiers, traversing and removing empty strings and noise data containing illegal special characters.

6. The method for batch extraction of clinical research information according to claim 4, characterized in that, In step S3, the medical logic mutual exclusion rule base of the verification unit includes at least type-sample size dependency rules and grouping-intervention integrity rules. The type-sample size dependency rule is that if the article type is a review, case report, or commentary, the sample size field is left empty. If there is a logical conflict between type and sample size in the primary structured key-value pair data, the article type is given priority and the sample size field is corrected. The grouping-intervention integrity rule is that if the article type is a randomized controlled trial, the intervention field must include descriptions of both the experimental group and the control group.

7. The method for batch extraction of clinical research information according to claim 6, characterized in that, In step S3, the specific operations of logical verification and correction are as follows: the primary structured key-value pair data is compared with the original literature metadata, and a medical logical mutual exclusion rule check is performed; if a logical conflict is detected, an automatic correction strategy is triggered to complete the data correction; if the completeness of the intervention field of the randomized controlled trial is detected to be missing, a secondary backtracking search is triggered to scan the abstract for supplementary data containing text fragments containing opposing semantic expressions for the relationship between the experimental group and the control group; the opposing semantic expressions include "vs", "compared to", and "placebo".

8. The method for batch extraction of clinical research information according to claim 4, characterized in that, In step S2, the metadata object also includes the author, journal name, and the full text that is freely available. In step S4, the header of the standardized structured dataset is PMID, title, publication year, journal name, abstract, DOI, article type, research population, sample size, intervention measures, research results, and conclusions, and the export format is specified as CSV / Excel.

Images