Clinical trial document auditing method and device, equipment and storage medium

Abstract

The application discloses a kind of clinical trial document auditing method, device, equipment and storage medium, the method includes: obtaining the clinical trial document to be audited, and the clinical trial document is structured and parsed, and structured field information is extracted according to the analysis result;Based on the clinical trial field knowledge rule base, the compliance reasoning is carried out to structured field information, and compliance analysis result is obtained;Based on compliance analysis result, cross-document association consistency analysis is carried out to clinical trial document, and the logical deviation between documents is identified according to consistency analysis result, and deviation detection result is obtained;Based on multidimensional risk quantification rule, risk assessment is carried out to deviation detection result, and structured audit report is generated according to risk assessment result.The application is structured and parsed, and compliance reasoning is carried out with field knowledge rule base, and cross-document association consistency analysis and multidimensional risk quantification evaluation are combined to generate structured audit report, improve the efficiency and reliability of clinical trial document auditing.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, device, and storage medium for reviewing clinical trial documents. Background Technology

[0002] Clinical trials are a crucial step in verifying the safety and effectiveness of drugs or medical devices, and the compliance review of their documentation system is a core task of regulatory on-site inspections. Clinical trial documentation encompasses various types of documents, including trial protocols, standard operating procedures (SOPs), case report forms (CRFs), monitoring reports, and electronic data collection (EDC) logs. The content is complex, logically interconnected, and must strictly adhere to Good Clinical Practice (GCP), ICH-GCP, and relevant NMPA guidelines.

[0003] Currently, compliance audits of clinical trial documents primarily rely on the following methods: First, manual review, where inspectors review each document based on their professional experience and compare it against regulatory requirements. This method is inefficient, with a complete on-site inspection taking several days to weeks, and is susceptible to personnel fatigue and subjective judgment, creating blind spots in the audit process. Second, rule-based screening tools based on keyword matching, such as the field validation functions built into some clinical trial management systems. These tools can only perform explicit logic or format checks, failing to understand the semantic meaning and contextual logic of documents, resulting in extremely low identification rates for deeper compliance issues such as "inconsistencies between SOP descriptions and regulatory intent" and "implicit deviations between execution records and SOP requirements." Third, using generalized large language models for document summarization or question-and-answer sessions, but these models are not fine-tuned with knowledge of the clinical trial field and lack systematic coding of professional regulations such as GCP, making it difficult to accurately determine compliance in specific scenarios (such as the distinction between adverse event reporting time limits). Furthermore, existing solutions generally lack cross-document correlation analysis capabilities and cannot automatically detect temporal logical contradictions or procedural violations between trial protocols, SOPs, and execution records. In summary, existing clinical trial document review technologies suffer from inefficiencies, insufficient semantic understanding, lack of cross-document consistency analysis, and strong subjectivity in risk quantification, making it difficult to meet regulatory requirements in terms of efficiency and reliability.

[0004] Therefore, there is an urgent need for a clinical trial document review method that can improve the efficiency and reliability of clinical trial document review. Summary of the Invention

[0005] The main objective of this invention is to provide a method, apparatus, device, and storage medium for reviewing clinical trial documents, aiming to solve the technical problems of insufficient efficiency and reliability in the review of clinical trial documents in the prior art.

[0006] To achieve the above objectives, the present invention provides a method for reviewing clinical trial documents, the method comprising the following steps: Obtain the clinical trial documents to be reviewed, perform structured parsing on the clinical trial documents, and extract structured field information based on the parsing results; Based on a knowledge rule base in the clinical trial domain, compliance reasoning is performed on the structured field information to obtain compliance analysis results; Based on the compliance analysis results, cross-document association consistency analysis is performed on the clinical trial documents. Logical deviations between documents are identified based on the consistency analysis results to obtain deviation detection results. The deviation detection results are assessed for risk based on multi-dimensional risk quantification rules, and a structured audit report is generated based on the risk assessment results.

[0007] Optionally, the step of performing compliance reasoning on the structured field information based on a clinical trial domain knowledge rule base to obtain compliance analysis results includes: The document type of the clinical trial document is identified based on the structured field information; Based on the document type, activate the corresponding subset of review dimension rules from the clinical trial domain knowledge rule base; The structured field information is matched against the subset of audit dimension rules to obtain the rule matching result; The rule matching results and the structured field information are input into a large language model for compliance reasoning to obtain compliance analysis results.

[0008] Optionally, before the step of performing compliance reasoning on the structured field information based on a clinical trial domain knowledge rule base to obtain compliance analysis results, the method further includes: An initial rule pool was constructed based on regulations and expert experience rules in the field of clinical trials. The initial rule pool is subjected to multiple rounds of expert consensus verification to obtain the target rule set that passes the consensus verification; The target rule set is encoded into a logical assertion structure, and the logical assertion structure is organized hierarchically to construct a knowledge rule base for the clinical trial domain.

[0009] Optionally, the step of performing cross-document association consistency analysis on the clinical trial documents based on the compliance analysis results, identifying logical deviations between documents based on the consistency analysis results, and obtaining deviation detection results includes: Based on the compliance analysis results, a hierarchical mapping model for the clinical trial documents is established. Based on the hierarchical relationship mapping model, cross-document association consistency analysis is performed on the clinical trial documents through a semantic vector comparison engine and a symbolic logic reasoning engine to obtain consistency analysis results. Based on the consistency analysis results, logical discrepancies between documents are identified, and discrepancy detection results are obtained.

[0010] Optionally, the step of performing cross-document association consistency analysis on the clinical trial documents based on the hierarchical relationship mapping model, using a semantic vector comparison engine and a symbolic logic reasoning engine, to obtain consistency analysis results includes: Based on the hierarchical relationship mapping model, standard documents, regulatory documents, and execution record documents are identified from the clinical trial documents; The semantic vector comparison engine is invoked to perform semantic vectorization comparison on the specification document and the regulatory document, generating semantic consistency analysis sub-results. The symbolic logic reasoning engine is invoked to compare the execution record document and the specification document with temporal logic and program logic, and to generate a logic consistency analysis sub-result. Based on the semantic consistency analysis sub-result and the logical consistency analysis sub-result, a consistency analysis result is generated.

[0011] Optionally, the step of performing a risk assessment on the deviation detection results based on multi-dimensional risk quantification rules and generating a structured audit report based on the risk assessment results includes: The deviation detection results are quantified and scored based on multi-dimensional risk quantification rules to obtain score values ​​for each dimension. The multi-dimensional risk quantification rules include scoring criteria for severity, frequency of occurrence, and traceability. The weighting coefficients for the severity dimension, the frequency of occurrence dimension, and the traceability dimension are determined according to the type of clinical trial study. Based on the score values ​​of each dimension and the corresponding weight coefficients, the comprehensive risk score value of the risk event in the deviation detection result is determined; The comprehensive risk score is compared with a preset risk level threshold. Based on the comparison result, the risk level of the risk event is determined and a structured audit report is generated.

[0012] Optionally, after determining the comprehensive risk score of the risk event in the deviation detection result based on the score values ​​of each dimension and the corresponding weight coefficients, the method further includes: Obtain the inference confidence of the large language model for the deviation detection results; When the inference confidence is less than a preset confidence threshold, the target dimension score among the score values ​​of each dimension is downgraded and a manual review mark is triggered. The comprehensive risk score is weighted and fused based on the inference confidence level to obtain the corrected comprehensive risk score.

[0013] Furthermore, to achieve the above objectives, the present invention also proposes a clinical trial document review device, the device comprising: The document parsing module is used to obtain clinical trial documents to be reviewed, perform structured parsing on the clinical trial documents, and extract structured field information based on the parsing results; The compliance analysis module is used to perform compliance reasoning on the structured field information based on a knowledge rule base in the clinical trial field, and obtain compliance analysis results. The document comparison module is used to perform cross-document association consistency analysis on the clinical trial documents based on the compliance analysis results, identify logical deviations between documents according to the consistency analysis results, and obtain deviation detection results. The report generation module is used to conduct risk assessment on the deviation detection results based on multi-dimensional risk quantification rules, and generate a structured audit report based on the risk assessment results.

[0014] Furthermore, to achieve the above objectives, the present invention also proposes a clinical trial document review device, the device comprising: a memory, a processor, and a clinical trial document review program stored in the memory and executable on the processor, the clinical trial document review program being configured to implement the steps of the clinical trial document review method as described above.

[0015] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing a clinical trial document review program, which, when executed by a processor, implements the steps of the clinical trial document review method as described above.

[0016] This invention discloses a method for acquiring clinical trial documents to be reviewed, performing structured parsing on the documents, and extracting structured field information based on the parsing results. Based on a clinical trial domain knowledge rule base, compliance reasoning is performed on the structured field information to obtain compliance analysis results. Based on the compliance analysis results, cross-document association consistency analysis is performed on the clinical trial documents, and logical deviations between documents are identified based on the consistency analysis results to obtain deviation detection results. A risk assessment is performed on the deviation detection results based on multi-dimensional risk quantification rules, and a structured review report is generated based on the risk assessment results. Because this invention uses structured parsing and domain knowledge rule base for compliance reasoning, combined with cross-document association consistency analysis and multi-dimensional risk quantification assessment to generate a structured review report, it improves the efficiency and reliability of clinical trial document review compared to existing technologies. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the first embodiment of the clinical trial document review method of the present invention; Figure 2This is a flowchart illustrating the second embodiment of the clinical trial document review method of the present invention; Figure 3 This is a flowchart illustrating the third embodiment of the clinical trial document review method of the present invention; Figure 4 This is a structural block diagram of the first embodiment of the clinical trial document review device of the present invention; Figure 5 This is a schematic diagram of the structure of a clinical trial document review device in the hardware operating environment involved in the embodiments of the present invention.

[0018] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0020] This invention provides a method for reviewing clinical trial documents, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the clinical trial document review method of the present invention.

[0021] In this embodiment, the clinical trial document review method includes steps S10 to S40: Step S10: Obtain the clinical trial document to be reviewed, perform structured parsing on the clinical trial document, and extract structured field information based on the parsing results.

[0022] It should be noted that the executing entity in this embodiment can be a computer server device with data processing, network communication, and program execution functions applied in clinical trial supervision scenarios, such as a server, tablet computer, or personal computer, or an electronic device capable of performing the above functions (such as a clinical trial document review device). The following uses a system including a clinical trial document review device (hereinafter referred to as the system) as an example to illustrate this embodiment and the following embodiments.

[0023] Understandably, the aforementioned clinical trial documents pending review can refer to various electronic or paper documents generated during the clinical trial process that require compliance review. Examples include standard operating procedures (SOPs), trial protocols, case report forms (CRFs), monitoring reports, electronic data capture system (EDC) logs, and informed consent forms.

[0024] It should be explained that the aforementioned structured parsing refers to the process of converting unstructured raw documents (such as PDFs, Word documents, or scanned images) into a machine-readable data format with clear hierarchy and tags through technologies such as Optical Character Recognition (OCR) and layout analysis. For example, extracting chapter titles, table content, paragraph text, and signature areas from a document.

[0025] Furthermore, the aforementioned structured field information can refer to key data items extracted from the structured parsed document content according to a predefined data model. Examples include subject ID, visit date, adverse event records, protocol deviation descriptions, data modification records, and signature information.

[0026] In its implementation, the system first acquires one or more clinical trial documents awaiting compliance review. These documents can originate from the system's front-end upload interface or from a pre-configured storage path. Next, the system performs structured parsing on each acquired clinical trial document. For scanned documents, the system uses an optical character recognition engine, combined with a semantic segmentation-based layout analysis algorithm, to distinguish the main text, table, and signature areas, converting the image content into editable text data. For editable PDF or Word documents, the system directly uses a rule-based structured extraction algorithm to identify the document type and its internal chapter hierarchy. Then, based on a pre-established unified data model, the system locates and extracts key field information from the parsed document content. These fields include, but are not limited to, the subject's unique identifier, the specific date of each visit, detailed records of adverse events, whether protocol deviations occurred and their causes, detailed data modification history, and various signature information. Finally, the system will classify and store all extracted structured field information according to document type and field category, and establish a three-level index structure of document-field-value to facilitate fast retrieval and retrieval in subsequent steps.

[0027] To facilitate understanding, the following example illustrates the concept, but does not limit the scope of this embodiment. For instance, suppose that during a site inspection of a multicenter randomized controlled clinical trial, the reviewer uploaded a batch of trial documents via the system's web interface. These documents included a scanned informed consent form (PDF) and an editable case report form (Word). After acquiring these two documents, the system, for the informed consent form (PDF), uses an optical character recognition engine to convert the scanned image into text and identifies the signature area and signing date field. For the case report form (Word), a structured extraction algorithm identifies the subject number as "SUBJ-001," the baseline visit date as "2025-03-15," and an empty systolic blood pressure field. Subsequently, the system extracts and stores "SUBJ-001," "2025-03-15," and the empty systolic blood pressure field as structured field information.

[0028] Step S20: Based on the clinical trial domain knowledge rule base, perform compliance reasoning on the structured field information to obtain compliance analysis results.

[0029] It should be explained that the aforementioned clinical trial knowledge rule base can refer to a pre-built, structured knowledge set specifically designed for clinical trial compliance review scenarios. This knowledge set integrates regulations such as GCP, ICH, and NMPA, as well as operational experience rules validated by experts. Each rule is typically encoded in the form of a logical assertion, such as "IF AE type = SAE AND (sponsor's report time - SAE occurrence time) > 7 calendar days, THEN mark it as an SAE report timeout violation".

[0030] It should be noted that the aforementioned compliance reasoning refers to the process by which the system matches the extracted structured field information with rules in the clinical trial knowledge rule base, and then uses a large language model to perform semantic understanding and logical judgment on the matching results. This reasoning process aims to determine whether specific operations or records in the document comply with regulatory and professional standards.

[0031] It should be understood that the above compliance analysis results may refer to the conclusive information output by the system after the above reasoning. This information may include at least the type of compliance issue discovered, the specific rule violation, the preliminary determination of the severity, and the cited regulatory provisions.

[0032] To enhance the professionalism and accuracy of the review process, the step of performing compliance reasoning on the structured field information based on a clinical trial domain knowledge rule base to obtain compliance analysis results may include: identifying the document type of the clinical trial document based on the structured field information; activating the corresponding review dimension rule subset from the clinical trial domain knowledge rule base based on the document type; performing conditional matching between the structured field information and the review dimension rule subset to obtain rule matching results; and inputting the rule matching results and the structured field information into a large language model for compliance reasoning to obtain compliance analysis results.

[0033] It should be understood that the above document types can refer to the functional category to which clinical trial documents belong in the review process, such as standard operating procedures (SOPs), trial protocols, case report forms (CRFs), monitoring reports, informed consent forms, or electronic data capture system (EDC) logs.

[0034] It should be explained that the aforementioned subset of review dimensions rules can refer to a set of rules in the clinical trial knowledge rule base that is associated with a specific document type, such as a set of rules for compliance of informed consent signing procedures, or a set of rules for completeness review of monitoring reports.

[0035] Understandably, the above conditional matching can refer to the operation of logically comparing the specific values ​​in the structured field information with the triggering conditions of each rule in the rule subset, such as determining whether the time difference exceeds a threshold, whether the field value is empty, or whether the value falls within a specified range.

[0036] Furthermore, the above rule matching result can refer to the information output after the condition matching operation is completed. This information includes at least the rule identifier that was successfully triggered, the condition item that was successfully matched, and the trigger basis value extracted from the structured field information.

[0037] It should be understood that the aforementioned large language model can refer to a large-scale pre-trained language model trained based on the Transformer architecture, which has the ability to understand and generate natural language and can perform context-related semantic reasoning and judgment on the structured information input.

[0038] In its implementation, after acquiring the structured field information, the system first identifies the document type of the clinical trial document being processed based on the metadata or content features carried in the extracted structured field information. For example, the system determines whether the document belongs to a trial protocol or a monitoring report by detecting whether the document title contains the words "standard operating procedure" or by parsing the internal chapter structure of the document.

[0039] Next, based on the identified document type, the system searches for and activates a subset of pre-associated review dimension rules from the clinical trial domain knowledge rule base. Different document types correspond to different review focuses. For example, for informed consent forms, the system activates a subset of rules regarding the signing date, the subject's or legal representative's signature, and the record of the informed consent process; for electronic data collection system logs, the system activates a subset of rules regarding the number of data modifications, the reasons for modifications, and the identity of the modifier.

[0040] The system then performs conditional matching between the extracted structured field information and each rule in the activated subset of audit dimension rules. The system checks each field value in the structured field information to see if it meets the triggering conditions defined in the rule; for example, it calculates the difference between two dates and compares it to a threshold, or checks if a necessary field is not empty. For successfully matched rules, the system records the corresponding rule identifier, the violation type defined by the rule, the preset severity score, and the field information that triggered the rule, generating a rule matching result.

[0041] Subsequently, the system inputs the rule matching results and the original structured field information into a large language model. The system then invokes pre-built prompt word templates, instructing the large language model to perform deeper compliance reasoning based on the input matching results and original field values. The large language model analyzes whether the rule matching results are consistent with the document's context, determines whether there are reasonable exceptions (such as communication interruptions caused by natural disasters), and supplements the relevant legal provisions.

[0042] Finally, the system receives the inference output returned by the large language model and combines it with the previous rule matching results to generate the final compliance analysis result.

[0043] It should be understood that by accurately activating a subset of document type identification and review dimension rules, the full rule traversal is avoided, thus improving reasoning efficiency. By combining the rule matching results with a large language model, the precise constraints of the rule engine and the deep integration of semantic reasoning are achieved, which not only ensures the professionalism and accuracy of the review, but also enhances the adaptability to different document types.

[0044] Step S30: Based on the compliance analysis results, perform cross-document association consistency analysis on the clinical trial documents, identify logical deviations between documents according to the consistency analysis results, and obtain deviation detection results.

[0045] It should be explained that the aforementioned cross-document association consistency analysis can refer to the process by which the system cross-validates the logical, temporal, or referential relationships between multiple clinical trial documents. For example, it can verify whether the operation time in the execution record document meets the time constraints specified in the standard operating procedure (SOP), or verify whether the subject enrollment date recorded in the case report form (CRF) is later than the date of signing the informed consent form.

[0046] It should be noted that the aforementioned logical deviations can refer to inconsistencies or contradictions found between different documents. Such deviations cannot be detected through internal review of a single document and are typically manifested as violations of timing, incorrect program order, missing operations, or semantic inconsistencies. For example, if a standard operating procedure requires that "a record be completed within 24 hours of the occurrence of an adverse event," but the execution record shows that the actual recording time was 48 hours later, this constitutes a timing logical deviation.

[0047] Furthermore, the aforementioned deviation detection results may refer to the conclusion information output by the system after cross-document consistency analysis. This information includes at least a description of the deviation type, the document identifier involved, the specific location where the deviation occurred (such as a field or chapter), and a preliminary judgment on the severity of the deviation.

[0048] In its implementation, after obtaining the compliance analysis results, the system performs cross-document consistency analysis to further uncover inconsistencies between documents that cannot be detected by a single document review. First, based on the structured field information and document identifiers involved in the compliance analysis results, the system determines the document combinations requiring consistency verification. For example, the system identifies a trial protocol, a corresponding standard operating procedure (SOP), and several execution record documents. Next, the system establishes a hierarchical mapping between these clinical trial documents. Based on the inherent logic of document types, the system defines the trial protocol as a higher-level regulatory document, the SOP as a detailed operational document, and the case report form and electronic data acquisition system log as execution documentation documents. Subsequently, based on this mapping, the system performs consistency analysis on different types of document pairs. For consistency between regulatory documents and standard documents, the system performs semantic-level comparison; for consistency between execution record documents and regulatory documents, the system performs temporal and procedural logic comparison. The system determines whether there are any violations of constraints by calculating time differences, verifying the order of operations, and checking whether necessary operations have corresponding records. When the system detects a violation of a constraint, it records this as a logical deviation. Each logical deviation detection result includes at least: the deviation type (e.g., timing violation, program violation, or operational omission), the source and target document identifiers involved, the specific field or section where the deviation occurred, and the specific value or content that caused the deviation. Finally, the system summarizes all detected logical deviations to form a deviation detection result, which is used in subsequent risk assessment steps.

[0049] To improve the comprehensiveness and depth of clinical trial document review, the step of performing cross-document association consistency analysis on the clinical trial documents based on the compliance analysis results, identifying logical deviations between documents based on the consistency analysis results, and obtaining deviation detection results may include: establishing a hierarchical relationship mapping model between the clinical trial documents based on the compliance analysis results; performing cross-document association consistency analysis on the clinical trial documents based on the hierarchical relationship mapping model using a semantic vector comparison engine and a symbolic logic reasoning engine to obtain consistency analysis results; and identifying logical deviations between documents based on the consistency analysis results to obtain deviation detection results.

[0050] It should be explained that the aforementioned hierarchical mapping model can refer to a data structure used to describe the inherent logical connections and dependencies between clinical trial documents. This model defines the hierarchical and referential relationships between document types. For example, the trial protocol is defined as the superordinate document, the standard operating procedure (SOP) is defined as the operational refinement of the trial protocol, and the case report form (CRF) and execution record document are defined as evidence of the execution of the standard operating procedure.

[0051] It should be understood that the aforementioned semantic vector comparison engine can refer to a functional module that converts text fragments into high-dimensional vector representations and measures the semantic similarity of texts by calculating the distance between vectors (e.g., cosine similarity). This engine is used to detect whether there are semantic differences between normative documents (such as SOPs) and regulatory documents (such as GCP clauses).

[0052] The aforementioned symbolic logic reasoning engine can refer to a functional module that performs reasoning and judgment on structured data based on formal logic rules (such as sequential logic and first-order predicate logic). This engine converts the operational requirements and execution facts in the document into logical assertions or triples, and detects the existence of logical violations through symbolic comparison (such as time difference calculation and sequence verification).

[0053] It should be noted that the consistency analysis results mentioned above refer to the comprehensive judgment information on the consistency between documents output by the system after comparison and analysis by the semantic vector comparison engine and the symbolic logic reasoning engine. This information includes at least semantic consistency indicators (such as similarity scores) and logical consistency indicators (such as Boolean values ​​indicating whether temporal constraints are met).

[0054] In its implementation, before performing cross-document association consistency analysis, the system first establishes a hierarchical relationship mapping model between the clinical trial documents based on the document types and structured field information involved in the aforementioned compliance analysis results. The system constructs the relationship between documents according to the preset classification of document types (such as protocols, SOPs, execution records, and monitoring reports), following a logical chain of "higher-level specifications → operational refinement → execution documentation → supervision confirmation." For example, the system identifies the trial protocol document as the top-level specification document, the multiple standard operating procedure documents corresponding to the protocol as mid-level operational documents, and the electronic data acquisition system logs and case report forms as bottom-level execution documents. The system records the parent-child or reference relationships between these documents, forming a hierarchical relationship mapping model.

[0055] Next, based on the hierarchical relationship mapping model, the system invokes a semantic vector comparison engine and a symbolic logic reasoning engine to perform cross-document association consistency analysis on the clinical trial documents. At the semantic comparison level, the system uses the semantic vector comparison engine to convert the specific operational clause texts in the standard operating procedure (SOP) document and the corresponding GCP or ICH regulatory clause texts into semantic vectors, calculating the cosine similarity of each pair of clauses. Clause pairs with similarity below a preset threshold (e.g., 0.75) are marked as candidates with semantic discrepancies. At the logical comparison level, the system uses the symbolic logic reasoning engine to extract execution fact triples from the execution record document, such as <operation subject, operation behavior, timestamp>, and simultaneously extract normative constraint triples from the SOP document, such as <operation subject, operation requirement, time constraint>. The symbolic logic reasoning engine performs logical judgments item by item by comparing whether the time difference exceeds the constraint threshold, whether the operation sequence meets the preconditions, and whether the necessary operation has a corresponding record. The system integrates the outputs of the semantic comparison engine and the symbolic logic reasoning engine to generate the consistency analysis results. The consistency analysis results include a semantic consistency list (containing similarity scores below a threshold and descriptions of their differences) and a logical consistency list (containing specific events that violate timing, procedural, or integrity constraints).

[0056] Subsequently, based on the consistency analysis results, the system automatically identifies logical discrepancies between documents. For each item in the semantic consistency list, the system classifies it as a semantic-level discrepancy, such as "a clause in the SOP differs semantically from the GCP regulation." For each item in the logical consistency list, the system classifies it as a logical-level discrepancy, such as "the execution record shows the operation completion time is later than the time limit specified in the SOP" or "an operation required by the SOP is missing in the execution record." The system generates a structured discrepancy record for each discrepancy, which includes at least the discrepancy type (semantic or logical), the source and target document identifiers involved, a detailed description of the discrepancy, and evidence information leading to the discrepancy (such as similarity values ​​or time differences). Finally, the system aggregates all discrepancy records to form a discrepancy detection result.

[0057] To facilitate understanding, the following explanation uses examples, but does not impose specific limitations on this embodiment. For example, suppose there is a Standard Operating Procedure (SOP-101) document containing the clause "Adverse events should be reported promptly," and a GCP regulatory document containing the clause "Serious adverse events should be reported within 24 hours of becoming known." The system establishes a hierarchical relationship mapping model, identifying SOP-101 as the regulatory document and the GCP document as the regulatory document. The system calls a semantic vector comparison engine to convert the text of the two clauses into semantic vectors, calculating a cosine similarity of 0.62, which is below the threshold of 0.75, indicating a semantic discrepancy. The system records the discrepancy: the word "promptly" in SOP-101 lacks a specific time limit definition, which is semantically inconsistent with the "24 hours" in the regulation. Simultaneously, suppose there is another execution record document showing that a serious adverse event was reported 48 hours after its occurrence. The system invokes the symbolic logic reasoning engine to extract the fact triple <Sponsor, reported adverse event, 2026-06-10> and the normative constraint triple <Sponsor, reported serious adverse event, ≤24 hours>. The time difference is calculated to be 24 hours, indicating a temporal logic deviation. The system also records another deviation: the execution record violates SOP-101 and regulatory time limits. The system combines these two deviations to generate the final deviation detection result.

[0058] It should be understood that by constructing a document hierarchy relationship mapping model to clarify the relationships between multiple documents, and combining semantic vector comparison and symbolic logic reasoning as dual engines, cross-document deep consistency analysis is achieved, thereby breaking through the limitations of isolated single-document review, effectively identifying implicit logical deviations such as temporal violations and program deviations, and improving the comprehensiveness and depth of review.

[0059] Furthermore, to improve the accuracy and coverage depth of cross-document consistency analysis, the step of performing cross-document association consistency analysis on the clinical trial documents based on the hierarchical relationship mapping model using a semantic vector comparison engine and a symbolic logic reasoning engine to obtain consistency analysis results may include: identifying regulatory documents, legal documents, and execution record documents from the clinical trial documents based on the hierarchical relationship mapping model; calling the semantic vector comparison engine to perform semantic vectorization comparison on the regulatory documents and the legal documents to generate semantic consistency analysis sub-results; calling the symbolic logic reasoning engine to perform temporal logic and program logic comparison on the execution record documents and the regulatory documents to generate logical consistency analysis sub-results; and generating a consistency analysis result based on the semantic consistency analysis sub-results and the logical consistency analysis sub-results.

[0060] Understandably, the aforementioned normative documents can refer to documents within the clinical trial documentation system that specify operational standards or procedural requirements, such as standard operating procedures (SOPs) or trial protocols. The aforementioned regulatory documents can refer to external laws, regulations, or guidelines that clinical trials must follow, such as GCP (Good Clinical Practice), ICH guidelines, or relevant regulations issued by the NMPA. The aforementioned execution record documents can refer to documents that record the specific operational processes or results of clinical trials, such as electronic data capture system (EDC) logs, case report forms (CRFs), or monitoring reports.

[0061] It should be explained that the aforementioned semantic vectorization comparison can refer to converting two text segments into semantic vector representations respectively, and measuring the degree of semantic consistency between the two text segments by calculating the similarity between the vectors (e.g., cosine similarity). The aforementioned semantic consistency analysis sub-result can refer to the information output after semantic vector comparison, which includes at least the similarity score between text pairs and the identification of text segments with similarity below a preset threshold.

[0062] The aforementioned temporal logic comparison can refer to the logical judgment process of verifying whether the operation time points or time intervals in the execution record document meet the time constraints specified in the specification document. The aforementioned procedural logic comparison can refer to the logical judgment process of verifying whether the operation sequence or preconditions in the execution record document meet the procedural requirements specified in the specification document. The aforementioned logical consistency analysis sub-result can refer to the information output after symbolic logic reasoning, which at least includes a list of events that violate temporal or procedural constraints and corresponding deviation descriptions.

[0063] In its implementation, after obtaining the hierarchical relationship mapping model, the system first identifies regulatory documents, standard documents, and execution record documents from the set of clinical trial documents to be reviewed, based on the document type tags defined in the model. For example, the system identifies documents with the type tags "SOP" or "trial protocol" as regulatory documents, documents with the type tags "GCP" or "ICH" as standard documents, and documents with the type tags "EDC log," "CRF," or "monitoring report" as execution record documents.

[0064] Next, the system invokes a semantic vector comparison engine to perform semantic vectorization comparison on the identified normative and regulatory documents. The system extracts specific operational clauses from the normative documents and corresponding regulatory clauses from the regulatory documents. The semantic vector comparison engine converts each pair of text segments into high-dimensional semantic vectors and calculates the cosine similarity between each pair. The system compares the calculated similarity value with a preset semantic consistency threshold (e.g., 0.75). For text pairs with similarity below the threshold, the system records the source text identifier, target text identifier, and similarity score, forming a semantic consistency analysis sub-result. This sub-result indicates semantic differences between the statements in the normative documents and the requirements in the regulatory documents.

[0065] Simultaneously or sequentially, the aforementioned system invokes the symbolic logic inference engine to perform temporal logic comparison and program logic comparison operations on the identified execution record documents and specification documents. In the temporal logic comparison, the symbolic logic inference engine extracts fact triples related to operation time from the execution record documents (e.g., <operation name, occurrence timestamp, operation subject>) and extracts the corresponding time constraints for the operation from the specification documents (e.g., <operation name, allowed delay limit, constraint type>). The engine calculates the difference between the actual time difference and the constraint threshold; if it exceeds the threshold, it is determined to be a temporal violation. In the program logic comparison, the symbolic logic inference engine extracts operation sequence information from the execution record documents and extracts the preconditions or sequence requirements for the operation from the specification documents (e.g., operation A must be completed before operation B). The engine verifies whether the actual operation sequence meets the above requirements; if not, it is determined to be a program violation. The system summarizes all temporal and program violation events. Each violation record includes at least the violation type (temporal or program), the document identifier involved, the operation description, and the specific deviation value (e.g., delay hours or incorrect order), forming a logical consistency analysis sub-result.

[0066] Subsequently, the system generates the final consistency analysis result based on the semantic consistency analysis sub-result and the logical consistency analysis sub-result. This consistency analysis result is a data structure comprising two parts: the first part is a semantic consistency list, where each item corresponds to a semantic deviation candidate; the second part is a logical consistency list, where each item corresponds to a timing or program logic deviation.

[0067] It should be understood that by distinguishing between normative, regulatory, and execution record documents and calling the dual engines respectively, complementary review at the semantic and logical levels is achieved. Semantic vector comparison captures implicit semantic deviations, and symbolic logic reasoning locks in timing and procedural violations, thereby improving the accuracy and coverage depth of cross-document consistency analysis.

[0068] Step S40: Perform a risk assessment on the deviation detection results based on multi-dimensional risk quantification rules, and generate a structured audit report based on the risk assessment results.

[0069] It should be noted that the aforementioned multi-dimensional risk quantification rules can refer to a predefined set of judgment criteria used to quantify and score compliance deviations or violations from multiple different perspectives. These multiple dimensions typically include severity, frequency of occurrence, and traceability. For example, the severity dimension is scored from 1 to 5 based on the impact of the violation on subject safety or data validity; the frequency of occurrence dimension is scored from 1 to 5 based on the number of times the violation occurred; and the traceability dimension is scored from 1 to 5 based on whether the operation records are traceable.

[0070] Understandably, the aforementioned risk assessment can refer to the process by which the system calculates scores, weights and summarizes each logical deviation or compliance issue in the deviation detection results, and classifies them into levels, based on multi-dimensional risk quantification rules.

[0071] It should be noted that the aforementioned structured audit report can refer to an audit conclusion document generated by the system and organized in a structured data format (such as JSON, XML, or a document with explicit headings and tables). This report must at least include a list of risk events, the risk level of each risk event, a comprehensive risk score, and the corresponding regulatory basis or rectification recommendations.

[0072] In its implementation, after obtaining the deviation detection results, the system begins a risk assessment based on multi-dimensional risk quantification rules to transform these results into risk rating conclusions that can be directly used by regulatory personnel. First, the system reads pre-built multi-dimensional risk quantification rules from its internal storage. These rules define clear scoring criteria for different types of deviation events. For example, for the severity dimension, the rules stipulate that deviations that "directly endanger the safety of subjects or affect the validity of trial conclusions" are scored 5 points, while deviations that "affect operational standardization but do not directly affect data" are scored 3 points. For the frequency dimension, the rules stipulate that one deviation is scored 1 point, two to three deviations are scored 2 points, and so on. For the traceability dimension, the rules stipulate that "no operation records or signatures" are scored 5 points, while "a complete electronic or paper record chain" is scored 1 point.

[0073] Next, the system iterates through each deviation record in the deviation detection results. For each deviation, the system matches the corresponding scoring rule from the multi-dimensional risk quantification rules based on its type and description, determining the severity score, frequency score, and traceability score respectively. Subsequently, based on the current clinical trial study type (e.g., interventional randomized controlled trial, observational study, or device trial), the system reads the weight coefficients corresponding to the severity dimension, frequency dimension, and traceability dimension from a preset weight coefficient table. The system multiplies the scores of each dimension by the corresponding weight coefficients and sums them to calculate the comprehensive risk score of the deviation. The system also compares the comprehensive risk score with preset risk level thresholds (e.g., high-risk threshold 7.0, medium-risk threshold 4.0) to determine whether the risk level of the deviation is high-risk, medium-risk, or low-risk.

[0074] The system then aggregates the risk assessment results of all deviations and generates a structured audit report. This structured audit report includes at least the following: an audit overview (e.g., total number of audited documents, total number of risk events identified), a detailed list of risk events (each risk event includes a deviation description, relevant document identifier, severity score, frequency score, traceability score, overall risk score, risk level, and applicable regulatory provisions), and a visual risk distribution chart (e.g., bar chart or pie chart). The system presents the generated structured audit report to the auditors in a downloadable or online format.

[0075] This embodiment discloses a method for acquiring clinical trial documents to be reviewed, performing structured parsing on the documents, and extracting structured field information based on the parsing results. Based on a clinical trial domain knowledge rule base, compliance reasoning is performed on the structured field information to obtain compliance analysis results. Based on the compliance analysis results, cross-document association consistency analysis is performed on the clinical trial documents, and logical deviations between documents are identified based on the consistency analysis results to obtain deviation detection results. A risk assessment is performed on the deviation detection results based on multi-dimensional risk quantification rules, and a structured review report is generated based on the risk assessment results. Because this embodiment uses structured parsing and a domain knowledge rule base for compliance reasoning, combined with cross-document association consistency analysis and multi-dimensional risk quantification assessment to generate a structured review report, it improves the efficiency and reliability of clinical trial document review compared to existing technologies.

[0076] refer to Figure 2 , Figure 2 This is a flowchart illustrating the second embodiment of the clinical trial document review method of the present invention.

[0077] Based on the first embodiment described above, in this embodiment, before step S20, steps S101 to S103 are further included: Step S101: Construct an initial rule pool based on regulatory provisions and expert experience rules in the field of clinical trials.

[0078] Step S102: Perform multiple rounds of expert consensus verification on the initial rule pool to obtain the target rule set that has passed the consensus verification.

[0079] Step S103: Encode the target rule set into a logical assertion structure, and organize the logical assertion structure hierarchically to construct a clinical trial domain knowledge rule base.

[0080] Understandably, the aforementioned regulations in the field of clinical trials can refer to specific provisions of Good Clinical Practice (GCP), the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) guidelines, and relevant regulations issued by the National Medical Products Administration (NMPA). For example, GCP Article 17 regarding the timing of informed consent signing, or GCP Article 27 regarding the time limit for reporting serious adverse events.

[0081] The aforementioned expert experience rules may refer to operational constraints or deviation identification patterns summarized by experts in the field of clinical trial quality management during actual inspections or reviews, such as "the date of random number generation must be later than the date of screening number generation" or "the number of times a data field in the same electronic data acquisition system is modified should not exceed 3 times within 24 hours".

[0082] It should be explained that the aforementioned initial rule pool may refer to a set of candidate rules initially compiled from the aforementioned legal provisions and expert experience rules, and the rules in this set have not yet been systematically verified.

[0083] It should be noted that the aforementioned multi-round expert consensus verification can refer to inviting multiple domain experts to conduct multiple rounds of scoring and discussion on each rule in the initial rule pool through the Delphi method or other structured expert consultation methods, until the expert opinions reach a statistically acceptable level of consensus (e.g., Kendall W's concordance coefficient is greater than or equal to 0.70).

[0084] The aforementioned target rule set can refer to a set of rules that has been verified through multiple rounds of expert consensus and has been deemed to have clinical operability and priority for violation identification.

[0085] It should be explained that the above logical assertion structure can refer to converting the rules described in natural language into a standardized format with clear triggering conditions, judgment results, exceptions and legal basis, such as "IF condition THEN conclusion EXCEPT exception".

[0086] It should be noted that the aforementioned hierarchical organization can refer to classifying and storing rules in layers according to document type (such as informed consent forms, case report forms, and electronic data collection system logs) or review dimensions (such as security reports, data integrity, and procedural compliance).

[0087] In its implementation, the system needs to pre-build a knowledge rule base for the clinical trial domain before performing compliance reasoning operations. First, the system constructs an initial rule pool based on regulatory provisions and expert experience rules in the clinical trial field. The system extracts specific regulatory provisions from GCP, ICH, NMPA, and other regulatory documents through text crawling or manual input, transforming them into candidate rules. Simultaneously, the system searches databases such as PubMed and CNKI for research literature on clinical trial violation patterns, extracting the operational deviation patterns described within. The system then aggregates rules from these two sources to form the initial rule pool. For example, in one implementation, the initial rule pool covers eight core areas, including informed consent, safety reporting, data management, and randomization, containing a total of 210 candidate rules.

[0088] Next, the system conducts multiple rounds of expert consensus verification on the initial rule pool to obtain a target rule set that passes the consensus verification. The system invites 10 to 15 experts in clinical trial quality management (covering registrars, GCP inspectors, contract research organization quality directors, and clinical trial experts from academic institutions) to participate in the verification process. In the first round of verification, the system sends each rule in the initial rule pool to each expert in the form of a questionnaire, requiring experts to independently score (e.g., 1 to 5 points) the clinical operability and violation identification priority of each rule. After collecting the scores, the system calculates the Kendall W. concordance coefficient. For rules with a concordance coefficient below 0.5, the system marks them as rules requiring revision. In the second round of verification, the system provides the statistical results of the first round (including anonymous expert opinions) to the expert group and asks the experts to conduct a second scoring. For rules with a coefficient of variation greater than 0.25, the system organizes panel discussions for revision. In the third round of verification, the system asks experts to conduct a final scoring. The system uses a Kendall W. concordance coefficient greater than or equal to 0.70 as the consensus threshold, eliminating rules that do not reach this threshold. After three rounds of iteration, the system obtains the target rule set that has passed consensus verification. In one specific implementation, after the initial 210 rules were verified, 157 rules that passed consensus verification were ultimately retained.

[0089] The system then encodes the target rule set into a logical assertion structure. Each target rule described in natural language is converted into a standardized assertion format. This assertion format includes at least a rule identifier, domain, trigger condition, judgment result, severity score, applicable regulatory clause, and exception condition. For example, a rule might be encoded as: "Rule ID: R_012 | Domain: AE Report | Trigger Condition: IF AE Type = SAE AND (Sponsor Report Time - SAE Occurrence Time) > 7 calendar days | Judgment Result: THEN Marked as SAE Report Timeout Violation, Severity S = 5 | Applicable Regulation: GCP Article 27 | Exception Condition: EXCEPT AE Occurrence Location is a Major Natural Disaster Area."

[0090] Then, the system hierarchically organizes the logical assertion structure to construct a knowledge rule base for the clinical trial domain. The system divides the encoded rules into three levels according to the document type and review dimension to which the rules apply: the first level is the document type identification layer, used to determine the document type and activate the corresponding review dimension; the second level is the dimension-specific review layer, which organizes corresponding rule subsets according to review dimensions (such as regulatory compliance, data integrity, and temporal logic); the third level is the cross-document association layer, used for association reasoning rules in multi-document cross-validation scenarios. The system stores the organized rule base in a database or configuration file for subsequent compliance reasoning steps.

[0091] To facilitate understanding, the following examples illustrate the points, but do not impose specific limitations on this embodiment. For instance, the system extracts the regulatory clause from Article 17 of GCP: "Informed consent must be signed by the subject before any trial-related procedure is performed." Simultaneously, the system extracts an expert rule of experience from a literature review: "An informed consent form signed after the date of the first trial-related procedure constitutes a procedural violation." The system merges these two rules into the initial rule pool. Subsequently, the system invites 12 GCP inspection experts to conduct three rounds of Delphi validation. In the first round, the rule's average clinical operability score was 4.8, its average priority score was 5.0, and its Kendall W. Concordance coefficient was 0.82, directly passing consensus validation. The system encodes this rule as a logical assertion structure: "Rule ID: R_031 | Domain: Informed Consent | Trigger Condition: IF Informed Consent Signing Date >= Date of Subject's First Trial-Related Procedure | Judgment Result: THEN Marked as Informed Consent Procedural Violation, Severity S=5 | Applicable Regulation: GCP Article 17 | Exception Condition: EXCEPT In an emergency where prior informed consent could not be obtained and an ethics committee waiver has been obtained." The system places this rule into the "Informed Consent Procedure Compliance Review Hint Phrases" in the second-level dimension of the special review layer, completing the construction of the clinical trial domain knowledge rule base.

[0092] This embodiment discloses a method for constructing an initial rule pool based on regulations and expert experience rules in the clinical trial field; performing multiple rounds of expert consensus verification on the initial rule pool to obtain a target rule set that passes the consensus verification; encoding the target rule set into a logical assertion structure and hierarchically organizing the logical assertion structure to construct a knowledge rule base for the clinical trial field. Because this embodiment constructs an initial rule pool based on regulations and expert experience rules in the clinical trial field, then performs multiple rounds of expert consensus verification on the initial rule pool, and finally encodes the target rule set into a logical assertion structure and organizes it hierarchically, compared to existing technologies, this embodiment ensures the scientific validity and clinical operability of the knowledge rule base for the clinical trial field, thereby improving the accuracy and reliability of compliance reasoning.

[0093] refer to Figure 3 , Figure 3 This is a flowchart illustrating the third embodiment of the clinical trial document review method of the present invention.

[0094] Based on the above embodiments, in this embodiment, step S40 includes steps S401 to S404: Step S401: Quantify and score the deviation detection results based on multi-dimensional risk quantification rules to obtain score values ​​for each dimension. The multi-dimensional risk quantification rules include scoring criteria for severity, frequency of occurrence, and traceability dimensions.

[0095] Step S402: Determine the weight coefficients corresponding to the severity dimension, the frequency of occurrence dimension, and the traceability dimension according to the clinical trial study type.

[0096] Step S403: Based on the score values ​​of each dimension and the corresponding weight coefficients, determine the comprehensive risk score value of the risk event in the deviation detection result.

[0097] Step S404: Compare the comprehensive risk score with the preset risk level threshold, determine the risk level of the risk event based on the comparison result, and generate a structured audit report.

[0098] It should be explained that the aforementioned severity dimension can refer to an evaluation perspective used in risk assessment to measure the degree of impact of a bias event on the safety of test subjects or the validity of trial data. For example, the severity dimension is divided into 1 to 5 points according to the degree of impact, ranging from suggestive to extremely severe, with 5 points representing a direct threat to the safety of test subjects or an impact on the validity of trial conclusions.

[0099] The frequency of occurrence dimension mentioned above can refer to the evaluation angle used in risk assessment to measure the frequency of occurrence of the same type of deviation event in a document or experiment. For example, the frequency of occurrence dimension is divided into 1 to 5 points according to the number of occurrences, with 1 point for 1 occurrence and 5 points for 11 or more occurrences.

[0100] The traceability dimension mentioned above can refer to the evaluation perspective used in risk assessment to measure whether the operational records involved in deviation events are complete and traceable. For example, the traceability dimension is divided into 1 to 5 points according to the completeness of the records, ranging from fully traceable to completely untraceable, with 5 points representing no operational records or signatures.

[0101] Understandably, the aforementioned weighting coefficients can refer to numerical multipliers assigned to the three dimensions of severity, frequency of occurrence, and traceability based on the characteristics of different clinical trial study types, reflecting the relative importance of each dimension within that study type. For example, in interventional randomized controlled trials, the subject safety risk has the highest weight, and the weighting coefficient for the severity dimension can be set to 0.5.

[0102] Understandably, the aforementioned comprehensive risk score can refer to a single value obtained by multiplying the scores of each dimension by their corresponding weighting coefficients and then summing them, used to characterize the overall risk level of a risk event. The aforementioned preset risk level threshold can refer to pre-set numerical boundaries used to classify risk levels (high risk, medium risk, low risk), for example, a high risk threshold set at 7.0 and a medium risk threshold set at 4.0.

[0103] In its implementation, the system calculates the comprehensive risk score for each risk event in the deviation detection results based on the scores of each dimension and their corresponding weighting coefficients. The system calculates the comprehensive risk score according to a preset formula: Comprehensive Risk Score = Severity Dimension Score × Severity Weighting Coefficient + Occurrence Frequency Dimension Score × Occurrence Frequency Weighting Coefficient + (6 - Traceability Dimension Score) × Traceability Weighting Coefficient. The system calculates the comprehensive risk score for each risk event individually.

[0104] Understandably, the system compares the calculated comprehensive risk score with preset risk level thresholds. The system reads pre-stored high-risk thresholds (e.g., 7.0) and medium-risk thresholds (e.g., 4.0). If the comprehensive risk score is greater than or equal to the high-risk threshold, the system determines the risk level of the risk event to be high-risk; if the comprehensive risk score is greater than or equal to the medium-risk threshold but less than the high-risk threshold, the system determines the risk level of the risk event to be medium-risk; if the comprehensive risk score is less than the medium-risk threshold, the system determines the risk level of the risk event to be low-risk.

[0105] It should be noted that a structured audit report should include at least the following: basic audit information (such as audit time and document list), a risk event summary table (including a description of each risk event, a comprehensive risk score, and a risk level), risk statistics categorized by level (number of high-risk events, number of medium-risk events, and number of low-risk events), and handling recommendations for different risk levels. The system will present the generated structured audit report to the auditors in the form of a visual interface or a downloadable file.

[0106] To facilitate understanding, the following example is used for explanation, but it does not impose specific limitations on this embodiment. For example, suppose the deviation detection result includes a risk event: "SAE report timeout violation, delayed by 9 days". The system, based on multi-dimensional risk quantification rules, determines the severity dimension as 5 points (extremely severe), the occurrence frequency dimension as 1 point (first occurrence), and the traceability dimension as 1 point (complete record). The system identifies the current clinical trial study type as an interventional randomized controlled trial and reads the weighting coefficients: severity 0.5, occurrence frequency 0.3, traceability 0.2. The system calculates the comprehensive risk score: 5×0.5+1×0.3+(6-1)×0.2=2.5+0.3+1.0=3.8. The system compares 3.8 with the preset risk level thresholds (high risk 7.0, medium risk 4.0). Since 3.8 is less than 4.0, it is determined to be low risk. The system generates a structured audit report, in which the risk event summary table records: "Risk Event: SAE report timed out by 9 days; Overall Score: 3.8; Risk Level: Low Risk; Recommendation: Optimize reporting process." The report statistics also show 0 high-risk, 0 medium-risk, and 1 low-risk item.

[0107] To improve the reliability of risk assessment results and the security of decision-making, after the step of determining the comprehensive risk score of risk events in the deviation detection results based on the score values ​​of each dimension and the corresponding weight coefficients, the method may further include: obtaining the inference confidence of the large language model for the deviation detection results; when the inference confidence is less than a preset confidence threshold, downgrading the target dimension score value among the score values ​​of each dimension and triggering manual review marking; and weighting and fusing the comprehensive risk score based on the inference confidence to obtain a corrected comprehensive risk score.

[0108] It should be explained that the aforementioned inference confidence score refers to the quantitative value of the certainty that a large language model assigns to its output of compliance inference results or deviation detection results. This confidence score is usually a value between 0 and 1, with a higher value indicating that the model's judgment is more certain, and a lower value indicating that the model's judgment is more ambiguous or uncertain.

[0109] Understandably, the aforementioned pre-set confidence threshold can refer to a confidence boundary value pre-defined by the system to determine whether the output of the large language model is reliable. For example, the pre-set confidence threshold can be set to 0.7. When the inference confidence is lower than 0.7, the system considers the model's judgment reliability insufficient.

[0110] It should be noted that the target dimension score mentioned above can refer to one or more dimension scores that need to be adjusted among the three dimensions of severity, frequency of occurrence, and traceability. For example, in one specific implementation, the target dimension score can be the severity dimension score.

[0111] It should be explained that the aforementioned downgrade process can refer to the operation of adjusting the dimension score value downward by a preset step size (e.g., reducing it by 1 point) to reflect the situation where the risk rating needs to be reduced due to insufficient model confidence.

[0112] Understandably, the aforementioned manual review marker can refer to an identifier added by the system to risk events to prompt reviewers to manually confirm or correct the risk event. The aforementioned weighted fusion can refer to the operation of combining the original comprehensive risk score and the inference confidence level according to a certain weight ratio to generate a corrected comprehensive risk score.

[0113] In its implementation, after calculating the comprehensive risk score, the system further obtains the inference confidence level of the large language model for the deviation detection results. This inference confidence level is a value output synchronously by the large language model when performing compliance inference or deviation detection. The system parses the confidence value corresponding to each deviation detection result from the output of the large language model, i.e., the inference confidence level.

[0114] Next, the system compares the obtained inference confidence score with a preset confidence threshold. If the inference confidence score is greater than or equal to the preset confidence threshold, the system considers the output of the large language model reliable, performs no additional processing, and directly uses the comprehensive risk score calculated above. If the inference confidence score is less than the preset confidence threshold, the system determines that the reliability of the model output is insufficient, triggering the insufficient confidence processing procedure.

[0115] When the inference confidence level is lower than a preset confidence threshold, the system downgrades the target dimension score among the various dimension scores. In one specific implementation, the target dimension score is set as the severity dimension score. The system adjusts the originally determined severity dimension score according to a preset downgrade step size (e.g., reducing it by 1 point). If the original severity dimension score is 5 points, it is downgraded to 4 points; if the original severity dimension score is 4 points, it is downgraded to 3 points, and so on. The system also triggers a manual review flag for this risk event. This manual review flag is displayed in the subsequently generated structured audit report to inform the auditors that the confidence level of the risk event's determination is low and requires manual reconfirmation.

[0116] Subsequently, the system performs a weighted fusion of the previously calculated comprehensive risk score based on the inference confidence level to obtain a revised comprehensive risk score. The system calculates according to a preset weighted fusion formula. For example, in one specific implementation, the weighted fusion formula is: Revised Comprehensive Risk Score = Original Comprehensive Risk Score × (0.6 + 0.4 × Inference Confidence Level). For example, if the original comprehensive risk score is 3.8 and the inference confidence level is 0.6, then the revised comprehensive risk score = 3.8 × (0.6 + 0.4 × 0.6) = 3.8 × (0.6 + 0.24) = 3.8 × 0.84 = 3.192. The system uses the revised comprehensive risk score as the final score for this risk event, which is then used for subsequent risk level determination and report generation.

[0117] To facilitate understanding, the following example illustrates the concept, but does not impose specific limitations on this embodiment. For instance, suppose the system calculates a comprehensive risk score of 3.8 for a certain SAE report timeout deviation, while the inference confidence level output by the large language model is 0.6. The system has a preset confidence threshold of 0.7, and determines that 0.6 is less than 0.7, triggering insufficient confidence processing. The system downgrades the severity dimension score: the original severity score of 5 points is downgraded by 1 point to 4 points. The system also adds a manual review mark to this risk event. Next, the system calculates the corrected comprehensive risk score: 3.8 × (0.6 + 0.4 × 0.6) = 3.8 × 0.84 = 3.192. The system uses the corrected 3.192 as the final comprehensive risk score for this risk event and displays the risk event with a "manual review" mark in the structured audit report, reminding the auditor to manually confirm the low confidence judgment output by the model.

[0118] It should be understood that by introducing the confidence level of the large language model to correct the risk score, and automatically downgrading and triggering manual review when the confidence level is insufficient, the risk of misjudgment caused by model illusion is effectively suppressed. This achieves the collaborative review of automatic review by artificial intelligence and verification by human experts, thereby improving the reliability of risk assessment results and the security of decision-making.

[0119] This embodiment discloses a method for quantitatively scoring deviation detection results based on multi-dimensional risk quantification rules to obtain score values ​​for each dimension. The multi-dimensional risk quantification rules include scoring standards for severity, frequency of occurrence, and traceability dimensions. Weighting coefficients are determined for each dimension (severity, frequency of occurrence, and traceability) according to the clinical trial study type. Based on the score values ​​for each dimension and the corresponding weighting coefficients, a comprehensive risk score for the risk event in the deviation detection results is determined. The comprehensive risk score is compared with a preset risk level threshold, and the risk level of the risk event is determined based on the comparison result, generating a structured review report. Because this embodiment transforms deviation detection results into an objective comprehensive risk score value through multi-dimensional quantitative scoring and differentiated weighting coefficients, and automatically classifies risk levels based on risk level thresholds, it achieves a standardized mapping from qualitative description to quantitative rating. Compared to existing technologies, this embodiment enhances the objectivity, comparability, and regulatory usability of clinical trial document review.

[0120] Furthermore, embodiments of the present invention also propose a storage medium storing a clinical trial document review program, wherein the clinical trial document review program, when executed by a processor, implements the steps of the clinical trial document review method described above.

[0121] Reference Figure 4 , Figure 4This is a structural block diagram of the first embodiment of the clinical trial document review device of the present invention.

[0122] like Figure 4 As shown, the clinical trial document review device proposed in this embodiment of the invention includes: a document parsing module 501, a compliance analysis module 502, a document comparison module 503, and a report generation module 504.

[0123] The document parsing module 501 is used to obtain the clinical trial document to be reviewed, perform structured parsing on the clinical trial document, and extract structured field information based on the parsing results.

[0124] The compliance analysis module 502 is used to perform compliance reasoning on the structured field information based on the clinical trial domain knowledge rule base to obtain compliance analysis results.

[0125] The document comparison module 503 is used to perform cross-document association consistency analysis on the clinical trial documents based on the compliance analysis results, identify logical deviations between documents according to the consistency analysis results, and obtain deviation detection results.

[0126] The report generation module 504 is used to perform risk assessment on the deviation detection results based on multi-dimensional risk quantification rules, and generate a structured audit report based on the risk assessment results.

[0127] The compliance analysis module 502 is further configured to identify the document type of the clinical trial document based on the structured field information; activate the corresponding audit dimension rule subset from the clinical trial domain knowledge rule base based on the document type; perform conditional matching between the structured field information and the audit dimension rule subset to obtain a rule matching result; and input the rule matching result and the structured field information into a large language model for compliance reasoning to obtain a compliance analysis result.

[0128] The document comparison module 503 is further configured to establish a hierarchical relationship mapping model between the clinical trial documents based on the compliance analysis results; perform cross-document association consistency analysis on the clinical trial documents through a semantic vector comparison engine and a symbolic logic reasoning engine based on the hierarchical relationship mapping model to obtain consistency analysis results; and identify logical deviations between documents based on the consistency analysis results to obtain deviation detection results.

[0129] The document comparison module 503 is further configured to identify standard documents, regulatory documents, and execution record documents from the clinical trial documents based on the hierarchical relationship mapping model; invoke a semantic vector comparison engine to perform semantic vectorization comparison on the standard documents and the regulatory documents to generate semantic consistency analysis sub-results; invoke a symbolic logic reasoning engine to perform temporal logic and program logic comparison on the execution record documents and the standard documents to generate logical consistency analysis sub-results; and generate consistency analysis results based on the semantic consistency analysis sub-results and the logical consistency analysis sub-results.

[0130] This device embodiment discloses the following steps: acquiring clinical trial documents to be reviewed; performing structured parsing on the clinical trial documents; extracting structured field information based on the parsing results; performing compliance reasoning on the structured field information based on a clinical trial domain knowledge rule base to obtain compliance analysis results; performing cross-document association consistency analysis on the clinical trial documents based on the compliance analysis results; identifying logical deviations between documents based on the consistency analysis results to obtain deviation detection results; conducting risk assessment on the deviation detection results based on multi-dimensional risk quantification rules; and generating a structured review report based on the risk assessment results. Because this device embodiment uses structured parsing and a domain knowledge rule base for compliance reasoning, combined with cross-document association consistency analysis and multi-dimensional risk quantification assessment to generate a structured review report, compared to existing technologies, this device embodiment improves the efficiency and reliability of clinical trial document review.

[0131] Based on the first embodiment of the clinical trial document review device of the present invention, a second embodiment of the clinical trial document review device of the present invention is proposed.

[0132] In this embodiment, the compliance analysis module 502 is further configured to construct an initial rule pool based on regulatory provisions and expert experience rules in the clinical trial field; perform multiple rounds of expert consensus verification on the initial rule pool to obtain a target rule set that passes the consensus verification; encode the target rule set into a logical assertion structure, and organize the logical assertion structure hierarchically to construct a knowledge rule base for the clinical trial field.

[0133] Other embodiments or specific implementations of the clinical trial document review device of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0134] This application provides a clinical trial document review device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the clinical trial document review method in Embodiment 1 above.

[0135] The following is for reference. Figure 5 The diagram illustrates a structural schematic suitable for implementing the clinical trial document review device in the embodiments of this application. The clinical trial document review device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The clinical trial document review device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments in this application.

[0136] like Figure 5 As shown, the clinical trial document review device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the clinical trial document review device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the clinical trial document review equipment to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows clinical trial document review equipment with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0137] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0138] The clinical trial document review device provided in this application, employing the clinical trial document review method described in the above embodiments, can solve the technical problems of insufficient efficiency and reliability in the review of clinical trial documents in the prior art. Compared with the prior art, the beneficial effects of the clinical trial document review device provided in this application are the same as those of the clinical trial document review method provided in the above embodiments, and other technical features of this clinical trial document review device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0139] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0140] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0141] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0142] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0143] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0144] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for reviewing clinical trial documents, characterized in that, The method includes: Obtain the clinical trial documents to be reviewed, perform structured parsing on the clinical trial documents, and extract structured field information based on the parsing results; Based on a knowledge rule base in the clinical trial domain, compliance reasoning is performed on the structured field information to obtain compliance analysis results; Based on the compliance analysis results, cross-document association consistency analysis is performed on the clinical trial documents. Logical deviations between documents are identified based on the consistency analysis results to obtain deviation detection results. The deviation detection results are assessed for risk based on multi-dimensional risk quantification rules, and a structured audit report is generated based on the risk assessment results.

2. The clinical trial document review method as described in claim 1, characterized in that, The step of performing compliance reasoning on the structured field information based on a clinical trial domain knowledge rule base to obtain compliance analysis results includes: The document type of the clinical trial document is identified based on the structured field information; Based on the document type, activate the corresponding subset of review dimension rules from the clinical trial domain knowledge rule base; The structured field information is matched against the subset of audit dimension rules to obtain the rule matching result; The rule matching results and the structured field information are input into a large language model for compliance reasoning to obtain compliance analysis results.

3. The clinical trial document review method as described in claim 2, characterized in that, Before the step of performing compliance reasoning on the structured field information based on a clinical trial domain knowledge rule base to obtain compliance analysis results, the method further includes: An initial rule pool was constructed based on regulations and expert experience rules in the field of clinical trials. The initial rule pool is subjected to multiple rounds of expert consensus verification to obtain the target rule set that passes the consensus verification; The target rule set is encoded into a logical assertion structure, and the logical assertion structure is organized hierarchically to construct a knowledge rule base for the clinical trial domain.

4. The clinical trial document review method as described in claim 1, characterized in that, The step of performing cross-document association consistency analysis on the clinical trial documents based on the compliance analysis results, identifying logical deviations between documents based on the consistency analysis results, and obtaining deviation detection results includes: Based on the compliance analysis results, a hierarchical mapping model for the clinical trial documents is established. Based on the hierarchical relationship mapping model, cross-document association consistency analysis is performed on the clinical trial documents through a semantic vector comparison engine and a symbolic logic reasoning engine to obtain consistency analysis results. Based on the consistency analysis results, logical discrepancies between documents are identified, and discrepancy detection results are obtained.

5. The clinical trial document review method as described in claim 4, characterized in that, The step of performing cross-document association consistency analysis on the clinical trial documents based on the hierarchical relationship mapping model, using a semantic vector comparison engine and a symbolic logic reasoning engine, to obtain the consistency analysis results includes: Based on the hierarchical relationship mapping model, standard documents, regulatory documents, and execution record documents are identified from the clinical trial documents; The semantic vector comparison engine is invoked to perform semantic vectorization comparison on the specification document and the regulatory document, generating semantic consistency analysis sub-results. The symbolic logic reasoning engine is invoked to compare the execution record document and the specification document with temporal logic and program logic, and to generate a logic consistency analysis sub-result. Based on the semantic consistency analysis sub-result and the logical consistency analysis sub-result, a consistency analysis result is generated.

6. The clinical trial document review method as described in claim 1, characterized in that, The steps of conducting a risk assessment on the deviation detection results based on multi-dimensional risk quantification rules and generating a structured audit report based on the risk assessment results include: The deviation detection results are quantitatively scored based on multi-dimensional risk quantification rules to obtain score values ​​for each dimension. The multi-dimensional risk quantification rules include scoring criteria for severity, frequency of occurrence, and traceability. The weighting coefficients for the severity dimension, the frequency of occurrence dimension, and the traceability dimension are determined according to the type of clinical trial study. Based on the score values ​​of each dimension and the corresponding weight coefficients, the comprehensive risk score value of the risk event in the deviation detection result is determined; The comprehensive risk score is compared with a preset risk level threshold. Based on the comparison result, the risk level of the risk event is determined and a structured audit report is generated.

7. The clinical trial document review method as described in claim 6, characterized in that, After the step of determining the comprehensive risk score of the risk event in the deviation detection result based on the score values ​​of each dimension and the corresponding weight coefficients, the method further includes: Obtain the inference confidence of the large language model for the deviation detection results; When the inference confidence is less than a preset confidence threshold, the target dimension score among the score values ​​of each dimension is downgraded and a manual review mark is triggered. The comprehensive risk score is weighted and fused based on the inference confidence level to obtain the corrected comprehensive risk score.

8. A clinical trial document review device, characterized in that, The device includes: The document parsing module is used to obtain clinical trial documents to be reviewed, perform structured parsing on the clinical trial documents, and extract structured field information based on the parsing results; The compliance analysis module is used to perform compliance reasoning on the structured field information based on a knowledge rule base in the clinical trial field, and obtain compliance analysis results. The document comparison module is used to perform cross-document association consistency analysis on the clinical trial documents based on the compliance analysis results, identify logical deviations between documents according to the consistency analysis results, and obtain deviation detection results. The report generation module is used to conduct risk assessment on the deviation detection results based on multi-dimensional risk quantification rules, and generate a structured audit report based on the risk assessment results.

9. A clinical trial document review device, characterized in that, The device includes: a memory, a processor, and a clinical trial document review program stored on the memory and executable on the processor, the clinical trial document review program being configured to implement the steps of the clinical trial document review method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a clinical trial document review program, which, when executed by a processor, implements the steps of the clinical trial document review method as described in any one of claims 1 to 7.

Images