Automatic checking method and device for clinical trial protocol deviation and computer equipment

Abstract

The application discloses a clinical trial scheme deviation automatic checking method and device and computer equipment. The method comprises the following steps: acquiring PD records and PD classification grading data from a CTMS system and an EDC system to obtain two data of the same project of the CTMS system and the EDC system; analyzing the two data by using a large language model to obtain two analysis results; performing consistency checking on the two data according to the two analysis results, and comparing the differences to obtain a detection result; generating a report containing consistency checking and difference comparison according to the detection result; and outputting the report. By implementing the method of the application, the checking efficiency, accuracy and system compatibility can be significantly improved, the manual intervention and learning cost can be reduced, and the seamless integration with the existing work flow can be ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to a data processing method, in particular to a clinical trial protocol deviation automatic checking method and device and a computer equipment. BACKGROUND

[0002] In clinical trials, PD (Protocol Deviation) refers to situations where the predetermined clinical trial protocol is not followed during the trial implementation. For example, a subject misses a scheduled visit, a dose of medication is incorrect, or certain tests are not completed as required. These deviations can be caused by researchers, subjects, or other related parties, and can affect the integrity of the trial, the reliability of the data, or the safety of the subjects. Therefore, accurately recording and reporting protocol deviations is crucial to ensure compliance, data integrity, and the safety of subjects.

[0003] In clinical trial management, CTMS (Clinical Trial Management System) and EDC (Electronic Data Capture) are core tools, and the protocol deviation data recorded by the two systems differs in source and function. First, the clinical trial management system is a software used to manage the operation of clinical trials, focusing on the administrative and project management of the trial. Its main functions include trial planning and tracking, managing the timeline, milestones, and resource allocation of the trial; site management, tracking research site and researcher information; subject management, monitoring the progress of subject recruitment, screening, and visits; document management, storing trial-related documents such as informed consent forms and ethical approval documents; and financial and compliance, managing budgets and payments to ensure compliance with regulations. CTMS can provide a global perspective, helping research teams optimize operations and ensure that the trial progresses as planned. The electronic data capture system, on the other hand, is used to collect and manage clinical trial data, replacing traditional paper data collection. Its core functions include direct collection of subject clinical data such as medical history and laboratory results; built-in data verification rules to ensure data accuracy; real-time data monitoring to reduce data entry errors; and data security to ensure compliance and provide audit tracking functions. EDC aims to improve data quality and collection efficiency, ensuring data reliability and traceability.

[0004] The CTMS and EDC systems have different functional focuses, but there is an intersection between them. The CTMS focuses more on the management of trial operations (such as site information and subject status), while the EDC focuses on the collection and verification of clinical data. Due to the differences in their functions and data entry methods, PD records may have inconsistencies, omissions, or classification errors. These inconsistencies can lead to missing or incorrect data, affecting the reliability of trial results and the effectiveness of statistical analysis. In addition, inconsistent records can cause compliance issues, increasing regulatory risks. Therefore, it is crucial to check the differences in protocol deviation records in the CTMS and EDC systems.

[0005] Currently, the methods for checking protocol deviations in clinical trials can be broadly divided into manual checking and semi-automated checking. First, manual checking is a traditional method of checking. The method is to export data related to protocol deviations from the CTMS and EDC systems, respectively, and then manually compare the records by data management personnel to check for differences, omissions, or duplicates. Differences are sorted into reports and noted for problem types and possible causes. However, this method has the disadvantage of being labor-intensive, especially in large multi-center trials, and is inefficient and prone to human error, such as missed or misjudged cases. In addition, manual checking also makes it difficult to achieve real-time monitoring, which may affect the progress of the trial. Semi-automated checking uses technical tools to extract data from the CTMS and EDC systems and convert the data into a unified format. Preliminary screening is performed through simple rules to identify differences. This method is more efficient than manual review and can reduce repetitive work, but still requires some technical ability, and scripting and maintenance can add additional work. For descriptive text or complex protocol deviations, automated processing is more difficult, so it cannot completely replace manual review.

[0006] Therefore, it is necessary to design a new method that significantly improves checking efficiency, accuracy, and system compatibility, reduces manual intervention and learning costs, and ensures seamless integration with existing workflows. SUMMARY

[0007] The purpose of the present application is to overcome the shortcomings of the prior art and provide an automatic protocol deviation checking method, device, and computer equipment for clinical trials.

[0008] To achieve the above-mentioned purpose, the present application adopts the following technical solution: an automatic protocol deviation checking method for clinical trials, comprising:

[0009] Obtaining PD records and PD classification and grading data from the CTMS system and the EDC system to obtain two sets of data for the same project from the CTMS system and the EDC system;

[0010] Using a large language model to analyze the two sets of data to obtain two sets of analysis results;

[0011] According to the two analysis results, consistency checking is performed on the two data, and differences are compared to obtain a detection result;

[0012] According to the detection result, a report containing consistency checking and difference comparison is generated;

[0013] The report is output.

[0014] Further technical solutions thereof are as follows: the two data are parsed by using a large language model to obtain two analysis results, including:

[0015] The two data are classified by using a large language model to obtain bias record data and classification and grading data;

[0016] The bias record data and the classification and grading data are automatically mapped to different system fields and unstructured PD descriptions are parsed to extract key information;

[0017] The key information is converted into a unified internal data structure to obtain the two analysis results.

[0018] Further technical solutions thereof are as follows: the two data are classified by using a large language model to obtain bias record data and classification and grading data, including:

[0019] The two data are segmented into data blocks, and each data block is classified by using a zero-shot classification capability of a large language model, and a final category is determined by combining a voting mechanism to obtain bias record data and classification and grading data.

[0020] Further technical solutions thereof are as follows: the bias record data and the classification and grading data are automatically mapped to different system fields and unstructured PD descriptions are parsed to extract key information, including:

[0021] A prompt word is constructed;

[0022] The bias record data and the classification and grading data are input into a large language model in combination with the prompt word to parse unstructured PD descriptions to obtain JSON format analysis results;

[0023] The JSON format analysis results are deserialized into objects of a programming language to obtain key information.

[0024] Further technical solutions thereof are as follows: according to the two analysis results, consistency checking is performed on the two data, and differences are compared to obtain a detection result, including:

[0025] Internal consistency of the two analysis results is checked by using a large language model to obtain a consistency result;

[0026] Compare the two sets of analysis results by subject and time, mark missing records, and evaluate the consistency of the two sets of bias record data to determine missing markers and difference causes;

[0027] The detection result includes consistency result, missing marker and difference cause.

[0028] Further technical solutions thereof are that the internal consistency of the two sets of analysis results is checked by the large language model to obtain a consistency result, which includes:

[0029] The prompt word is constructed according to the two sets of analysis results.

[0030] The two sets of analysis results and the prompt word are input into the large language model for internal consistency analysis to obtain a consistency result.

[0031] The consistency result includes consistency result, confidence, inconsistency reason and suggestion.

[0032] Further technical solutions thereof are that the two sets of analysis results are compared by subject and time, missing records are marked, and the consistency of the two sets of bias record data is evaluated to determine missing markers and difference causes, which include:

[0033] The two sets of analysis results are grouped according to the subject screening number, and the records of the same subject are classified and processed according to the PD type to obtain multiple groups of data.

[0034] Each group of data is sorted according to the event occurrence time to obtain a sorting result.

[0035] The comparison prompt word is constructed.

[0036] The sorting result and the comparison prompt word are input into the large language model to obtain a comparison result, wherein the comparison result includes missing marker and difference cause.

[0037] Further technical solutions thereof are that a report containing consistency verification and difference comparison is generated according to the detection result, which includes:

[0038] The detection result is used to generate an Excel file containing consistency verification and difference comparison in a structured manner to obtain a report.

[0039] The application also provides a clinical trial scheme deviation automatic verification device, which includes:

[0040] The acquisition unit is used to acquire PD records and PD classification and grading data from the CTMS system and the EDC system to obtain two sets of data of the same project of the CTMS system and the EDC system.

[0041] The analysis unit is used for analyzing the two data by using a large language model to obtain two analysis results.

[0042] The PD checking unit is used for checking the consistency of the two data according to the two analysis results, and comparing the differences to obtain a detection result.

[0043] The report generation unit is used for generating a report containing consistency checking and difference comparison according to the detection result.

[0044] The output unit is used for outputting the report.

[0045] The application also provides a computer device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the method described above when executing the computer program.

[0046] The application has the following beneficial effects compared with the prior art: the application obtains PD records and classification and grading data from a CTMS system and an EDC system, and uses a large language model to analyze the data, so that the system can efficiently and automatically check the consistency of the data and compare the differences; this process significantly improves the efficiency and accuracy of the checking, ensures the compatibility between different systems, and reduces manual intervention and learning costs. By generating a report containing consistency checking and difference comparison, the system can seamlessly integrate the existing workflow, realize the automation and optimization of the process, and greatly improve the work efficiency and quality.

[0047] The application will be further described below in combination with the drawings and specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0049] Figure 1 The application provides an application scenario diagram of the clinical trial scheme deviation automatic checking method;

[0050] Figure 2 The application provides a flowchart of the clinical trial scheme deviation automatic checking method;

[0051] Figure 3 The application provides a sub-flowchart of the clinical trial scheme deviation automatic checking method;

[0052] Figure 4A sub-flow schematic diagram of an automatic checking method for a clinical trial scheme deviation provided by the embodiment of the present application;

[0053] Figure 5 A sub-flow schematic diagram of an automatic checking method for a clinical trial scheme deviation provided by the embodiment of the present application;

[0054] Figure 6 A sub-flow schematic diagram of an automatic checking method for a clinical trial scheme deviation provided by the embodiment of the present application;

[0055] Figure 7 A sub-flow schematic diagram of an automatic checking method for a clinical trial scheme deviation provided by the embodiment of the present application;

[0056] Figure 8 A schematic block diagram of an automatic checking device for a clinical trial scheme deviation provided by the embodiment of the present application;

[0057] Figure 9 A schematic block diagram of a computer device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0059] It should be understood that when used in the specification and the appended claims, the terms “comprise” and “include” indicate the presence of described features, integers, steps, operations, elements, and / or components, but do not exclude one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0060] It should also be understood that the terms used in the present application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, unless otherwise clearly indicated by the context, the singular forms “a”, “an” and “the” are intended to include the plural forms.

[0061] It should be further understood that the term “and / or” used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.

[0062] Please refer to Figure 1 and Figure 2 , Figure 1An application scenario diagram of the automatic checking method for deviation of a clinical trial scheme provided by an embodiment of the present application. Figure 2 An illustrative flowchart of the automatic checking method for deviation of a clinical trial scheme provided by an embodiment of the present application. The automatic checking method for deviation of a clinical trial scheme is applied in a server. The server interacts with a terminal to significantly improve checking efficiency, accuracy and system compatibility through application of a large language model. The method automatically analyzes data from a CTMS and an EDC system, performs consistency checking and difference comparison, thereby reducing manual intervention; uses a large language model to classify, automatically map fields and analyze unstructured data, effectively improving data processing accuracy and ensuring consistency between data; in addition, the method can automatically generate a report and export it in a structured format, simplifying the report generation process, reducing learning costs, ensuring seamless integration with existing workflows, and significantly improving the automation and intelligence level of checking.

[0063] Figure 2 is a flowchart of the automatic checking method for deviation of a clinical trial scheme provided by an embodiment of the present application. As shown in Figure 2 , the method includes the following steps S110 to S150.

[0064] S110, obtaining PD records and PD classification and grading data from a CTMS system and an EDC system to obtain two sets of data of the same project of the CTMS system and the EDC system.

[0065] In this embodiment, the two sets of data of the same project of the CTMS system and the EDC system are the PD records and the PD classification and grading data of the CTMS system and the PD records and the PD classification and grading data of the EDC system.

[0066] Obtain deviation-related data from a clinical trial management system and an electronic data collection system.

[0067] The CTMS is a system for managing and monitoring clinical trials. Here, the visit data of the subjects and the deviations in the trial process are recorded in the CTMS system.

[0068] The system extracts data from the CTMS system through an API interface or a file export method. The extracted data includes:

[0069] Structured fields: such as subject ID (subject_id), visit date (visit_date), deviation type (deviation_type), etc.

[0070] Unstructured data: for example, the description text of the PD (such as “the subject missed the 3rd visit due to traffic problems”).

[0071] EDC is a system used to record and manage data in clinical trials, commonly used to collect patient data and various records during the trial process.

[0072] Similar to the CTMS system, the EDC system also provides an API interface or supports file export functions to obtain PD-related data, the content and structure of which are usually different from the data format in the CTMS.

[0073] The data extracted from the EDC system also includes structured fields (such as subject ID, visit date) and unstructured description text (such as PD description text).

[0074] The types of data obtained include:

[0075] PD records: records about deviations or events of subjects during the clinical trial process.

[0076] PD classification and grading data: refers to the classification (such as "visit delay", "dosing error", etc.) and grading information (such as "minor", "major", etc.) of PD records.

[0077] Accessing data through the interface of CTMS and EDC usually requires login and authentication. Through the interface, the system can automatically pull the data of the project, including PD records and classification information.

[0078] If the API interface cannot be used, data can also be obtained by importing files in CSV, Excel, etc. format. In this case, the file format needs to conform to the specified structure to ensure that the data can be correctly parsed. There is no need to adapt to different CTMS and EDC systems.

[0079] This step is very critical in the entire system, as it is the basis for subsequent data parsing and verification. Any data acquisition and quality issues will directly affect the analysis and verification results.

[0080] The accuracy and completeness of data acquisition ensure the successful execution of subsequent steps (such as data parsing, verification, etc.).

[0081] Step S110 is the first link in the entire automated verification system, which aims to obtain two sets of PD record data through the API interface or file import method of the CTMS and EDC systems. These data provide a basis for the subsequent intelligent parsing and verification modules, ensuring the consistency and accuracy of the data.

[0082] In this embodiment, the PD recorded data is sensitive private data within the enterprise, and for data security considerations, these data are not allowed to be transmitted to the cloud. Therefore, data processing cannot be performed using large models deployed externally in the cloud, and all data processing must be completed on the enterprise's internal servers or workstations to reduce the risk of data leakage. Therefore, enterprises need to privately deploy large models in their internal environment.

[0083] Ollama is an open-source tool designed for efficient operation of large language models in local environments, helping enterprises achieve secure, efficient, and low-cost AI deployment. In conjunction with Nginx reverse proxy, Ollama can ensure that the interface of the large model will not be accessed by unauthorized access. The following is the specific process of deploying Ollama with Nginx reverse proxy:

[0084] First, the existing hardware environment needs to be evaluated to ensure that it can support the implementation of subsequent steps. This evaluation process includes checking whether the processor, memory, storage space, and network connection are sufficient, and if necessary, adjusting or upgrading the hardware to meet the deployment requirements.

[0085] After confirming that the hardware environment meets the requirements, the next step is to download and install the "Ollama" software. This process usually involves accessing the official website or designated download link to obtain the latest installation package and following the installation instructions to complete the software installation.

[0086] After installing Ollama, the next step is to download and run the "Qwen2.5 32B" large model. This usually means obtaining a pre-trained model file from a specific model library or platform, and then using Ollama or other related tools to load and start the model for subsequent application or experiment.

[0087] In order to optimize the network communication of the system and improve its security and stability, the next step is to install and configure Nginx as a reverse proxy server. This process includes downloading the Nginx installation package, executing the installation command, and modifying the Nginx configuration file according to actual needs to achieve load balancing, SSL encryption and other functions to ensure the efficient operation and stability of the system.

[0088] Finally, in order to further enhance the security of the system, it is necessary to configure the HTTP basic authentication mechanism for Nginx. This configuration can be achieved by editing the Nginx configuration file, adding corresponding authentication instructions and user credentials. After completing these settings, only clients who have passed identity verification can access protected resources, thereby improving the security protection capabilities of the system.

[0089] Through the above steps, enterprises can ensure efficient and secure deployment and operation of large language models in a local private environment, effectively avoid data leakage risks, and ensure control and security of system access.

[0090] S120, parsing the two data using a large language model to obtain two parsing results.

[0091] In this embodiment, the two parsing results refer to the key information corresponding to the PD record data and the PD classification and grading data obtained after classification and parsing.

[0092] In an embodiment, referring to Figure 3 The above step S120 can include steps S121-S123.

[0093] S121, classifying the two data using a large language model to obtain bias record data and classification and grading data.

[0094] In this embodiment, the two data are divided into data blocks and classified using the zero-shot classification capability of the large language model, and the final class is determined by combining the voting mechanism to obtain the bias record data and the classification and grading data.

[0095] The input two data (usually from different CTMS and EDC systems) are divided, and the zero-shot classification capability of the large language model is used to classify each data block. In this way, the model can identify the type of data block, such as PD record data and PD classification and grading data.

[0096] First, the two data are split into multiple data blocks (chunks). Each data block contains a number of rows of data, and the specific number is usually determined according to the input limit of the model (for example, 5 to 10 rows). Each data block is treated as an independent input item and processed by the large language model.

[0097] Next, the large language model is used to classify each data block. Due to the zero-shot learning capability of the large language model, the system can identify the class of the data block according to the prompt (Prompt) without explicit training. For example, the data block may be labeled as "PD record", "PD classification and grading", or "other".

[0098] To improve the accuracy of classification, a voting mechanism is used. Specifically, the classification results of multiple data blocks are aggregated through the reasoning capability of the large language model to determine the final class of each data block. Each data block returns a classification result and its confidence, and the final class is determined according to the aggregation result of the confidence.

[0099] Through these steps, the system can automatically identify and classify PD record data and classification hierarchical data, eliminating the need for manual classification or traditional hard-coded classification.

[0100] S122, automatically mapping different system fields and parsing unstructured PD descriptions to extract key information from the deviation record data and classification hierarchical data.

[0101] In an embodiment, referring to Figure 4 The above step S122 can include steps S1221-S1223.

[0102] S1221, constructing a prompt word;

[0103] S1222, inputting the deviation record data and classification hierarchical data combined with the prompt word into a large language model to parse unstructured PD descriptions and obtain a JSON format parsing result;

[0104] S1223, deserializing the JSON format parsing result into an object of a programming language to obtain key information.

[0105] In this embodiment, key information refers to key fields extracted from PD record data and PD classification hierarchical data, such as subject ID, deviation type, classification code, etc.

[0106] In this embodiment, data fields obtained from different CTMS and EDC systems are automatically mapped to a unified data structure, and unstructured PD descriptions are parsed to extract key information. This process aims to solve the differences in field names and data formats between different systems and ensure data consistency.

[0107] Since there may be differences in field names in CTMS and EDC systems (e.g., "subject_id" vs. "patient_id"), it is necessary to automatically map field names in different systems. The large language model can recognize these differences and automatically map them through its semantic understanding ability. For example, "patient_id" is mapped to "subject_id", ensuring that data from different systems can be uniformly understood.

[0108] In addition to structured fields, PD records may also contain unstructured descriptions (e.g., "the subject missed the 3rd visit due to traffic problems"). The large language model can extract key information such as deviation type and deviation reason from these texts using its semantic understanding ability. This step is crucial for handling complex and irregular text data.

[0109] Through these mapping and parsing processes, the system can ensure that data from different systems is standardized and structured, facilitating subsequent processing and analysis.

[0110] S123, transforming the key information into a unified internal data structure to obtain two parsing results.

[0111] In this embodiment, the classified and parsed key information is transformed into a unified internal data structure for subsequent data processing, verification and analysis.

[0112] In this embodiment, two main data structures are defined:

[0113] PD record data structure (PD_RECORD), including "subject_id", visit date, category, description text, level, etc.

[0114] PD classification and grading data structure (PD_TYPE_LEVEL), including main classification code, sub-classification code, classification name and grading, etc.

[0115] Through the prompt word design of the large language model, the extracted key information is transformed into a unified data structure. This process automatically selects the corresponding structure according to the type of input data (PD record or classification and grading data), and fills the parsed field values into the structure. The final structured data can be represented in JSON format, ensuring data standardization.

[0116] After completing this step, the system will output two structured data results, which facilitate subsequent PD verification modules for consistency checking and comparative analysis.

[0117] Step S120 uses the powerful semantic understanding and adaptive ability of the large language model to solve the problem of data format differences between CTMS and EDC systems, and automatically classifies, parses and structures the data from different systems. Specifically, step S121 classifies and classifies data through data blocking, S122 automatically maps fields and parses unstructured descriptions, and S123 transforms key information into a unified data structure, ensuring that the two data can be effectively parsed and unified. Through this process, the system realizes efficient, automated PD record verification and data integration, significantly improving data processing efficiency and accuracy.

[0118] Specifically, through the semantic understanding ability of the large language model (LLM), intelligent parsing and structured processing of patient data (PD) records and classification and grading data in the clinical trial management system (CTMS) and electronic data capture system (EDC) is realized. This module breaks through the limitations of traditional methods in handling heterogeneous data, significantly improving the system's automation level, adaptability and maintainability.

[0119] In clinical trial data management, CTMS and EDC systems typically store PD records and classification and grading data in different formats. While this data is relatively standardized (i.e., field structures are relatively standardized), subtle formatting differences still exist, such as differing field names (e.g., "subject_id" versus "patient_id"), inconsistent date formats (e.g., "YYYY-MM-DD" versus "DD / MM / YYYY"), and differences in classification and grading representations (e.g., "minor deviation" versus "minor deviation"). Traditional approaches typically rely on hard-coding to adapt to different CTMS and EDC systems individually. This requires developing specific parsing logic, mapping field names, and converting formats for each system, increasing code size and complexity. Furthermore, updates to the CTMS or EDC system (e.g., API changes or field adjustments) require code modifications and system redeployment, which is time-consuming and labor-intensive, and lacks scalability. Consequently, the addition of new systems necessitates the development of new adapters, hindering rapid adaptation to diverse data sources.

[0120] To address these issues, this embodiment utilizes the semantic understanding capability and adaptability of LLM to automatically parse and structure PD records and classify and grade data, fundamentally resolving the limitations of traditional methods.

[0121] Specifically, PD records (including structured fields and unstructured descriptions) and classification and hierarchical data are identified from the input data.

[0122] Leveraging the semantic understanding capabilities of LLM, it automatically adapts to heterogeneous data formats of different CTMS and EDC systems without the need for hard-coded adaptation.

[0123] Identify field semantics and automatically map field names from different systems (for example, mapping "patient_id" to "subject_id"), including automatic mapping of Chinese and English field names.

[0124] Parse unstructured PD descriptions to extract key information (e.g., deviation type, cause).

[0125] The LLM is guided by prompts to convert the parsed data into a unified internal data structure.

[0126] Automatic search locates PD classification and grading data and PD record data from the two provided data sets. Formally speaking, automatic search can be viewed as a text classification process: the input data table is divided into appropriately sized chunks, each chunk is classified using the zero-shot classification capabilities of a large language model, and the final category of the chunk is determined through a voting mechanism, thereby locating the required data. The following describes the process in detail from four aspects: data preprocessing, the classification process, the voting mechanism, and the output results.

[0127] Assume that the input data can be divided into K data tables, and assume that one of the data tables is D, which contains N rows of data:

[0128] D = {d1, d2, ..., dN};

[0129] In order to improve classification efficiency and model comprehension ability, continuous (k) rows of data are combined into a data block (chunk) to form a data block set (C).

[0130] C = {c1, c2, ..., cM}, ;

[0131] Where cj is the jth data block, containing k rows of data (the last data block may contain fewer than k rows). k is usually selected based on the data size and model input constraints (such as the maximum number of tokens), with a typical value of 5-10 rows.

[0132] Leveraging the zero-shot classification capabilities of the large language model, each data block cj is classified to determine whether it contains PD classification and grading data or PD record data. The classification label set L = {"PD record", "PD classification and grading", "other"} represents the three possible categories of the data block. Using a cue word design, each data block is input into the large language model for classification. The model returns the classification result (e.g., "PD record") and provides a confidence score P.

[0133] Due to the volatility of large-scale model results, individual classification results may be inaccurate. Therefore, a voting mechanism aggregates the classification results to determine the final category of the data block. For each data block set C in table D, the results (Lj, Pj) for each data block Cj in C are obtained. The confidence sum for each category L is calculated, and the category L with the highest confidence sum is selected as the category for the data table.

[0134] The PD classification and grading data structure is designed as follows:

[0135] json

[0136] {

[0137] "main_category_code":"string", / / main category code;

[0138] "main_category_name":"string", / / main category name

[0139] "sub_category_code":"string", / / subcategory code;

[0140] "sub_category_name":"string", / / Sub-category name

[0141] "level":"string" / / Classification level

[0142] }

[0143] The PD record data structure is designed as follows:

[0144] json

[0145] {

[0146] "subject_id":"string", / / Subject ID

[0147] "visit_date":"string", / / Visit date (format: YYYY-MM-DD)

[0148] "main_category_name":"string", / / Main category name

[0149] "sub_category_name":"string", / / Sub-category name

[0150] "description":"string", / / PD description text (e.g., "Subject missed the 3rd visit due to traffic issues")

[0151] "level":"string", / / Classification level

[0152] "source":"string" / / Data source ("CTMS" or "EDC")

[0153] }

[0154] After determining the category of the data table (e.g., PD record or classification level data) through automatic search and defining a unified data structure, the data parsing module utilizes the semantic understanding capabilities of large language models (LLMs) to automatically parse the content in the data table and map the required fields to the unified data structure. This process uses JSON as an intermediary to ensure the structured and consistent data during parsing and mapping. The specific process is as follows:

[0155] Construct prompt words and input them into the large language model. The prompt words are constructed in the following way:

[0156] The text of the data row is {text};

[0157] It comes from the {type} data table;

[0158] Try to parse the array of JSON objects into {struct} structures. If a row cannot be parsed, output a JSON object of the UNKOWN string

[0159] Where text is the text of the data row, type is the type of the data table (may be a PD classification level table or a PD record table), and struct is the structure determined according to type. If type is a PD classification level table, struct is a PD_TYPE_LEVEL structure; if type is a PD record table, struct is a PD_RECORD structure.

[0160] Deserialize the JSON string returned by the large language model into an object in the programming language, which completes the data parsing.

[0161] S130, according to the two parsing results, the consistency of the two data is checked, and the difference is compared to obtain the detection result.

[0162] In this embodiment, the detection result refers to the final output generated on the basis of consistency checking and data comparison, including the comparison result of PD records in the two systems, the marking of missing records, and the difference found in the comparison process. The detection result provides a basis for data cleaning and quality control, and helps to identify and correct inconsistencies or missing problems in the data.

[0163] The process of checking the consistency and comparing the differences of PD (adverse event) records in CTMS (clinical trial management system) and EDC (electronic data collection system) through a large language model (LLM). Specifically, the semantic understanding and reasoning ability of LLM are used to check the consistency of data, identify missing data, and analyze the differences between the two systems.

[0164] This step is responsible for checking the consistency and comparing the differences of the two data parsing results extracted from CTMS and EDC, and finally obtaining the detection result. The detection result includes the consistency evaluation of the data, the marking of missing records, and the cause analysis of the differences.

[0165] In an embodiment, see Figure 5 The above step S130 can include steps S131-S132.

[0166] S131, check the internal consistency of the two parsing results through a large language model to obtain a consistency result.

[0167] In this embodiment, the consistency result refers to the evaluation result obtained by judging whether the data description, category, time, and other elements in the two systems match after the PD records of the same subject and the same visit in the CTMS and EDC systems are analyzed by the large language model. If the data in the two systems are consistent at the semantic level, it is judged as consistent; otherwise, it is inconsistent, and the specific reason for inconsistency is given.

[0168] In an embodiment, referring to Figure 6 The step S131 described above can include steps S1311-S1312.

[0169] S1311, constructing a prompt word according to the two analysis results;

[0170] S1312, inputting the two analysis results and the prompt word into a large language model for internal consistency analysis to obtain a consistency result;

[0171] The consistency result includes consistency result, confidence, inconsistency reason, and suggestion.

[0172] In this embodiment, first, the internal consistency of the two analysis results is checked. Specifically, the following sub-steps are included:

[0173] Based on the data in the CTMS and EDC systems, a prompt word (Prompt) is constructed. The prompt word will include “description”, “main category name”, “sub-category name” and other information, the purpose is to provide further analysis to the LLM.

[0174] The constructed prompt word and the analysis results of CTMS and EDC are input into a large language model for consistency check. The large language model automatically evaluates whether the description and category are consistent according to its deep semantic understanding ability. For example, the model will check whether the “visit delay” category and “subject missed the third visit due to traffic problems” are consistent in semantic logic.

[0175] The results include the following aspects:

[0176] Consistency result (whether consistent);

[0177] Confidence (between 0 and 1, indicating the reliability of consistency);

[0178] Inconsistent reason (if there is inconsistency, the model will infer the reason and give suggestions).

[0179] S132, compare the two sets of analysis results by subject and time, mark missing records and evaluate the consistency of the two sets of deviation record data to determine missing markers and difference causes;

[0180] The detection result includes consistency result, missing marker and difference cause.

[0181] In an embodiment, referring to Figure 7 The above step S132 can include steps S1321-S1324.

[0182] S1321, group the two sets of analysis results according to the subject screening number, and classify the records of the same subject according to the PD type to obtain multiple sets of data;

[0183] S1322, sort each set of data according to the event occurrence time to obtain a sorted result;

[0184] S1323, construct a comparison prompt word;

[0185] S1324, input the sorted result and the comparison prompt word into a large language model to obtain a comparison result, wherein the comparison result includes a missing marker and a difference cause.

[0186] In this embodiment, the S132 step mainly focuses on comparing the PD records of the same subject and the same visit in the CTMS and EDC systems to identify data differences and missing. This process includes the following sub-steps:

[0187] Group the PD records in CTMS and EDC according to the screening number (such as unique ID) of the subject. Ensure that each group represents the data of the same subject.

[0188] In the data of the same subject, further classify according to the PD type. This is done to ensure that the data structure of the comparison is uniform, avoiding errors caused by different categories of records.

[0189] Sort each set of data by event occurrence time. By sorting by time sequence, the same event (such as a visit delay) in the two systems can be correctly compared.

[0190] Construct a comparison prompt word, which includes the list of PD records of the same subject and the same visit in the CTMS and EDC systems, and requires the LLM to compare and identify the differences between the records according to the description, time, etc. The comparison result is divided into three categories:

[0191] Both systems have records and are consistent;

[0192] CTMS has a record but EDC does not (marked as "missing");

[0193] EDC has a record but CTMS does not (marked as "missing").

[0194] The output of LLM will include cases where records are marked as missing, as well as an analysis of the reasons for the discrepancies (such as data entry errors or system synchronization issues).

[0195] In this embodiment, the final detection results contain the following key elements:

[0196] Consistency results: Through internal consistency checks, assess whether the two sets of data match semantically.

[0197] Missing markers: Compare PD records in both systems and mark which records are missing in one system.

[0198] Discrepancy reasons: Analyze and explain the reasons for data inconsistencies between the two systems, such as data errors, system synchronization issues, etc.

[0199] This embodiment takes full advantage of the powerful semantic analysis and reasoning capabilities of large language models to achieve comprehensive and automated verification of PD records in CTMS and EDC systems. Through the following technical measures:

[0200] Intelligent prompt word generation: Construct prompt words based on actual data and pass them to the large language model for verification;

[0201] Automated consistency checks and comparative verification: Without human intervention, it can efficiently check the consistency of massive data and analyze discrepancies;

[0202] Deep reasoning and correction suggestions: Provide reasonable analysis and suggestions for inconsistent records to support data cleaning.

[0203] This step not only improves verification efficiency and accuracy, but also reduces manual intervention, significantly improving the intelligence and compliance of data management.

[0204] In the method of this embodiment, each PD record in the CTMS and EDC systems is verified, and its internal consistency is checked to ensure that the PD category (e.g., "visit delay") and the PD description (e.g., "subject missed the 3rd visit due to traffic problems") are semantically consistent. With the zero-shot text classification capabilities of the large language model (LLM), inconsistent records are intelligently identified and marked, and correction suggestions are provided. This provides necessary support for subsequent data cleaning and quality control.

[0205] By comparing the PD records of the same subject and the same visit in the CTMS and EDC systems, differences between the two systems can be identified (e.g., missing records, inconsistent classification, inconsistent descriptions, etc.). With the deep reasoning capabilities of the LLM, the module can not only find differences but also infer their root causes (such as data entry errors or system synchronization problems) and assess the severity of the differences, thereby providing assistance for data consistency improvement.

[0206] The method of this embodiment takes advantage of the semantic analysis and reasoning capabilities of large language models, achieving a fully automated process from consistency checking to difference comparison, significantly improving verification efficiency and accuracy, and reducing the need for manual intervention. In addition, the module design is flexible and expandable, capable of adapting to clinical trials of different scales and diverse CTMS / EDC systems, providing intelligent data management and regulatory compliance solutions.

[0207] Specifically, for each PD record in CTMS and EDC, extract the main_category_name, sub_category_name, and description fields.

[0208] Construct the prompt word, refer to the following structure:

[0209] Description: {description};

[0210] Category: {main_category_name}, {sub_category_name};

[0211] Is the description and category consistent? Please return the consistency result (consistent / inconsistent), confidence, and reason and suggestion when inconsistent.

[0212] The LLM returns the result in text format, which needs to be parsed to extract the consistency result ("consistent" or "inconsistent"), confidence (0-1), inconsistent reason, and suggestion. To improve efficiency, batch prompt words can be constructed.

[0213] The PD record comparison and verification process between CTMS and EDC is as follows:

[0214] Group the PD records in CTMS and EDC according to the subject screening number.

[0215] For each subject screening number, group the PD records by PD type.

[0216] For the same type of PD record under the same subject screening number, sort them in chronological order.

[0217] Construct the prompt word, refer to the following structure:

[0218] The PD record list from the CTMS is: {CTMS_PD_RECORDS};

[0219] The PD record list from the EDC is: {EDC_PD_RECORDS};

[0220] Please compare the records from the PD record list from the CTMS and the PD record list from the EDC one by one, and infer whether they are the same PD according to the description and time. The comparison results are divided into three categories:

[0221] Both systems have records and are consistent.

[0222] CTMS has, EDC does not: marked as "missing".

[0223] EDC has, CTMS does not: marked as "missing".

[0224] The LLM returns results in text format, which needs to be parsed and the required data structure object is generated according to the parsing results.

[0225] The prompt word needs to be optimized according to the specific situation.

[0226] Through the above technical implementation steps, the data verification work can be efficiently and accurately completed, and the quality and compliance of clinical trial data management are further improved.

[0227] S140, generating a report containing consistency verification and difference comparison according to the detection results.

[0228] In this embodiment, the report refers to a structured Excel file generated based on the detection results, containing the consistency verification of the PD record and the difference comparison between the CTMS and the EDC system, aiming to provide detailed verification results and correction suggestions.

[0229] Specifically, the consistency verification and difference comparison are generated in a structured Excel file according to the detection results to obtain the report.

[0230] The result record module is the core output part of the PD record verification system, responsible for saving and displaying the verification results in the form of a structured Excel file, which is convenient for users to view, analyze and archive. The main functions of this module are as follows:

[0231] Record verification results:

[0232] CTMS system verification results: Record the internal consistency verification results of the PD record in the CTMS system, check whether the PD category and description match.

[0233] EDC System Verification Results: Record the internal consistency verification results of PD records in the EDC system, also check the consistency of PD categories and descriptions.

[0234] CTMS vs EDC System Verification Results: Record the comparison verification results of PD records in the CTMS and EDC systems, identify the differences between the two.

[0235] Generate Structured Reports:

[0236] Output the verification results in an Excel file, divided into three independent sheets (Sheets), corresponding to the three parts above.

[0237] Provide a clear field structure for users to quickly understand the verification results.

[0238] The Excel file generated by the Results Recording Module contains three sheets, each corresponding to the verification results of one part, with clear structure and reasonable field design. The detailed structure of each part is as follows:

[0239] Excel File Overall Structure

[0240] File Name: PD_Verification_Result_[Timestamp].xls;

[0241] Sheets:

[0242] CTMS_Consistency: CTMS PD record internal consistency verification results.

[0243] EDC_Consistency: EDC PD record internal consistency verification results.

[0244] CTMS_vs_EDC_Differences: CTMS and EDC PD record comparison verification results.

[0245] CTMS_Consistency Sheet Field Structure;

[0246] Record_ID: Unique identification of PD records (composed of subject ID and visit number).

[0247] Visit_Date: Visit date.

[0248] Deviation_Type: PD category.

[0249] Description: PD description text.

[0250] Consistency_Result: Consistency check result (labeled as "Consistent" or "Inconsistent").

[0251] Confidence_Score: Confidence score of consistency check (range 0-1, based on large language model zero-shot classification).

[0252] Inconsistency_Details: If there is inconsistency, provide detailed explanation (e.g., "Described as visit delay, but category labeled as dosing error").

[0253] Suggestion: Correction suggestion (e.g., "Suggest updating category to 'visit delay'").

[0254] EDC_Consistency worksheet:

[0255] The field structure of this worksheet is the same as CTMS_Consistency, but the data comes from the EDC system.

[0256] CTMS_vs_EDC_Differences worksheet:

[0257] This worksheet is grouped by subject ID to facilitate tracking of subject status.

[0258] Record_ID: Unique identification of the PD record.

[0259] Visit_Date: Visit date (dates from CTMS and EDC may not be consistent).

[0260] Deviation_Type_CTMS: PD category in CTMS.

[0261] Deviation_Type_EDC: PD category in EDC.

[0262] Description_CTMS: PD description in CTMS.

[0263] Description_EDC: PD description in EDC.

[0264] Difference_Type: Difference type (e.g., "Missing", "Value Mismatch", "Semantic Inconsistency").

[0265] Missing: One side has a record, while the other side has no record.

[0266] Value Mismatch: Inconsistency between structured fields (such as category).

[0267] Semantic Inconsistency: Differences in described semantics.

[0268] Similarity_Score: Similarity of described semantics (0-1, based on semantic vector comparison of large language models).

[0269] Severity: Severity of differences (e.g., "minor" or "major").

[0270] Details: Details of differences (e.g., "CTMS record exists, EDC is missing").

[0271] Suggestion: Correction suggestions (e.g., "suggest supplementing records in EDC").

[0272] S150, output the report.

[0273] In this embodiment, the report is output to a terminal.

[0274] The method of this embodiment obtains PD (product data) records by interfacing with CTMS (clinical trial management system) and EDC (electronic data collection) systems, or through file import. Using the semantic understanding capabilities of a large language model, the obtained PD records are intelligently parsed, automatically adapting to heterogeneous data formats and description methods between different systems, converting originally unstructured text data into structured data, and unifying formats to provide consistent input data for subsequent verification. Based on the parsed data, consistency verification of PD records is performed, and PD records in the CTMS and EDC systems are compared. Through precise matching of structured fields and semantic analysis of unstructured descriptions, inconsistent records (such as missing, classification errors, or semantic inconsistencies) are identified, and they are classified according to the type and priority of the differences, ensuring the accuracy and operability of the verification results. A detailed verification report is generated, clearly showing the difference details, semantic analysis results, and proposing appropriate correction suggestions.

[0275] By developing based on the Microsoft Office ecosystem, it can run in a familiar environment for users, reduce operational barriers, and achieve seamless integration with existing workflows, improving compatibility and convenience. With the semantic understanding ability of large language models, it can automatically adapt to different PD record formats in CTMS and EDC systems in the data parsing module, intelligently parse and unify various heterogeneous data formats and description methods, solve structural and expression differences between systems, support cross-platform data integration, and significantly improve verification efficiency and consistency. Relying on the semantic understanding and zero-shot text classification capabilities of large language models, the system can perform consistency checks on PD categories and descriptions within PD records, automatically find and correct classification mismatches (such as "visit delay" being incorrectly labeled as "dose error"), and adapt to different PD description scenarios without additional training data. Using the deep reasoning capabilities of large language models, the system can perform differential comparison and verification of PD records in CTMS and EDC systems, analyze structured fields and unstructured descriptions, identify subtle differences between records (such as semantic inconsistencies, classification errors, or data missing), and infer potential root causes, providing accurate verification results and improvement suggestions, thereby improving data consistency and automating the verification process.

[0276] Traditional methods (such as manual review or semi-automatic processing) often require a lot of manual intervention, which is time-consuming and inefficient. For example, in a large multi-center clinical trial involving thousands of PD records, manually comparing data in CTMS and EDC often takes several days, while the method of this embodiment shortens the required time to a few minutes, and the total time after manual confirmation can be shortened to within 1 hour.

[0277] Traditional methods often rely on manual judgment or simple rules when dealing with unstructured PD descriptions, making it difficult to accurately identify records with similar expressions but different semantics, leading to inaccurate verification results. The method of this embodiment can more accurately verify data through intelligent means.

[0278] The method of this embodiment fully utilizes the widespread application of the Microsoft Office environment and uses a user-familiar operation interface, thereby reducing the invasiveness of the system and ensuring seamless integration into existing workflows. This design greatly reduces the learning cost of users and improves the compatibility and convenience of system deployment.

[0279] The above clinical trial scheme deviation automatic checking method deviates from the automatic checking method, obtains PD records and classification grading data from the CTMS system and the EDC system, and parses the data using a large language model. The system can efficiently and automatically process the consistency checking and difference comparison of the data. This process significantly improves the efficiency and accuracy of the checking, ensures the compatibility between different systems, and reduces manual intervention and learning costs. By generating a report containing consistency checking and difference comparison, the system can seamlessly integrate the existing workflow, realize the automation and optimization of the process, and greatly improve the work efficiency and quality.

[0280] Figure 9 is a schematic block diagram of a clinical trial scheme deviation automatic checking device 300 provided by an embodiment of the present application. As shown in Figure 9 corresponding to the above clinical trial scheme deviation automatic checking method, the present application also provides a clinical trial scheme deviation automatic checking device 300. The clinical trial scheme deviation automatic checking device 300 includes units for executing the above clinical trial scheme deviation automatic checking method. The device can be configured in a server. Specifically, please refer to Figure 9 , the clinical trial scheme deviation automatic checking device 300 includes an acquisition unit 301, an analysis unit 302, a PD checking unit 303, a report generation unit 304, and an output unit 305.

[0281] The acquisition unit 301 is configured to acquire PD records and PD classification grading data from the CTMS system and the EDC system to obtain two sets of data of the same project from the CTMS system and the EDC system. The analysis unit 302 is configured to analyze the two sets of data using a large language model to obtain two sets of analysis results. The PD checking unit 303 is configured to perform consistency checking on the two sets of data according to the two sets of analysis results, and compare the differences to obtain detection results. The report generation unit 304 is configured to generate a report containing consistency checking and difference comparison according to the detection results. The output unit 305 is configured to output the report.

[0282] In an embodiment, the analysis unit 302 includes:

[0283] The classification sub-unit is configured to classify the two sets of data using a large language model to obtain deviation record data and classification grading data. The information extraction sub-unit is configured to automatically map different system fields and analyze unstructured PD descriptions of the deviation record data and the classification grading data to extract key information. The conversion sub-unit is configured to convert the key information into a unified internal data structure to obtain two sets of analysis results.

[0284] In an embodiment, the classification subunit is configured to split the two pieces of data into data blocks, classify each data block by using a zero-shot classification capability of a large language model, and determine a final category by combining a voting mechanism to obtain bias record data and classification grading data.

[0285] In an embodiment, the information extraction subunit includes:

[0286] A first construction module is configured to construct a prompt word; an analysis module is configured to input the bias record data and the classification grading data in combination with the prompt word into a large language model to analyze unstructured PD descriptions to obtain an analysis result in JSON format; and a reverse serialization module is configured to reverse serialize the analysis result in JSON format into an object in a programming language to obtain key information.

[0287] In an embodiment, the PD checking unit 303 includes:

[0288] An internal consistency checking subunit is configured to check internal consistency of the two pieces of analysis result by using a large language model to obtain a consistency result; and a comparison subunit is configured to compare the two pieces of analysis result by subject and time, mark missing records, and evaluate consistency of the two pieces of bias record data to determine a missing mark and a difference reason; wherein the detection result includes the consistency result, the missing mark, and the difference reason.

[0289] In an embodiment, the internal consistency checking subunit includes:

[0290] A second construction module is configured to construct a prompt word according to the two pieces of analysis result; and an analysis module is configured to input the two pieces of analysis result and the prompt word into a large language model for internal consistency analysis to obtain a consistency result.

[0291] The consistency result includes a consistency result, a confidence level, an inconsistency reason, and a suggestion.

[0292] In an embodiment, the comparison subunit includes:

[0293] A grouping module is configured to group the two pieces of analysis result by subject identification number, and classify records under the same subject by PD type to obtain a plurality of groups of data; an ordering module is configured to order each group of data according to event occurrence time to obtain an ordered result; a third construction module is configured to construct a comparison prompt word; and a comparison module is configured to input the ordered result and the comparison prompt word into a large language model to obtain a comparison result, wherein the comparison result includes a missing mark and a difference reason.

[0294] In an embodiment, the report generation unit 304 is configured to generate, according to the detection result, an Excel file containing the consistency check and the difference comparison in a structured manner to obtain a report.

[0295] It should be noted that the specific implementation process of the clinical trial scheme deviation automatic checking device 300 and the units can be clearly understood by those skilled in the art, and can refer to the corresponding description in the foregoing method embodiments. For the convenience and brevity of description, it will not be repeated here.

[0296] The clinical trial scheme deviation automatic checking device 300 can be implemented in the form of a computer program, which can run on a computer device as shown in the computer device. Figure 9

[0297] Please refer to Figure 9 , Figure 9 is a schematic block diagram of a computer device provided by an embodiment of the present application. The computer device 500 can be a server, wherein the server can be a stand-alone server or a server cluster composed of multiple servers.

[0298] Refer to Figure 9 , the computer device 500 includes a processor 502, a memory, and a network interface 505 connected through a system bus 501, wherein the memory can include a non-volatile storage medium 503 and an internal memory 504.

[0299] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, which when executed, can cause the processor 502 to perform a clinical trial scheme deviation automatic checking method.

[0300] The processor 502 is configured to provide computing and control capabilities to support the operation of the entire computer device 500.

[0301] The internal memory 504 provides an environment for the running of the computer program 5032 in the non-volatile storage medium 503, which when executed by the processor 502, can cause the processor 502 to perform a clinical trial scheme deviation automatic checking method.

[0302] The network interface 505 is configured to perform network communication with other devices. Those skilled in the art can understand that Figure 9 the structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device 500 to which the scheme of the present application is applied. The specific computer device 500 can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. ​

[0303] The processor 502 is configured to run the computer program 5032 stored in the memory to perform the following steps:

[0304] Obtaining PD records and PD classification and grading data from a CTMS system and an EDC system to obtain two sets of data of the same project from the CTMS system and the EDC system; parsing the two sets of data using a large language model to obtain two sets of parsing results; performing consistency checking on the two sets of data according to the two sets of parsing results, and comparing differences to obtain a detection result; generating a report containing consistency checking and difference comparison according to the detection result; and outputting the report.

[0305] In an embodiment, when implementing the step of parsing the two sets of data using a large language model to obtain two sets of parsing results, the processor 502 specifically implements the following steps:

[0306] classifying the two sets of data using a large language model to obtain deviation record data and classification and grading data; automatically mapping different system fields and parsing unstructured PD descriptions of the deviation record data and the classification and grading data to extract key information; and converting the key information into a unified internal data structure to obtain the two sets of parsing results.

[0307] In an embodiment, when implementing the step of classifying the two sets of data using a large language model to obtain deviation record data and classification and grading data, the processor 502 specifically implements the following steps:

[0308] segmenting the two sets of data into data blocks and classifying each of the data blocks using a zero-shot classification capability of a large language model, and determining a final category in combination with a voting mechanism to obtain the deviation record data and the classification and grading data.

[0309] In an embodiment, when implementing the step of automatically mapping different system fields and parsing unstructured PD descriptions of the deviation record data and the classification and grading data to extract key information, the processor 502 specifically implements the following steps:

[0310] constructing a prompt word; inputting the deviation record data and the classification and grading data in combination with the prompt word into a large language model to parse unstructured PD descriptions and obtain a JSON format parsing result; and de-serializing the JSON format parsing result into an object of a programming language to obtain key information.

[0311] In an embodiment, when implementing the step of performing consistency checking on the two sets of data according to the two sets of parsing results, and comparing differences to obtain a detection result, the processor 502 specifically implements the following steps:

[0312] The two sets of analysis results are checked for internal consistency by a large language model to obtain a consistency result; the two sets of analysis results are compared by subject and time, missing records are marked, and the consistency of two sets of bias record data is evaluated to determine missing markers and difference causes; wherein the detection result includes the consistency result, the missing marker, and the difference cause.

[0313] In an embodiment, the processor 502, when implementing the step of checking the internal consistency of the two sets of analysis results by a large language model to obtain a consistency result, specifically implements the following steps:

[0314] Construct a prompt word according to the two sets of analysis results; input the two sets of analysis results and the prompt word into a large language model for internal consistency analysis to obtain a consistency result;

[0315] The consistency result includes a consistency result, a confidence level, an inconsistency reason, and a suggestion.

[0316] In an embodiment, the processor 502, when implementing the step of comparing the two sets of analysis results by subject and time, marking missing records, and evaluating the consistency of two sets of bias record data to determine missing markers and difference causes, specifically implements the following steps:

[0317] The two sets of analysis results are grouped according to the subject screening number, and the records of the same subject are classified by PD type to obtain multiple sets of data; each set of data is sorted according to the event occurrence time to obtain a sorting result; a comparison prompt word is constructed; the sorting result and the comparison prompt word are input into a large language model to obtain a comparison result, wherein the comparison result includes a missing marker and a difference cause.

[0318] In an embodiment, the processor 502, when implementing the step of generating a report containing consistency verification and difference comparison according to the detection result, specifically implements the following steps:

[0319] An Excel file containing consistency verification and difference comparison is generated according to the detection result to obtain a report.

[0320] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0321] Those skilled in the art will appreciate that all or part of the steps in the method of the above-described embodiment can be implemented by instructing the relevant hardware through a computer program. The computer program includes program instructions, which can be stored in a storage medium that is computer-readable. The program instructions are executed by at least one processor in the computer system to implement the steps in the method of the above-described embodiment.

[0322] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein when the computer program is executed by a processor, the processor performs the following steps:

[0323] Acquire PD records and PD classification and grading data from the CTMS system and the EDC system to obtain two sets of data for the same project in the CTMS system and the EDC system; use a large language model to parse the two sets of data to obtain two parsing results; perform a consistency check on the two sets of data based on the two parsing results, and compare the differences to obtain a test result; generate a report including consistency verification and difference comparison based on the test result; and output the report.

[0324] In one embodiment, when the processor executes the computer program to implement the step of parsing the two pieces of data using a large language model to obtain two parsing results, the processor specifically implements the following steps:

[0325] Use a large language model to classify the two data to obtain deviation record data and classification and grading data; automatically map different system fields to the deviation record data and classification and grading data and parse the unstructured PD description to extract key information; convert the key information into a unified internal data structure to obtain two parsing results.

[0326] In an embodiment, the processor, when executing the computer program to implement the step of classifying the two pieces of data using a large language model to obtain bias record data and classification grading data, specifically implements the following steps:

[0327] segmenting the two pieces of data into data blocks and classifying each of the data blocks using the zero-shot classification capability of a large language model, and determining a final category in combination with a voting mechanism to obtain bias record data and classification grading data.

[0328] In an embodiment, the processor, when executing the computer program to implement the step of automatically mapping different system fields and parsing unstructured PD descriptions to extract key information from the bias record data and classification grading data, specifically implements the following steps:

[0329] constructing a prompt word; inputting the bias record data and classification grading data in combination with the prompt word into a large language model to parse unstructured PD descriptions to obtain a JSON format parsing result; and de-serializing the JSON format parsing result into an object of a programming language to obtain key information.

[0330] In an embodiment, the processor, when executing the computer program to implement the step of performing consistency checking on the two pieces of data according to the two pieces of parsing results and comparing differences to obtain a detection result, specifically implements the following steps:

[0331] checking the internal consistency of the two pieces of parsing results by a large language model to obtain a consistency result; comparing the two pieces of parsing results by subject and time, marking missing records and evaluating the consistency of the two pieces of bias record data to determine missing marks and difference reasons;

[0332] The detection result includes the consistency result, the missing marks and the difference reasons.

[0333] In an embodiment, the processor, when executing the computer program to implement the step of checking the internal consistency of the two pieces of parsing results by a large language model to obtain a consistency result, specifically implements the following steps:

[0334] constructing a prompt word according to the two pieces of parsing results; inputting the two pieces of parsing results and the prompt word into a large language model for internal consistency analysis to obtain a consistency result; and the consistency result includes a consistency result, a confidence, an inconsistency reason and a suggestion.

[0335] In an embodiment, when the processor implements the comparing of the two analysis results by subject and time, marking missing records and evaluating consistency of two bias record data to determine missing markers and difference causes step by executing the computer program, the following steps are implemented:

[0336] The two analysis results are grouped according to subject screening numbers, and records of the same subject are classified by PD types to obtain multiple groups of data; each group of data is sorted according to event occurrence time to obtain a sorted result; a comparison prompt word is constructed; and the sorted result and the comparison prompt word are input into a large language model to obtain a comparison result, wherein the comparison result includes missing marker information and difference causes.

[0337] In an embodiment, when the processor implements the generating of a report including consistency checking and difference comparison according to the detection result step by executing the computer program, the following steps are implemented:

[0338] The report including consistency checking and difference comparison is generated in a structured Excel file according to the detection result to obtain a report.

[0339] The storage medium can be a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various computer readable storage media that can store program codes.

[0340] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0341] In several embodiments provided by the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed.

[0342] The steps in the method of the embodiments of the present application can be adjusted in sequence, combined and deleted according to actual needs. The units in the device of the embodiments of the present application can be combined, divided and deleted according to actual needs. In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit.

[0343] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art, or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a terminal or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0344] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for automatically checking deviations from a clinical trial protocol, characterized in that: include: Obtain PD records and PD classification and grading data from the CTMS system and EDC system to obtain two sets of data for the same project in the CTMS system and EDC system; Parsing the two pieces of data using a large language model to obtain two parsing results; Performing a consistency check on the two data sets based on the two analysis results and comparing the differences to obtain a test result; Generate a report including consistency verification and difference comparison based on the test results; outputting the report; The using of the large language model to parse the two pieces of data to obtain two parsing results includes: Classifying the two pieces of data using a large language model to obtain deviation record data and classification grade data; Automatically map different system fields to the deviation record data and classification and grading data and parse unstructured PD descriptions to extract key information; Convert the key information into a unified internal data structure to obtain two parsing results; The use of the large language model to classify the two pieces of data to obtain deviation record data and classification and grading data includes: Splitting the two data into data blocks and classifying each data block using the zero-shot classification capability of the large language model, and determining the final category in combination with a voting mechanism to obtain deviation record data and classification and grading data; The deviation record data and classification and grading data are automatically mapped to different system fields and unstructured PD descriptions are parsed to extract key information, including: Constructing prompt words; Inputting the deviation record data and classification and grading data into a large language model in combination with the prompt words to parse the unstructured PD description and obtain a parsing result in JSON format; The parsed result in the JSON format is deserialized into an object in a programming language to obtain key information.

2. The method for automatically checking deviations from a clinical trial protocol according to claim 1, characterized in that: The consistency check of the two data sets according to the two analysis results and the comparison of differences to obtain the test results include: Checking the internal consistency of the two parsing results using a large language model to obtain a consistency result; Compare the two parsed results by subject and time, mark missing records, and evaluate the consistency of the two discrepant records to determine the missing marks and the reasons for the discrepancies; The test results include consistency results, missing markers and reasons for differences.

3. The method for automatically checking deviations from clinical trial protocols according to claim 2, characterized in that: The internal consistency of the two parsing results is checked by the large language model to obtain a consistency result, including: Constructing prompt words according to the two parsing results; Inputting the two parsing results and the prompt word into the large language model for internal consistency analysis to obtain a consistency result; The consistency result includes the consistency result, confidence level, inconsistency reason and suggestion.

4. The method for automatically checking deviations from a clinical trial protocol according to claim 3, wherein: The two sets of analytical results are compared by subject and time, missing records are marked, and the consistency of the two deviation records is evaluated to determine the missing marks and the reasons for the differences, including: The two analysis results are grouped according to the subject screening number, and the records under the same subject are classified according to the PD type to obtain multiple groups of data; Sort each set of data according to the time of event occurrence to obtain a sorting result; Construct contrast cue words; The sorting result and the comparison prompt word are input into a large language model to obtain a comparison result, wherein the comparison result includes the missing mark situation and the reason for the difference.

5. The method for automatically checking deviations from clinical trial protocols according to claim 1, characterized in that: The report containing consistency verification and difference comparison is generated based on the test results, including: Generate a structured Excel file containing consistency check and difference comparison based on the test results to obtain a report.

6. An automatic verification device for clinical trial protocol deviation, characterized in that: The device uses the automatic verification method for clinical trial protocol deviation according to any one of claims 1 to 5, comprising: An acquisition unit is used to acquire PD records and PD classification and grading data from the CTMS system and the EDC system, so as to obtain two sets of data for the same item in the CTMS system and the EDC system; a parsing unit, configured to parse the two pieces of data using a large language model to obtain two parsing results; A PD checking unit, configured to perform a consistency check on the two pieces of data according to the two analysis results, and compare the differences to obtain a test result; A report generation unit, configured to generate a report including consistency verification and difference comparison based on the test results; An output unit is used to output the report.

7. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 5 when executing the computer program.

Images