System for Identifying Anticipated Issues in Clinical Trial Operations and Generating Response Plans Using Artificial Intelligence

Abstract

The artificial intelligence-based clinical trial operation expected issue derivation and response plan generation system according to the present invention comprises: one or more processors; a memory storing commands and an artificial intelligence model executed by said processors; a database of completed response cases; a database of clinical trial operation data; a database of response grounds; and a database of rules. and response package database; wherein the completed response case database stores completed response cases, which are completed clinical trial operational issue response cases, wherein the completed response case includes at least one of the following: an initial operational issue that occurred in a specific clinical trial; a subsequent operational issue identified or processed after the initial operational issue; a phrase indicating the occurrence time, processing time, processing sequence, stage of occurrence, cause of occurrence, conditional relationship of the initial operational issue or the subsequent operational issue; a phrase indicating the causal relationship; an affected subject; an affected subject identifier; clinical trial drug information; clinical trial drug identifier; lot identifier; subject identifier; institution identifier; administration status; subject safety impact information; verification data; response requirements; response procedures; action results; reporting subjects; reporting procedure information; documentation items; preventive measures; follow-up observation items; source data location information; source data type; and source data version information, wherein the reporting procedure information includes at least one of the reporting subjects; reporting deadline; reporting form; reporting path; and the clinical trial drug information includes clinical trial drug identification information, candidate substance name, modality code, regulatory classification code, It includes at least one of a mechanism of action (MOA) code, target molecule identifier, direction of action code, site of action code, route of administration code, exposure characteristic code, storage conditions, preparation conditions, administration conditions, and lot information, and the clinical trial operational data database stores new clinical trial operational data including at least one of a clinical trial protocol, investigational medicinal product management manual, investigational medicinal product management information, visit schedule, administration schedule, specimen management information, institutional role information, and documentation standards for the new clinical trial.The above response basis database stores at least one of clinical trial-related regulations, guidelines, Q&A materials, standard operating procedures, internal review records, clinical trial protocol provisions, investigational medicinal product management manual provisions, investigator's manuals, approved product labels, and safety data; the above rule database stores at least one of issue classification rules, issue chain extraction rules, response procedure extraction rules, response package application rules, applicability score calculation rules, risk classification rules, weight sets, mapping rules, and top issue group definition information; the top issue group defined by the above top issue group definition information corresponds to any one of clinical trial subject medicinal product management, medication management, visit management, specimen management, reporting management, documentation management, and clinical trial protocol deviation management; and the above processor extracts the initial operational issue, subsequent operational issue, occurrence conditions of the said subsequent operational issue, verification materials, materials requiring verification, response requirements, response procedures, action results, reporting procedure information, source material location information, source material type, and source material version information from the said completed response case using the above artificial intelligence model, and converts the said initial operational issue and the said subsequent operational issue into a standard response case profile, wherein the above The standard response case profile includes at least one of an issue type code, a parent issue group code, an occurrence stage code, an occurrence cause code, an affected target code, an investigational drug code, an investigational drug standardization code, a modality code, a regulatory classification code, a mechanism of action (MOA) code, a target molecule identifier, a dosing status code, a subject safety impact code, a reporting target code, a reporting procedure code, a documentation code, source data location information, source data type, and source data version information, and includes the time of occurrence, time of processing, processing sequence, phrases indicating conditional relationships, phrases indicating causal relationships, results of actions, an affected target identifier, an investigational drug identifier, a lot identifier, a subject identifier, and an institution identifier included in the completed response case.Based on at least one of documentation items and reporting procedure information, the initial operational issue and the subsequent operational issue are classified into respective operational issue units, and at least one of the following relationships is determined between the operational issue units to generate an issue chain relationship: a chronological relationship, a causal relationship, a conditional relationship, a relationship involving the same affected subject, a relationship involving the same clinical trial drug, a relationship involving the same lot, a relationship involving the same subject, a relationship involving the same institution, a relationship involving the same documentation item, or a relationship involving the same reporting procedure; wherein the issue chain relationship includes a subsequent operational issue to be identified or processed following the initial operational issue, the occurrence conditions of the subsequent operational issue, data required for verification, conditions required for response, and a processing order; and a sequential response procedure sequence is generated from the issue chain relationship and the response procedure, including at least one of securing verification data, initial action, reporting, documentation, measures to prevent recurrence, and follow-up observation; and based on the issue chain relationship and the sequential response procedure sequence, for each issue type, an issue type code, a parent issue group code, occurrence conditions, subsequent operational issue, data required for verification, conditions required for response, a sequential response procedure sequence, information on the clinical trial drug, standardization code of the clinical trial drug, reporting subject, and reporting procedure information, Generate a plurality of response packages including at least one of documentation items, preventive measures, follow-up items, source data location information, source data type, and source data version information, store the plurality of response packages in the response package database, and generate a new operational profile from the new clinical trial operational data, wherein the new operational profile comprises an issue type code, a parent issue group code, a clinical trial drug management element, a dosing element, a visit element, a specimen management element, a reporting element, a documentation element, clinical trial drug information, a clinical trial drug standardization code, a modality code, a regulatory classification code, a mechanism of action (MOA) code, a target molecule identifier, and a dosing status code.It includes at least one of a subject safety impact code and a standard operating code comparable to the above standard response case profile, and A response package applicability score is calculated based on at least one of the issue type match, occurrence stage match, affected target match, administration status match, operational environment match, similarity of the clinical trial drug, and evidence reliability between each of the above new operational profile and the plurality of response packages stored in the response package database; if the response package applicability score is greater than or equal to a preset applicability threshold value, location information of the supporting data included in the response package exists, and version information of the supporting data included in the response package corresponds to the current version applied to the new clinical trial operational data, one or more response packages applicable to the new clinical trial operational data are selected from the plurality of response packages stored in the response package database; an issue type corresponding to the selected response package is derived as an expected operational issue that may occur in the new clinical trial; a corrected response package is generated by correcting items among the selected response packages that differ from the new clinical trial operational data; and a response plan for the new clinical trial operational data is generated based on the selected response package or the corrected response package, wherein the response plan includes data requiring verification, initial measures, reporting targets, It includes at least one of reporting procedure information, documentation items, preventive measures, follow-up items, items requiring modification, and items requiring expert review, and if the applicability score of the response package is greater than or equal to the applicability threshold value, but there exists a protocol difference, a difference in the investigational medicinal product management manual, a change in regulations, a difference in the version of supporting data, a difference in dosing status, a difference in subject safety impact, or a difference in the investigational medicinal product between the new operational profile and the selected response package, the item corresponding to the difference among the items included in the response plan is output as an item requiring modification, and if the applicability score of the response package is less than the applicability threshold value, or if the location information of the supporting data included in the response package does not exist,If the version information of the supporting data included in the above-mentioned response package does not correspond to the current version applied to the new clinical trial operational data, or if a correspondence between the supporting sentence extracted from the above-mentioned supporting data location information and the items included in the above-mentioned response package or the above-mentioned response plan is not confirmed, the above-mentioned response package may not be selected as an applicable response package, and the above-mentioned response package or at least some of the items included in the above-mentioned response package may be output as items requiring expert review.

Description

Technology Field

[0001] The present invention relates to an AI-based system for deriving anticipated operational issues and generating response measures for clinical trials. More specifically, it relates to an AI-based clinical trial operation support system that generates anticipated operational issues and response measures for new clinical trials using issue chain relationships and response packages based on completed response cases. Background Technology

[0002] Clinical trials are conducted in accordance with clinical trial protocols, investigational medicinal product management manuals, standard operating procedures, clinical trial-related regulations, guidelines, and institution-specific operational standards. During the execution of clinical trials, various operational issues may arise, such as deviations from storage conditions for the investigational medicinal product, medication errors, omissions or delays in visits, specimen handling errors, reporting delays, omissions in documentation, and deviations from the clinical trial protocol.

[0003] These operational issues often do not resolve as a single incident. For example, in the event of a temperature deviation of the drug under investigation, sequential follow-up actions may be required, such as verifying temperature records, calculating the time of deviation, verifying lot information, confirming whether the deviation occurred before or after administration, reviewing stability data, assessing usability, determining the need for subject safety monitoring, and reporting and documentation. Similarly, in the case of a missed visit, subsequent actions may be required, including verifying the allowable visit period, determining the possibility of a revisit, identifying missing evaluation items, reviewing the impact on key endpoints, determining whether the clinical trial protocol has been deviated from, and reporting and documentation.

[0004] Traditionally, responses to such clinical trial operational issues often relied primarily on the experience and manual reviews of clinical trial operators, monitors, principal investigators, sponsors, or quality control personnel. These personnel individually reviewed the clinical trial protocol, investigational medicinal product management manual, standard operating procedures, relevant regulations, guidelines, internal review records, and past similar cases to determine corrective actions, reporting targets, documentation items, and follow-up observation items for the relevant operational issue.

[0005] However, clinical trial operational data encompasses a wide variety of document types, with differences in version, clauses, items, tables, schedules, and role information across documents. Furthermore, completed responses to clinical trial operational issues can exist in various formats, including emails, Q&A records, meeting minutes, issue reports, monitoring logs, dropout reports, and internal review records. Consequently, there is a problem in that it is difficult to reuse past completed response records in a consistent manner to determine exactly what initial operational issue occurred, what subsequent issues were identified or handled, and what response procedures and supporting documents were used.

[0006] Furthermore, regarding a single operational issue, the data to be subsequently verified, the subjects to be reported, the items to be documented, and the items requiring follow-up observation may vary depending on the characteristics of the investigational drug, administration status, lot scope, subject safety impact, roles of each institution, visit schedules, specimen management conditions, and the versions of applicable evidence. Therefore, it is difficult to generate response strategies suitable for new clinical trial operational data by relying solely on methods such as searching for similar past cases or providing general response phrases.

[0007] Meanwhile, clinical trial management systems, electronic data collection systems, electronic document management systems, pharmacovigilance systems, or document search systems are used as electronic systems to support clinical trial operations. These systems can support clinical trial schedule management, data collection, document storage, adverse event reporting, or data retrieval. However, these systems have limitations in generating issue chain relationships by structuring chronological relationships, causal relationships, conditional relationships, same affected subjects, same clinical trial drugs, same lot relationships, same subjects, same institution relationships, same documentation items, or same reporting procedures between the initial operational issue and subsequent operational issues included in completed response cases.

[0008] Furthermore, while conventional document search or Q&A methods can present relevant documents or sentences, they have limitations in extracting verification data, response requirements, sequential response procedures, reporting procedure information, documentation items, preventive measures, and follow-up observation items from completed response cases, and structuring them into response packages applicable to new clinical trial operational data.

[0009] Furthermore, to apply past response cases to a new clinical trial, the applicability between the new clinical trial operational data and the past response cases or response packages must be assessed. This applicability needs to be determined by comprehensively considering not only simple keyword matching but also the issue type, superordinate issue group, stage of occurrence, affected subjects, administration status, operational environment, modality of the investigational drug, regulatory classification, mechanism of action (MOA), target molecule, route of administration, exposure characteristics, and the reliability of the evidence.

[0010] Furthermore, response measures for clinical trial operational issues must be traceable based on the location and version information of the supporting materials. For example, it must be possible to verify which source—such as the clinical trial protocol, investigational medicinal product management manual, standard operating procedures, guidelines, or internal review records—is the basis for specific reporting subjects, reporting deadlines, documentation items, or follow-up observation items. However, under conventional methods, there is a possibility that response measures based on past versions of documents or cases with unclear evidence may be reused without modification; in such cases, responses that are not suitable for the current version of the new clinical trial may be implemented.

[0011] Furthermore, depending on the characteristics of the drug under investigation, operational risks requiring caution may be related not only to past responses to clinical trial operational issues but also to safety events, toxicity events, reasons for discontinuation, or grounds for clinical hold identified in similar development cases. However, conventional clinical trial operational support methods lacked sufficient functionality to analyze development cases similar to the drug's modality, mechanism of action (MOA), target molecule, route of administration, and exposure characteristics, and to convert safety or toxicity-related information identified in those cases into operational response items.

[0012] Therefore, a technology is required that can extract initial and subsequent operational issues from completed response cases, determine the relationships between operational issue units to generate issue chain relationships, generate sequential response procedure sequences and response packages from the aforementioned issue chain relationships and response procedures, and determine the applicability of response packages to new clinical trial operational data to generate anticipated operational issues and response plans.

[0013] In addition, technology is required to link the location, type, and version information of supporting materials for each item of the response plan, and to classify items requiring expert review if the evidence is insufficient, does not correspond to the current version, or if the correspondence between the supporting sentence and the output item cannot be verified.

[0014] In addition, technology is required to maintain the up-to-dateness and traceability of knowledge regarding responses to clinical trial operational issues by updating existing response packages and saving package change history when new completed response cases or revised / changed response basis data are entered.

[0015] Furthermore, technology is required to classify safety events, toxicity events, reasons for development discontinuation, or clinical hold identified in similar development cases as operational risks, and to reflect them in response plans or correction response packages for new clinical trials. Prior art literature

[0016] Republic of Korea Published Patent 10-2013-0089901 The problem to be solved

[0017] The problem that the present invention aims to solve is to provide a system capable of generating an issue chain relationship between an initial operational issue and a subsequent operational issue from completed response cases, and structuring this into a response package applicable to new clinical trial operational data.

[0018] Furthermore, the problem that the present invention aims to solve is to provide a system capable of calculating the applicability between new clinical trial operational data and corresponding packages, and distinguishing items requiring modification or expert review based on source data location information, source data type, and source data version information.

[0019] Furthermore, the problem that the present invention aims to solve is to provide a system capable of updating or correcting response packages and response measures by reflecting new completed response cases, revised / changed response basis data, and operational risks of similar development cases. means of solving the problem

[0020] The artificial intelligence-based clinical trial operation expected issue derivation and response plan generation system according to the present invention comprises: one or more processors; a memory storing commands and an artificial intelligence model executed by said processors; a database of completed response cases; a database of clinical trial operation data; a database of response grounds; and a database of rules. and response package database; wherein the completed response case database stores completed response cases, which are completed clinical trial operational issue response cases, wherein the completed response case includes at least one of the following: an initial operational issue that occurred in a specific clinical trial; a subsequent operational issue identified or processed after the initial operational issue; a phrase indicating the occurrence time, processing time, processing sequence, stage of occurrence, cause of occurrence, conditional relationship of the initial operational issue or the subsequent operational issue; a phrase indicating the causal relationship; an affected subject; an affected subject identifier; clinical trial drug information; clinical trial drug identifier; lot identifier; subject identifier; institution identifier; administration status; subject safety impact information; verification data; response requirements; response procedures; action results; reporting subjects; reporting procedure information; documentation items; preventive measures; follow-up observation items; source data location information; source data type; and source data version information, wherein the reporting procedure information includes at least one of the reporting subjects; reporting deadline; reporting form; reporting path; and the clinical trial drug information includes clinical trial drug identification information, candidate substance name, modality code, regulatory classification code, It includes at least one of a mechanism of action (MOA) code, target molecule identifier, direction of action code, site of action code, route of administration code, exposure characteristic code, storage conditions, preparation conditions, administration conditions, and lot information, and the clinical trial operational data database stores new clinical trial operational data including at least one of a clinical trial protocol, investigational medicinal product management manual, investigational medicinal product management information, visit schedule, administration schedule, specimen management information, institutional role information, and documentation standards for the new clinical trial.The above response basis database stores at least one of clinical trial-related regulations, guidelines, Q&A materials, standard operating procedures, internal review records, clinical trial protocol provisions, investigational medicinal product management manual provisions, investigator's manuals, approved product labels, and safety data; the above rule database stores at least one of issue classification rules, issue chain extraction rules, response procedure extraction rules, response package application rules, applicability score calculation rules, risk classification rules, weight sets, mapping rules, and top issue group definition information; the top issue group defined by the above top issue group definition information corresponds to any one of clinical trial subject medicinal product management, medication management, visit management, specimen management, reporting management, documentation management, and clinical trial protocol deviation management; and the above processor extracts the initial operational issue, subsequent operational issue, occurrence conditions of the said subsequent operational issue, verification materials, materials requiring verification, response requirements, response procedures, action results, reporting procedure information, source material location information, source material type, and source material version information from the said completed response case using the above artificial intelligence model, and converts the said initial operational issue and the said subsequent operational issue into a standard response case profile, wherein the above The standard response case profile includes at least one of an issue type code, a parent issue group code, an occurrence stage code, an occurrence cause code, an affected target code, an investigational drug code, an investigational drug standardization code, a modality code, a regulatory classification code, a mechanism of action (MOA) code, a target molecule identifier, a dosing status code, a subject safety impact code, a reporting target code, a reporting procedure code, a documentation code, source data location information, source data type, and source data version information, and includes the time of occurrence, time of processing, processing sequence, phrases indicating conditional relationships, phrases indicating causal relationships, results of actions, an affected target identifier, an investigational drug identifier, a lot identifier, a subject identifier, and an institution identifier included in the completed response case.Based on at least one of documentation items and reporting procedure information, the initial operational issue and the subsequent operational issue are classified into respective operational issue units, and at least one of the following relationships is determined between the operational issue units to generate an issue chain relationship: a chronological relationship, a causal relationship, a conditional relationship, a relationship involving the same affected subject, a relationship involving the same clinical trial drug, a relationship involving the same lot, a relationship involving the same subject, a relationship involving the same institution, a relationship involving the same documentation item, or a relationship involving the same reporting procedure; wherein the issue chain relationship includes a subsequent operational issue to be identified or processed following the initial operational issue, the occurrence conditions of the subsequent operational issue, data required for verification, conditions required for response, and a processing order; and a sequential response procedure sequence is generated from the issue chain relationship and the response procedure, including at least one of securing verification data, initial action, reporting, documentation, measures to prevent recurrence, and follow-up observation; and based on the issue chain relationship and the sequential response procedure sequence, for each issue type, an issue type code, a parent issue group code, occurrence conditions, subsequent operational issue, data required for verification, conditions required for response, a sequential response procedure sequence, information on the clinical trial drug, standardization code of the clinical trial drug, reporting subject, and reporting procedure information, Generate a plurality of response packages including at least one of documentation items, preventive measures, follow-up items, source data location information, source data type, and source data version information, store the plurality of response packages in the response package database, and generate a new operational profile from the new clinical trial operational data, wherein the new operational profile comprises an issue type code, a parent issue group code, a clinical trial drug management element, a dosing element, a visit element, a specimen management element, a reporting element, a documentation element, clinical trial drug information, a clinical trial drug standardization code, a modality code, a regulatory classification code, a mechanism of action (MOA) code, a target molecule identifier, and a dosing status code.It includes at least one of a subject safety impact code and a standard operating code comparable to the above standard response case profile, and

[0021] A response package applicability score is calculated based on at least one of the issue type match, occurrence stage match, affected target match, administration status match, operational environment match, similarity of the clinical trial drug, and evidence reliability between each of the above new operational profile and the plurality of response packages stored in the response package database; if the response package applicability score is greater than or equal to a preset applicability threshold value, location information of the supporting data included in the response package exists, and version information of the supporting data included in the response package corresponds to the current version applied to the new clinical trial operational data, one or more response packages applicable to the new clinical trial operational data are selected from the plurality of response packages stored in the response package database; an issue type corresponding to the selected response package is derived as an expected operational issue that may occur in the new clinical trial; a corrected response package is generated by correcting items among the selected response packages that differ from the new clinical trial operational data; and a response plan for the new clinical trial operational data is generated based on the selected response package or the corrected response package, wherein the response plan includes data requiring verification, initial measures, reporting targets, It includes at least one of reporting procedure information, documentation items, preventive measures, follow-up items, items requiring modification, and items requiring expert review, and if the applicability score of the response package is greater than or equal to the applicability threshold value, but there exists a protocol difference, a difference in the investigational medicinal product management manual, a change in regulations, a difference in the version of supporting data, a difference in dosing status, a difference in subject safety impact, or a difference in the investigational medicinal product between the new operational profile and the selected response package, the item corresponding to the difference among the items included in the response plan is output as an item requiring modification, and if the applicability score of the response package is less than the applicability threshold value, or if the location information of the supporting data included in the response package does not exist,If the version information of the supporting data included in the above-mentioned response package does not correspond to the current version applied to the new clinical trial operational data, or if a correspondence between the supporting sentence extracted from the above-mentioned supporting data location information and the items included in the above-mentioned response package or the above-mentioned response plan is not confirmed, the above-mentioned response package may not be selected as an applicable response package, and the above-mentioned response package or at least some of the items included in the above-mentioned response package may be output as items requiring expert review.

[0022] Additionally, the processor calculates an issue type gateway coefficient based on whether there is an issue type match between the new operation profile and each of the plurality of corresponding packages stored in the corresponding package database, wherein the issue type gateway coefficient is calculated as 1 when the issue type code of the new operation profile and the issue type code of the corresponding package are identical, and as 0 when the issue type code of the new operation profile and the issue type code of the corresponding package are different but belong to the same upper issue group.It is calculated as 5, and if the issue type code of the new operational profile and the issue type code of the response package are different and do not belong to the same upper issue group, it is calculated as 0; a problem situation match score is calculated based on at least one of the match in occurrence stage, match in affected subjects, match in administration status, and match in subject safety impact between each of the multiple response packages stored in the new operational profile and the response package database; an operational environment adjustment score is calculated based on whether there is a match in at least one of the clinical trial site role, clinical trial drug management conditions, lot scope, visit schedule, specimen management conditions, reporting deadline, and documentation standards between each of the multiple response packages stored in the response package database and the new operational profile; and a clinical trial drug similarity score is calculated based on at least one of the match in modality code, regulatory classification code, mechanism of action (MOA) code, target molecule match score, direction of action match score, route of administration match score, and exposure characteristic match score between the clinical trial drug information included in the new operational profile and the clinical trial drug information included in the response package, and the A reliability score is calculated based on at least one of the existence of location information of supporting data included in the response package, the type of supporting data, whether the version information of supporting data corresponds to the current version, and whether there is a correspondence between the supporting sentence extracted from the location information of supporting data and the item included in the response package. The applicability score of the response package is calculated by multiplying the issue type gateway coefficient by a weighted sum to which weights corresponding to the problem situation match score, the operating environment correction score, the clinical trial target drug similarity score, and the reliability score of supporting data are respectively applied. If the issue type gateway coefficient is 0, the response package may not be selected as an applicable response package and may be output as an item requiring expert review.

[0023] When a new completed response case or revised / modified response basis data is input, the processor identifies a response package subject to update among a plurality of response packages stored in the response package database that is related to the new completed response case or the revised / modified response basis data, wherein the revised / modified response basis data includes at least one of revised clinical trial regulations, revised guidelines, modified Q&A data, modified standard operating procedures, modified clinical trial protocols, or modified investigational medicinal product management manuals, and compares the new completed response case with at least one of the issue type, top-level issue group, occurrence condition, subsequent operational issue, verification data, data requiring verification, response requirements, sequential response procedure sequence, reporting procedure information, and action result included in the response package subject to update, compares the revised / modified response basis data with at least one of the source data location information and source data version information included in the response package subject to update, and based on the comparison result, the occurrence condition, subsequent operational issue, data requiring verification, response requirements, reporting target, reporting procedure information, and documentation included in the response package subject to update Determining whether at least one of the items, sequential response procedure sequence, recurrence prevention measures, and follow-up observation items needs to be changed,

[0024] If there are items requiring change, the above-mentioned update target response package is updated to create an update response package, and for the above-mentioned update response package, a package change history including at least one of a pre-update version, a post-update version, identification information of revised / changed response basis data, changed items, reason for change, and update date and time can be stored in the above-mentioned response package database.

[0025] also,

[0026] The system further includes a development case database storing multiple development cases, wherein the development case database stores a development case profile for each development case, the profile including at least one of a modality code, regulatory classification code, mechanism of action (MOA) code, target molecule identifier, signaling pathway identifier, direction of action code, site of action code, route of administration code, exposure characteristics code, development phase information, safety event information, toxicity event information, reason for discontinuation of development, reason for clinical hold information, and source data location information; the processor generates a candidate substance profile based on the clinical trial subject drug information included in the novel clinical trial operational data, converts the candidate substance profile into the same standard tag system as the development case profile to generate a standardized profile including a modality code, regulatory classification code, mechanism of action (MOA) code, target molecule identifier, signaling pathway identifier, direction of action code, site of action code, route of administration code, and exposure characteristics code, and based on whether there is a semantic correspondence between the standardized profile and the development case profile, modality gateway coefficient, mechanism concordance score, application environment correction score, and Calculate an evidence availability adjustment score, and calculate a case similarity score by multiplying the weighted sum of the modality gateway coefficient, the mechanism consistency score, the application environment adjustment score, and the evidence availability adjustment score, respectively, and rank multiple development cases based on the case similarity score, and select the top N development cases among the ranked development cases as the group of development cases to be analyzed, wherein N is a pre-set positive integer, and extract at least one of a safety event, a toxicity event, a reason for discontinuation of development, and a reason for clinical hold from the group of development cases to be analyzed, wherein the case similarity score is greater than or equal to a first risk classification threshold value, which is set to a value greater than the similarity threshold value, and the safety event or toxicity event is associated with at least one of death, life-threatening, hospitalization, clinical hold, discontinuation of development, discontinuation of administration, or dose reduction, andIf the location information of the supporting data for the above safety event or toxicity event corresponds to at least one of regulatory agency documents, approved product labels, investigator's manuals, clinical trial outcome documents, safety data, or discontinuation records, the above safety event or toxicity event is classified as a foreseeable operational risk; if the above case similarity score is greater than or equal to the above similarity threshold value and the above case similarity score is less than the above first risk classification threshold value, or if the location information of the supporting data for the above safety event or toxicity event corresponds to clinical trial registration information, discontinuation press releases, company announcements, completed response cases, internal review records, or secondary summary data of unknown source, or if such data does not exist, the above safety event or toxicity event is classified as an operational risk requiring further verification; if the above case similarity score is less than the above similarity threshold value, the above safety event or toxicity event is classified as an operational risk with low applicability; and when generating the above response plan or generating the above correction response package, the above foreseeable operational risk is mapped to at least one of subject safety observation items, discontinuation criteria, dose adjustment criteria, reporting targets, or investigator's manual reflection review items, and when generating the above response plan or the above correction When generating a response package, the aforementioned operational risk requiring additional verification is mapped to at least one of additional verification data, additional safety review, additional nonclinical study review, items requiring expert review, or items requiring supplementary review for clinical trial approval; and if the aforementioned reason for discontinuation of development or reason for clinical hold is related to safety or toxicity, when generating the aforementioned response plan or the aforementioned corrective response package, the aforementioned reason for discontinuation of development or reason for clinical hold is mapped to at least one of items requiring review for regulatory agency reporting, items requiring supplementary review for clinical trial approval, or items requiring review of the risk summary in the investigator's manual; and for each item included in the aforementioned response plan, source data location information is output together, and if the aforementioned source data location information does not exist,If a correspondence between the supporting sentence extracted from the location information of the above-mentioned supporting data and the items included in the above-mentioned countermeasure is not confirmed, or if the severity, stage of occurrence, or relevance to development discontinuation of events confirmed from multiple location information of supporting data is inconsistent with one another, the relevant item may be output as an item requiring expert review. Effects of the invention

[0027] According to the present invention, by structuring operational issues and response procedures included in completed response cases into issue chain relationships and response packages, it is possible to systematically generate anticipated operational issues that may occur in new clinical trials and response measures thereto.

[0028] In addition, according to the present invention, by utilizing the applicability score of the response package and the location, type, and version information of the supporting data, it is possible to select response measures applicable to new clinical trials and distinguish items requiring modification or expert review.

[0029] In addition, according to the present invention, by reflecting new completed response cases, revised / modified response basis data, and operational risks derived from similar development cases, the recency, traceability, and suitability of response packages and response measures to the characteristics of the drug subject to clinical trials can be improved. Brief explanation of the drawing

[0030] FIG. 1 is a block diagram showing the overall configuration of an artificial intelligence-based system for deriving expected clinical trial operation issues and generating response measures according to one embodiment of the present invention. FIG. 2 is a flowchart showing the overall processing flow of a method for deriving expected issues in clinical trial operations and generating countermeasures based on artificial intelligence according to an embodiment of the present invention. FIG. 3 is a flowchart illustrating the process of creating a response package based on a completed response case according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating the process of generating a response plan based on novel clinical trial operation data according to one embodiment of the present invention. FIG. 5 is a flowchart illustrating a corresponding package update process according to one embodiment of the present invention. Specific details for implementing the invention

[0031] Embodiments of the present invention are described in detail below. However, the embodiments described below are merely examples to aid in understanding the present invention, and the scope of the present invention is not limited to the embodiments described below. Additionally, the names of components used below are for convenience of explanation, and components performing the same or equivalent functions may be referred to by different names.

[0032] An artificial intelligence-based system for deriving expected clinical trial operation issues and generating response measures according to one embodiment of the present invention may be a system that analyzes completed clinical trial operation issue response cases to derive expected operational issues that may occur in a new clinical trial and generates response measures for said expected operational issues.

[0033] More specifically, the system of the present invention can extract an initial operational issue and a subsequent operational issue from a completed response case and convert the initial operational issue and the subsequent operational issue into a standard response case profile. In addition, the system can generate an issue chain relationship by classifying the initial operational issue and the subsequent operational issue into respective operational issue units and determining the chronological relationship, causal relationship, conditional relationship, same affected subject relationship, same clinical trial target drug relationship, same lot relationship, same subject relationship, same institution relationship, same documentation item relationship, or same reporting procedure relationship between the operational issue units.

[0034] In addition, the system of the present invention can generate a sequential response procedure sequence using the issue chain relationship and the response procedure included in the completed response case, and generate a response package for each issue type based on the issue chain relationship and the sequential response procedure sequence.

[0035] The above-mentioned response package may be a standardized response unit applicable to new clinical trial operational data. The system of the present invention can generate a new operational profile from new clinical trial operational data and calculate a response package applicability score by comparing the new operational profile with a plurality of response packages stored in a response package database.

[0036] In one embodiment, the response package applicability score may be calculated based on at least one of the issue type gateway factor, problem situation match score, operating environment adjustment score, clinical trial target drug similarity score, and evidence reliability score. For example, the response package applicability score P may be calculated based on the following mathematical formula.

[0037]

[0038] Here, G is the issue type gateway factor, C is the problem situation consistency score, O is the operating environment adjustment score, D is the clinical trial target drug similarity score, and R is the evidence reliability score. w1 through w4 may be weights applied to each score.

[0039] The above system may select the corresponding package as a corresponding package applicable to new clinical trial operational data if the applicability score of the corresponding package is greater than or equal to the applicable threshold value, location information of the supporting data included in the corresponding package exists, and version information of the supporting data included in the corresponding package corresponds to the current version.

[0040] In one embodiment, the system of the present invention may apply the selected response package as is. In another embodiment, if there is a difference between the new clinical trial operational data and the selected response package, the system of the present invention may generate a corrected response package by correcting some items of the selected response package. Based on the selected response package or the corrected response package, the system may generate a response plan including data requiring verification, initial measures, reporting targets, documentation items, measures to prevent recurrence, follow-up observation items, items requiring modification, or items requiring expert review.

[0041] In addition, when a new completed response case or revised / changed response basis data is input, the system identifies the response package to be updated and can generate an updated response package by comparing the new completed response case and the revised / changed response basis data with the response package to be updated. The system can store a package change history, including the version before update, the version after update, the changed basis data, the changed items, the reason for the change, and the update date and time, in the response package database.

[0042] According to FIG. 2, a method according to one embodiment of the present invention may include steps from inputting or storing a completed response case (S100) to updating a response package and storing a package change history (S250).

[0043] Specifically, the system may input or store completed response cases (S100). A completed response case may include the initial operational issue that occurred in a specific clinical trial, subsequent operational issues identified or processed after the initial operational issue, the time of occurrence, time of processing, processing order, stage of occurrence, cause of occurrence, affected subjects, verification data, response procedure, result of action, reporting procedure information, location information of supporting data, and version information of supporting data for each operational issue.

[0044] Next, the system can extract the initial operational issue and subsequent operational issue from the completed response case (S110). The system can classify the initial operational issue and the subsequent operational issue into respective operational issue units (S120).

[0045] Next, the system can determine the relationship between operational issue units to create an issue chain relationship (S130). The relationship may include at least one of a chronological relationship, a causal relationship, a conditional relationship, a relationship with the same affected subject, a relationship with the same clinical trial drug, a relationship with the same lot, a relationship with the same subject, a relationship with the same institution, a relationship with the same documented item, or a relationship with the same reporting procedure.

[0046] Next, the system can generate a sequential response procedure sequence from the response procedures included in the issue chain and completed response cases (S140). The sequential response procedure sequence may include at least one of securing verification data, initial action, reporting, documentation, measures to prevent recurrence, and follow-up observation.

[0047] Next, the system can generate a response package based on an issue chain relationship and a sequential response procedure sequence (S150). The response package may include at least one of an issue type code, a parent issue group code, occurrence conditions, subsequent operational issues, materials requiring verification, response requirements, a sequential response procedure sequence, information on the drug subject to clinical trial, standardization code of the drug subject to clinical trial, reporting targets, documentation items, measures to prevent recurrence, follow-up observation items, source data location information, source data type, and source data version information.

[0048] Next, the system can store the generated response package in the response package database (S160).

[0049] Next, the system may receive new clinical trial operational data (S170). The new clinical trial operational data may include at least one of a clinical trial protocol, a manual for managing investigational medicinal products, information on managing investigational medicinal products, a visit schedule, a dosing schedule, information on managing specimens, information on roles by institution, and documentation standards.

[0050] Next, the system can generate a new operational profile from the new clinical trial operational data (S180). The new operational profile may be standardized data that allows the new clinical trial operational data to be compared with a corresponding package.

[0051] Next, the system can calculate a response package applicability score between each of the new operational profile and multiple response packages (S190). The response package applicability score can be calculated based on the issue type gateway factor, problem situation match score, operational environment adjustment score, clinical trial target drug similarity score, and evidence reliability score.

[0052] Next, the system can select an applicable response package (S200). The system can derive an issue type corresponding to the selected response package as an expected operational issue (S210).

[0053] Next, the system can generate a corrected response package by correcting items among the selected response packages that differ from the new clinical trial operation data (S220). The system can generate a response plan based on the selected response package or the corrected response package (S230).

[0054] Next, the system can output items requiring modification and items requiring expert review (S240). Items requiring modification may be items to which a corresponding package can be applied, but which need to reflect differences with the new clinical trial operational data. Items requiring expert review may be items to which automatic application is difficult due to insufficient supporting data, discrepancies in supporting data versions, unclear correspondence between the supporting sentence and the output item, or insufficient score for the applicability of the corresponding package.

[0055] In addition, when a new completed response case or revised / changed response basis data is entered, the system can update the response package and save the package change history (S250).

[0056] The order of each of the above steps is an example for convenience of explanation, and within the scope of the technical concept of the present invention, some steps may be performed in parallel, repeatedly, or with some order changed.

[0057] One embodiment of the present invention can be distinguished from simply generating natural language responses for new clinical trial operational data. The system of the present invention extracts operational issue units and issue chain relationships from completed response cases, converts the issue chain relationships and sequential response procedure sequences into structured data called response packages, and then calculates applicability with new clinical trial operational data.

[0058] Accordingly, the system of the present invention can systematically derive operational issues that need to be subsequently verified or processed when an operational issue occurs. For example, if a temperature deviation of a drug subject to a clinical trial occurs, the system does not simply output a result stating "respond to the temperature deviation," but can derive subsequent operational issues such as verifying temperature records, calculating the deviation time, calculating the cumulative deviation time, verifying lot information, verifying whether it is before or after administration, verifying stability data, determining the necessity of observing subject safety, and reporting and documentation.

[0059] Furthermore, the system of the present invention can quantitatively or semi-quantitatively determine whether a response package generated from completed response cases is actually applicable to new clinical trial operational data by calculating a response package applicability score. The response package applicability score may consider not only the identity of the issue type but also the stage of occurrence, affected subjects, administration status, operational environment, characteristics of the clinical trial drug, and the reliability of the evidence.

[0060] In addition, the system of the present invention can track the basis of a response package or response plan by utilizing source data location information, source data type, and source data version information. Accordingly, if source data location information does not exist, if source data version information does not correspond to the current version, or if the correspondence between the source sentence extracted from source data location information and the output item is not confirmed, the system can output the corresponding item as an item requiring expert review.

[0061] In addition, the system of the present invention can update existing response packages and store package change history when new completed response cases or revised / modified response basis data are input. The revised / modified response basis data may include at least one of revised clinical trial regulations, revised guidelines, modified Q&A data, modified standard operating procedures, modified clinical trial protocols, or modified investigational medicinal product management manuals. Accordingly, the timeliness and traceability of the response package can be maintained.

[0062] In one embodiment, the system of the present invention may further include a development case database that stores a plurality of development cases. The development case database may be distinguished from a completed response case database that stores the issues themselves that occurred during the operation of a clinical trial.

[0063] The completed response case database may be a database that stores operational issues that actually occurred during the course of clinical trial operations and their corresponding response cases. For example, response cases for clinical trial operational issues such as temperature deviations, missed visits, specimen handling errors, medication errors, deviations from the clinical trial protocol, reporting delays, or omissions in documentation may be stored in the completed response case database.

[0064] On the other hand, the development case database may be a database that stores development cases similar to the drug or candidate substance subject to clinical trials. The development case database may store a development case profile for each development case, which includes a modality code, regulatory classification code, mechanism of action (MOA) code, target molecule identifier, signaling pathway identifier, direction of action code, site of action code, route of administration code, exposure characteristic code, development phase information, safety event information, toxicity event information, reason for discontinuation information, reason for clinical hold information, and location information of supporting data.

[0065] The system can identify development cases similar to the target drug of a new clinical trial by utilizing a development case database. Additionally, the system classifies safety events, toxicity events, reasons for development discontinuation, or reasons for clinical hold identified in similar development cases as operational risks, and can incorporate these operational risks into the response plan or corrective response package for the new clinical trial.

[0066] For example, if the investigational drug of a new clinical trial is associated with a specific modality, mechanism of action (MOA), and target molecule, the system may search the development case database for development cases having the same or similar modality, mechanism of action (MOA), or target molecule. The system may extract safety or toxicity events related to death, life-threatening, hospitalization, clinical hold, discontinuation of development, discontinuation of administration, or dose reduction from the retrieved development cases. Based on the location, type, and severity of the evidence for the extracted safety or toxicity events, the system may classify said events as predictable operational risks, operational risks requiring further verification, or operational risks of low applicability.

[0067] The development case database and operational risk reflection configuration of claim 4 may not be illustrated in a separate drawing. In one embodiment, the development case database may be indicated as an optional configuration in the overall system configuration diagram, and specific processing details regarding operational risk classification and reflection of countermeasures may be described in sentences in the detailed description.

[0068] According to FIG. 1, an artificial intelligence-based clinical trial operation expected issue derivation and response plan generation system (100) according to one embodiment of the present invention may include one or more processors (110), memory (120), a completed response case database (130), a clinical trial operation data database (140), a response basis database (150), a rule database (160), and a response package database (170).

[0069] The memory (120) can store commands and an artificial intelligence model (121) executed by the processor (110). The processor (110) can use the artificial intelligence model (121) to perform the analysis of completed response cases, classification of operational issue units, creation of issue chain relationships, creation of sequential response procedure sequences, creation of response packages, creation of new operational profiles, calculation of response package applicability scores, selection of response packages, creation of corrective response packages, creation of response plans, output of items requiring modification and items requiring expert review, creation of updated response packages, and storage of package change history.

[0070] In one embodiment, the system (100) may further include a development case database (180) that stores a plurality of development cases. The development case database (180) may be an optional configuration and may be used to classify safety events, toxicity events, reasons for discontinuation of development, or reasons for clinical hold as operational risks by analyzing development cases similar to the target drug of a new clinical trial. When creating drawings, the development case database (180) may be indicated by a dotted box to show that it is an optional configuration.

[0071] A user terminal (190) may be placed outside the system (100) and may be connected to communicate with the system (100). The user terminal (190) may input new clinical trial operational data, new completed response cases, or revised / changed response basis data into the system (100), or receive and display expected operational issues, response plans, items requiring modification, items requiring expert review, or package change history generated by the system (100).

[0072] The above processor (110), memory (120), completed response case database (130), clinical trial operation data database (140), response basis database (150), rule database (160), response package database (170) and development case database (180) may be implemented within a single physical device, or may be distributed and implemented in a plurality of servers connected via a network, a cloud computing environment, a private network-based computing environment, or an institution's internal network-based computing environment.

[0073] Additionally, each database may be implemented as a physically separated storage or as a logically separated data area within a single storage device. For example, the completed response case database (130), the clinical trial operation data database (140), the response basis database (150), the rule database (160), the response package database (170), and the development case database (180) may each be implemented as separate database servers. In another example, the databases may be implemented as different tables, collections, indexes, or schemas within a single database management system.

[0074] The processor (110) can perform each function of the present invention by executing instructions and an artificial intelligence model (121) stored in memory (120).

[0075] In one embodiment, the processor (110) can perform completed response case analysis, extraction of operational issue units, creation of standard response case profiles, creation of issue chain relationships, creation of sequential response procedure sequences, creation of response packages, storage of response package databases, creation of new operational profiles, calculation of response package applicability scores, selection of response packages, derivation of expected operational issues, creation of corrective response packages, creation of response plans, outputting items requiring modification, outputting items requiring expert review, creation of updated response packages, and storage of package change history.

[0076] Additionally, in an embodiment that includes a development case database (180), the processor (110) can generate a candidate substance profile, generate a standardization profile, compare with the development case profile, calculate a case similarity score, select a group of development cases to be analyzed, classify operational risks, and map operational risks for response measures.

[0077] The processor (110) may be implemented as a central processing unit, a graphics processing unit, a neural network processing unit, a tensor processing unit, an application processor, a microprocessor, or a combination thereof. Additionally, the processor (110) may be implemented as a single processor or as multiple processors that perform different functions.

[0078] The memory (120) can store instructions and an artificial intelligence model (121) executed by the processor (110). The memory (120) can be implemented as volatile memory, non-volatile memory, a database storage device, a file storage device, a cloud storage device, or a combination thereof.

[0079] The artificial intelligence model (121) may be a model for analyzing natural language sentences, tables, items, document structures, codes, or metadata included in completed response cases, new clinical trial operation data, response basis data, or development case data.

[0080] In one embodiment, the artificial intelligence model (121) may be implemented as a natural language processing model, a sentence embedding model, a document embedding model, a classification model, a named entity recognition model, a relationship extraction model, a document summarization model, a question answering model, a large-scale language model, or a combination thereof.

[0081] For example, a named entity recognition model can extract the initial operational issue, subsequent operational issue, time of occurrence, time of resolution, affected entities, lot identifier, subject identifier, agency identifier, administration status, reporting target, reporting deadline, reporting form, reporting path, or documentation items from completed response cases. A relationship extraction model can extract conditional or causal relationships between operational issue units based on phrases indicating conditional relationships or phrases indicating causal relationships. A sentence embedding model or a document embedding model can be used to calculate semantic similarity or clinical trial drug similarity between a new operational profile and a response package. A large-scale language model can be used to convert the extracted operational issue units, issue chains, and response procedures into a structured response package format.

[0082] The artificial intelligence model (121) can be used with rules stored in the rule database (160). For example, when the artificial intelligence model (121) extracts candidate operational issue units from completed response cases, the processor (110) can verify the type and relationship of the candidate operational issue units using issue classification rules or issue chain extraction rules stored in the rule database (160).

[0083] The completed response case database (130) can store completed response cases, which are completed clinical trial operation issue response cases.

[0084] A completed response case may include at least one of the following: an initial operational issue that occurred in a specific clinical trial; a subsequent operational issue identified or processed after the initial operational issue; a phrase indicating the time of occurrence, time of processing, processing sequence, stage of occurrence, cause of occurrence, conditional relationship of the initial operational issue or the subsequent operational issue; a phrase indicating the causal relationship; an affected subject; an affected subject identifier; information on the clinical trial drug; an identifier of the clinical trial drug; a lot identifier; a subject identifier; an institution identifier; drug administration status; subject safety impact information; verification data; response requirements; response procedures; results of actions; reporting subjects; reporting procedure information; documentation items; measures to prevent recurrence; follow-up observation items; source data location information; and source data version information.

[0085] Here, the initial operational issue may refer to an operational problem, incident, error, deviation, or situation requiring verification that is initially identified in a completed response case. Subsequent operational issues may refer to operational problems, incidents, errors, deviations, or situations requiring verification that need to be additionally identified or handled after the initial operational issue has been identified.

[0086] Confirmation materials may be data actually obtained or reviewed in completed response cases. For example, confirmation materials may include temperature records, medication records, visit records, specimen handling records, reporting records, safety data, internal review records, or supporting documents.

[0087] Reporting procedure information may refer to information including at least one of the reporting subject, reporting deadline, reporting format, reporting path, and reporting requirements for an operational issue. Reporting procedure information may be used to determine identical reporting procedure relationships when creating issue chain relationships, and may be used to derive reporting subjects or items requiring modification related to reporting when creating response plans.

[0088] For example, if a drug subject to a clinical trial is exposed to a temperature outside of acceptable storage conditions, the “occurrence of temperature deviation” may become the initial operational issue. In this case, “verification of temperature records,” “calculation of deviation time,” “verification of lot information,” “verification of whether it occurred before or after administration,” “verification of stability data,” “determination of the need for subject safety observation,” “reporting,” and “documentation” may become subsequent operational issues.

[0089] Completed response cases stored in the completed response case database (130) can be generated from question and answer logs, emails, meeting minutes, issue reports, monitoring records, clinical trial protocol deviation reports, temperature deviation reports, safety review records, internal review records, report documents, or documentation records.

[0090] The clinical trial operation data database (140) can store new clinical trial operation data related to the operation of a new clinical trial.

[0091] The new clinical trial operational data may include at least one of the following: a clinical trial protocol, a manual for the management of investigational medicinal products, information on the management of the medicinal products subject to the clinical trial, a visit schedule, a dosing schedule, information on specimen management, information on the roles of each institution, and documentation standards.

[0092] The clinical trial protocol may include subject selection criteria, exclusion criteria, visit schedule, dosing schedule, evaluation items, primary endpoints, safety evaluation items, discontinuation criteria, reporting criteria, protocol withdrawal handling criteria, and documentation criteria.

[0093] The investigational drug management manual may refer to documents regarding the storage, dispensing, dispensing, administration, retrieval, disposal, lot management, temperature record management, and response to temperature deviations of the investigational drug.

[0094] Clinical trial drug management information may include at least one of clinical trial drug identification information, candidate substance name, modality code, regulatory classification code, mechanism of action (MOA) code, target molecule identifier, direction of action code, site of action code, route of administration code, exposure characteristic code, storage conditions, preparation conditions, administration conditions, and lot information.

[0095] The visit schedule may include information regarding the subject's screening visit, baseline visit, medication visit, follow-up visit, termination visit, or additional visit. The medication schedule may include the medication date, number of doses, dosage, dosing interval, administration route, dosage adjustment criteria, and medication discontinuation criteria. Specimen management information may include the time of sample collection, sample processing time, storage conditions, transportation conditions, analysis laboratory information, and whether re-collection is possible.

[0096] The response basis database (150) can store response basis data that serves as the basis for a response package or response plan.

[0097] In one embodiment, the corresponding basis database (150) may store at least one of clinical trial-related regulations, guidelines, Q&A materials, standard operating procedures, internal review records, clinical trial protocol provisions, investigational drug management manual provisions, investigator's manual, approved product labels, and safety data.

[0098] The response basis database (150) can store a document identifier, document name, document type, source material type, version information, effective date, revision date, authoring organization, location information within the document, page number, paragraph number, item name, table number, clause number, link information, or original text for each response basis material.

[0099] Source data location information may be information indicating which location within which response source data a specific item included in a response package or response plan is based on. For example, source data location information may include information identifying a specific provision of a clinical trial protocol, a specific item of a manual for the management of investigational medicinal products, a specific safety summary item of an investigator's manual, a specific paragraph of a guideline, or a specific section of an approved product label.

[0100] The type of supporting data may be information indicating the source or reliability level of the data serving as the basis for the response package or response plan. The type of supporting data may include any one of the following: regulatory agency documents, approved product labels, investigator's manuals, clinical trial outcome documents, safety data, clinical trial registration information, Q&A materials, standard operating procedures, internal review records, completed response cases, or company press releases.

[0101] The source data version information may be information indicating the version of the response source data referenced when generating a response package or response plan. The system can use the source data version information to determine whether the response package corresponds to the current version applied to the new clinical trial operational data.

[0102] The current version may refer to the approved, effective, or valid version of the clinical trial protocol, investigational medicinal product management manual, standard operating procedures, clinical trial-related regulations, guidelines, or Q&A materials actually applied to the new clinical trial operational data.

[0103] The rule database (160) can store at least one of an issue classification rule, an issue chain extraction rule, a response procedure extraction rule, a response package application rule, an applicability score calculation rule, a risk classification rule, a weight set, a mapping rule, and top issue group definition information.

[0104] Issue classification rules may be rules for classifying operational issue units into issue type codes or parent issue group codes.

[0105] A top-level issue group may be classification information for grouping multiple issue type codes into a top-level category. In one embodiment, the top-level issue group may correspond to any one of the management of the clinical trial subject drug, medication management, visit management, specimen management, reporting management, documentation management, and clinical trial protocol deviation management.

[0106] For example, temperature deviations, deviations from storage conditions, lot isolation, and recall of the investigational drug may be classified under the investigational drug management group. Missed visits, delayed visits, and exceeding the permitted visit period may be classified under the visit management group. Delayed specimen processing, deviations from specimen storage conditions, or specimen transport errors may be classified under the specimen management group.

[0107] The issue chain extraction rule may be a rule for determining the relationship between operational issue units based on the time of occurrence, time of processing, order of processing, phrases indicating conditional relationships, phrases indicating causal relationships, results of actions, identifiers of affected subjects, identifiers of drugs subject to clinical trials, lot identifiers, subject identifiers, institution identifiers, documentation items, or reporting procedure information included in completed response cases.

[0108] The response procedure extraction rule may be a rule for classifying response procedures included in completed response cases into procedural steps such as securing verification data, initial action, reporting, documentation, preventive measures, or follow-up observation.

[0109] The response package application rules can define comparison criteria between the new operational profile and the response package, applicable threshold values, exclusion conditions, correction conditions, and output conditions for items requiring expert review.

[0110] The applicability score calculation rule may define coefficients, scores, weights, and calculation methods for calculating the response package applicability score. For example, the applicability score calculation rule may be defined to calculate the response package applicability score using the issue type gateway factor, problem situation match score, operating environment adjustment score, clinical trial target drug similarity score, and evidence reliability score.

[0111] In an embodiment using a development case database (180), the risk classification rule may be a rule for classifying safety events, toxicity events, reasons for development discontinuation or reasons for clinical hold into predictable operational risks, operational risks requiring further verification, or operational risks with low applicability.

[0112] Mapping rules may be rules for converting items extracted from response packages or development cases into items included in response plans. For example, predictable operational risks may be mapped to subject safety observation items, discontinuation criteria, dose adjustment criteria, reporting subjects, or items for review for inclusion in the investigator's manual.

[0113] The response package database (170) can store multiple response packages. Additionally, the response package database (170) can store correction response packages, update response packages, and package change history.

[0114] A response package may be data generated by issue type based on the issue chain relationships and sequential response procedure sequences generated from completed response cases.

[0115] The response package may include at least one of an issue type code, a parent issue group code, occurrence conditions, subsequent operational issues, materials requiring verification, response requirements, a sequential response procedure sequence, information on the drug subject to clinical trial, standardized code of the drug subject to clinical trial, reporting subjects, reporting procedure information, documentation items, preventive measures, follow-up observation items, source data location information, source data type, source data version information, creation date and time, update date and time, and history of applicability score calculation.

[0116] Data requiring verification may refer to data that must be secured or verified in the future to apply a response package or a correction response package to new clinical trial operational data. Data requiring verification may be derived based on verified data actually secured or reviewed in completed response cases.

[0117] The corrective response package may be a modified version in which some items of the selected response package are adjusted to reflect the differences between the new clinical trial operational data and the selected response package. For example, if the reporting targets, reporting procedure information, documentation items, data requiring verification, or follow-up observation items of the selected response package differ from the new clinical trial protocol or the investigational medicinal product management manual, the system may correct those items to align with the new clinical trial operational data.

[0118] The updated response package may be the result of updating an existing response package based on new completed response cases or revised / changed response basis materials. The package change history may include at least one of the following: the version before the update, the version after the update, the changed basis materials, the changed items, the reason for the change, and the update date and time.

[0119] The response package database (170) may additionally store the application history, exclusion history, output history of items requiring expert review, or correction history for each response package. Through this, the system can track which new clinical trial operational data a specific response package was applied to, which items were corrected, which supporting data was referenced, and for what reason items requiring expert review were output.

[0120] In one embodiment, the system (100) may further include a development case database (180) that stores a plurality of development cases.

[0121] The development case database (180) can store a development case profile for each development case that includes at least one of a modality code, regulatory classification code, mechanism of action (MOA) code, target molecule identifier, signaling pathway identifier, direction of action code, site of action code, administration route code, exposure characteristic code, development stage information, safety event information, toxicity event information, reason for discontinuation of development information, reason for clinical hold information and source data location information.

[0122] Development cases may include approved product cases, drugs under development cases, clinical trial cases, development discontinuation cases, clinical hold cases, publicly disclosed clinical trial results cases, or regulatory review cases.

[0123] The development case database (180) can be compared with the clinical trial target drug information included in the new clinical trial operation data. The system can generate a candidate substance profile based on the clinical trial target drug information of the new clinical trial and convert the candidate substance profile into the same standard tag system as the development case profile. Subsequently, the system can calculate a case similarity score based on whether there is a semantic correspondence between the standardized profile and the development case profile, and identify similar development cases based on the case similarity score.

[0124] The development case database (180) can be linked with the response package database (170). For example, when the system selects a response package applicable to new clinical trial operational data or generates a correction response package, it can reflect predictable operational risks or operational risks requiring additional verification derived from the development case database (180) in the response plan.

[0125] A completed response case may refer to an instance where, following the occurrence of an operational issue in a specific clinical trial, verification data was secured, initial measures were taken, reporting was done, documentation was done, preventive measures were taken, or follow-up observation was performed regarding the said operational issue.

[0126] A completed response case may include only one operational issue, or it may include an initial operational issue and multiple subsequent operational issues. For example, a case of temperature deviation of a drug subject to clinical trial may include the occurrence of temperature deviation as the initial operational issue, and as subsequent operational issues, include verification of temperature records, calculation of deviation time, verification of lot information, verification of stability data, verification of whether it occurred before or after administration, reporting, and documentation.

[0127] An operational issue unit may be a data unit that classifies the initial or subsequent operational issues included in a completed response case into individually processable units.

[0128] For example, the occurrence of temperature deviation, verification of temperature records, calculation of deviation time, calculation of cumulative deviation time, verification of lot information, verification of whether it was before or after administration, verification of stability data, determination of the need for subject safety observation, and reporting and documentation can each be units of operational issues.

[0129] The operational issue unit may include an issue type code, time of occurrence, time of processing, stage of occurrence, cause of occurrence, affected subjects, verification materials, response requirements, results of actions, reporting procedure information, documentation items, or location information of supporting materials.

[0130] Confirmation data may refer to data actually obtained or reviewed in completed response cases. For example, confirmation data may include temperature records, medication records, visit records, specimen handling records, reporting records, safety data, internal review records, or supporting documents.

[0131] Data requiring verification may refer to data that must be secured or verified in the future to apply a response package or a correction response package to new clinical trial operational data. Data requiring verification may be derived based on verified data actually secured or reviewed in completed response cases.

[0132] For example, in completed response cases, temperature records, lot information, and stability data actually verified for temperature deviation response can serve as verification data. On the other hand, temperature records, lot information, and stability data that need to be secured in the future to apply the temperature deviation response package in new clinical trials may serve as verification-needed data.

[0133] Reporting procedure information may mean information including at least one of the reporting subject, reporting deadline, reporting form, reporting path, and reporting requirements for an operational issue.

[0134] Reporting procedure information can be used to determine identical reporting procedure relationships when creating issue chain relationships. Additionally, reporting procedure information can be used to identify reporting targets, deadlines, reporting formats, reporting paths, or items requiring modification related to reporting when generating response plans.

[0135] For example, in a completed response case, if a specific operational issue is reported to the clinical trial principal investigator and sponsor within 24 hours and is documented according to a specific reporting form, the subject, deadline, form, and reporting path may be stored as reporting procedure information.

[0136] Issue chain relationships may be structured data that defines chronological relationships, causal relationships, conditional relationships, same affected subjects, same clinical trial drugs, same lot relationships, same subjects, same institution relationships, same documentation items, or same reporting procedures between an initial operational issue and a subsequent operational issue.

[0137] The issue chain relationship may include subsequent operational issues, conditions for the occurrence of subsequent operational issues, data requiring verification, conditions for response, and processing sequence.

[0138] For example, if a temperature deviation occurs in a drug subject to a clinical trial, verifying the temperature record may be a subsequent operational issue that must be performed after the temperature deviation occurs. Calculating the time of deviation may be a subsequent operational issue that must be performed after verifying the temperature record. Determining whether it is before or after administration may have a conditional relationship with the assessment of safety impact on the relevant lot or subject.

[0139] A sequential response procedure sequence may be data indicating the order of response procedures to be performed for an operational issue.

[0140] The sequential response procedure sequence may include at least one of securing verification data, initial action, reporting, documentation, measures to prevent recurrence, and follow-up observation.

[0141] For example, a sequential response procedure sequence regarding temperature deviation response may include securing temperature records, verifying acceptable storage conditions, calculating actual deviation time, calculating cumulative deviation time, lot isolation, verifying stability data, determining availability, determining the need for subject safety observation, and reporting and documentation.

[0142] A response package may be an intermediate deliverable generated by issue type based on the issue chain relationships and sequential response procedure sequences generated from completed response cases.

[0143] The response package may include at least one of an issue type code, a parent issue group code, occurrence conditions, subsequent operational issues, data requiring verification, response requirements, sequential response procedure sequence, information on the drug subject to clinical trial, standardization code of the drug subject to clinical trial, reporting targets, reporting procedure information, documentation items, measures to prevent recurrence, follow-up observation items, source data location information, source data type, and source data version information.

[0144] The response package can be applied directly to new clinical trial operational data, or it can be applied after being adjusted to reflect differences from the new clinical trial operational data.

[0145] The response plan may be the final output generated as a result of selecting or correcting a response package for new clinical trial operational data.

[0146] The response plan may include at least one of the following: materials requiring verification, initial measures, reporting targets, documentation items, measures to prevent recurrence, follow-up observation items, items requiring correction, and items requiring expert review.

[0147] Response plans can be generated based on the current versions of the clinical trial protocol, investigational medicinal product management manual, visit schedule, dosing schedule, specimen management information, institutional role information, documentation standards, and response basis data included in the new clinical trial operational data.

[0148] The corrective response package may be a response package in which some items of the selected response package have been modified to suit the current situation, reflecting the difference between the new clinical trial operational data and the selected response package.

[0149] The above differences may include differences in the protocol, differences in the investigational drug management manual, changes in regulations, differences in the version of the supporting data, differences in the administration status, differences in subject safety effects, or differences in the investigational drug.

[0150] For example, if the reporting target is set to the internal quality control department in the response package generated from a completed response case, but is set to the Principal Investigator, Sponsor, and Safety Officer in the new clinical trial operational data, the system can adjust the reporting target items to match the new clinical trial operational data.

[0151] An updated response package may be a response package in which the version of the response package itself has been updated based on new completed response cases or revised / changed response basis materials.

[0152] The above-mentioned revised / changed response basis materials may include at least one of the revised clinical trial regulations, revised guidelines, revised Q&A materials, revised standard operating procedures, revised clinical trial protocols, or revised investigational medicinal product management manuals.

[0153] The updated response package may be a result in which at least one of the occurrence conditions, subsequent operational issues, materials requiring verification, response requirements, reporting targets, documentation items, sequential response procedure sequence, recurrence prevention measures, or follow-up observation items of the existing response package has been changed.

[0154] The package change history for the update response package can be stored in the response package database (170).

[0155] The candidate issue type code may be a code representing a potential operational issue type derived from new clinical trial operational data.

[0156] Candidate issue type codes can be generated prior to the stage of calculating the response package applicability score. The system can calculate the issue type gateway coefficient by comparing the candidate issue type codes with the issue type codes of the response package.

[0157] Candidate issue type codes may be confirmed as final expected operational issues based on the response package applicability score, or they may be excluded if deemed to have low applicability.

[0158] A top-level issue group can be classification information for grouping multiple issue type codes into a top-level category.

[0159] In one embodiment, the top issue group may correspond to any one of the management of the clinical trial subject drug, medication management, visit management, specimen management, reporting management, documentation management, and clinical trial protocol deviation management.

[0160] For example, deviations in temperature, deviations in storage conditions, lot isolation, and the recall or disposal of the investigational drug may be included in the investigational drug management group. Missed visits, delayed visits, or exceeding the permitted visit period may be included in the visit management group. Delays in specimen processing, deviations in specimen storage conditions, or specimen transport errors may be included in the specimen management group.

[0161] The top issue group can be managed based on the top issue group definition information stored in the rule database (160) and can be used together with the top issue group code of the response package or the candidate issue type code of the new operation profile.

[0162] The type of supporting data may be information indicating the source or reliability level of the data serving as the basis for the response package or response plan.

[0163] The types of supporting materials may include any one of regulatory agency documents, approved product labels, investigator's manuals, clinical trial results documents, safety data, clinical trial registration information, Q&A materials, standard operating procedures, internal review records, completed response cases, or company press releases.

[0164] The system can calculate the reliability score of evidence using the type of evidence. For example, regulatory agency documents, approved product labels, investigator's brochures, clinical trial results documents, or safety data may have a relatively high reliability score of evidence. On the other hand, secondary summaries from unknown sources or data with unclear location information may have a relatively low reliability score of evidence.

[0165] The current version may refer to the approved, effective, or valid version of the clinical trial protocol, investigational medicinal product management manual, standard operating procedures, clinical trial-related regulations, guidelines, or Q&A materials actually applied to the new clinical trial operational data.

[0166] The system can determine whether the version information of the supporting data included in the response package corresponds to the current version. If the version information of the supporting data does not correspond to the current version, the system may not select the corresponding response package as an applicable response package, or may output the item as an item requiring expert review.

[0167] Items requiring modification may refer to items where the applicability score of the corresponding package is above the applicable threshold value, allowing the corresponding package to be applied to the new clinical trial operational data, but modification is required due to differences between the new operational profile and the selected corresponding package.

[0168] The above differences may include at least one of the following: differences in the protocol, differences in the investigational drug management manual, changes in regulations, differences in the version of the supporting data, differences in the administration status, differences in subject safety effects, or differences in the investigational drug.

[0169] For example, if the reporting target of the selected response package differs from the reporting target of the new clinical trial protocol, the system may output the reporting target item as an item requiring modification.

[0170] Items requiring expert review may refer to items that require expert review because automatic application is difficult, such as when the applicability score of the response package is below the applicable threshold value, when source data location information is missing, when source data version information does not correspond to the current version, or when the correspondence between the source sentence extracted from the source data location information and the output item cannot be confirmed.

[0171] Additionally, in an embodiment using the development case database (180), if the severity, stage of occurrence, or development discontinuation correlation of an event identified from multiple source data location information is inconsistent with each other, the system may output the corresponding operational risk or response plan item as an item requiring expert review.

[0172] Operational risk may refer to the risk that, based on the characteristics of the drug under investigation, safety events, toxicity events, or reasons for discontinuation or hold in similar development cases, it becomes necessary to monitor subject safety, establish discontinuation criteria, adjust dosage, establish reporting targets, incorporate into the investigator's data book, or conduct a supplementary review for clinical trial plan approval during the operation of a new clinical trial.

[0173] Operational risk can be derived in an embodiment using a development case database (180). The system calculates the similarity between the information on the target drug of a new clinical trial and the development case profile, and can classify safety events or toxicity events identified in similar development cases as operational risk.

[0174] In one embodiment, operational risks may be classified as foreseeable operational risks, operational risks requiring further verification, or operational risks of low applicability. Foreseeable operational risks may be classified when there is a high degree of similarity to similar development cases and the location information of the evidence for safety or toxicity events corresponds to highly reliable data. Operational risks requiring further verification may be classified when the degree of similarity is above a certain level but further review of the reliability of the evidence or the severity of the event is required. Operational risks of low applicability may be classified when there is low similarity to similar development cases or low correspondence with the information of the investigational drug in a novel clinical trial.

[0175] According to FIG. 3, in one embodiment of the present invention, the processor (110) receives a completed response case (S300) and can extract an initial operational issue, a subsequent operational issue, a occurrence condition, data requiring verification, conditions requiring response, a response procedure, a result of action, location information of supporting data, and version information of supporting data from the completed response case (S310). In addition, the processor (110) converts the initial operational issue and the subsequent operational issue into a standard response case profile (S320) and can separate the initial operational issue and the subsequent operational issue into respective operational issue units (S330).

[0176] The processor (110) determines the relationship between operational issue units (S340) and can generate an issue chain relationship based on the result of the relationship determination (S350). Additionally, the processor (110) can generate a sequential response procedure sequence from the issue chain relationship and the response procedure (S360), generate a response package by issue type (S370), and then store the generated response package in the response package database (170) (S380).

[0177] At this time, the relationships between operational issue units may include at least one of the following: a chronological relationship, a causal relationship, a conditional relationship, a relationship involving the same affected subject, a relationship involving the same clinical trial drug, a relationship involving the same lot, a relationship involving the same subject, a relationship involving the same institution, a relationship involving the same documented item, and a relationship involving the same reporting procedure.

[0178] The processor (110) can read out completed response cases stored in the completed response case database (130) or receive completed response cases from a user terminal (190) or an external system.

[0179] Completed response cases may be stored in the form of structured, semi-structured, or unstructured data. For example, completed response cases may be stored in the form of issue reports, Q&A logs, emails, meeting minutes, monitoring records, clinical trial protocol deviation reports, temperature deviation reports, specimen handling error reports, medication error reports, internal review records, follow-up records, or reporting documents.

[0180] The processor (110) can use an artificial intelligence model (121) to extract initial operational issues, subsequent operational issues, occurrence conditions, verification data, response requirements, response procedures, action results, reporting procedure information, source data location information, and source data version information from completed response cases.

[0181] Here, verification materials may be materials actually obtained or reviewed in completed response cases. For example, in a temperature deviation case, temperature records, lot information, dispensing records, stability data, or internal review records actually attached to or reviewed in the completed response case may qualify as verification materials.

[0182] Based on the extracted verification data, the processor (110) can derive data that needs to be secured or verified in the future when applying the same or similar response package to new clinical trial operation data. For example, if temperature records and lot information are actually verified in a completed response case, the processor (110) can include temperature records and lot information as data that needs to be verified for the temperature deviation response package.

[0183] Reporting procedure information may include at least one of the reporting subject, reporting deadline, reporting form, reporting path, and reporting requirements for an operational issue. The processor (110) may extract reporting procedure information by analyzing the report sentence, report record, email recipient, report form, reporting deadline, or reporting path included in the completed response case.

[0184] In one embodiment, the processor (110) can analyze sentences included in completed response cases to extract phrases indicating operational issues. For example, phrases such as “the investigational drug was stored at a temperature exceeding the permitted storage temperature,” “the subject did not visit on the scheduled visit date,” “the sample was not centrifuged within the time specified in the clinical trial protocol,” or “a dose different from that specified in the clinical trial protocol was administered to the subject” can be extracted as phrases indicating operational issues.

[0185] In another embodiment, the processor (110) can distinguish between an initial operational issue and a subsequent operational issue by analyzing time information, sequence information, or procedure information included in a completed response case. For example, if the completed response case includes the content “after confirming a temperature deviation, the temperature record was reviewed, and subsequently stability data was checked,” the processor (110) can extract “confirmation of temperature deviation” as the initial operational issue and “review of temperature record” and “checking stability data” as subsequent operational issues.

[0186] In another embodiment, the processor (110) may extract phrases indicating a conditional relationship or phrases indicating a causal relationship. Phrases indicating a conditional relationship may include expressions such as “if,” “if,” “corresponding to,” “if,” “if,” “if,” “if,” “if,” or “if.” Phrases indicating a causal relationship may include expressions such as “due to,” “according to,” “based on,” “because,” or “as a result of.”

[0187] The processor (110) can convert the extracted initial operation issue and subsequent operation issue into a standard response case profile.

[0188] A standard response case profile may be standardized data for comparing, classifying, and reusing completed response cases recorded in different formats based on the same criteria.

[0189] In one embodiment, the standard response profile may include at least one of an issue type code, a parent issue group code, an occurrence stage code, an occurrence cause code, an affected target code, a clinical trial drug code, a clinical trial drug standardization code, a modality code, a regulatory classification code, a mechanism of action (MOA) code, a target molecule identifier, a dosing status code, a subject safety impact code, a reporting target code, a reporting procedure code, a documentation code, source data location information, source data type, and source data version information.

[0190] The processor (110) can convert operational issues into issue type codes and parent issue group codes using issue classification rules and parent issue group definition information stored in the rule database (160).

[0191] A top-level issue group may be classification information for grouping multiple issue type codes into a top-level category. In one embodiment, the top-level issue group may correspond to any one of the management of the clinical trial subject drug, medication management, visit management, specimen management, reporting management, documentation management, and clinical trial protocol deviation management.

[0192] For example, “Temperature deviation,” “Storage condition deviation,” “Lot isolation,” and “Recall of investigational medicinal products” can be converted to the investigational medicinal product management group. “Missed visit,” “Visit delay,” and “Exceeding allowed visit period” can be converted to the visit management group. “Sample processing delay,” “Sample storage condition deviation,” and “Sample transport error” can be converted to the specimen management group.

[0193] The processor (110) can convert the candidate substance name, modality code, regulatory classification code, mechanism of action (MOA) code, target molecule identifier, direction of action code, site of action code, route of administration code, exposure characteristic code, storage conditions, preparation conditions, administration conditions, or lot information included in the clinical trial drug information into a standardized code for the clinical trial drug.

[0194] The processor (110) can link the source material location information and source material type included in the completed response case to the standard response case profile. The source material type may be any one of regulatory agency documents, approved product labels, investigator's manuals, clinical trial results documents, safety data, clinical trial registration information, Q&A data, standard operating procedures, internal review records, completed response cases, or company presentation materials.

[0195] Additionally, the processor (110) can store version information of the clinical trial protocol, investigational drug management manual, standard operating procedure, clinical trial related regulations, guidelines, or Q&A materials referenced in the completed response case as source material version information.

[0196] The processor (110) can separate the initial operational issue and subsequent operational issue included in the standard response case profile into each operational issue unit.

[0197] An operational issue unit may be the minimum processing unit that can be individually verified, judged, or processed within a completed response case. A single completed response case may include one operational issue unit or multiple operational issue units.

[0198] In one embodiment, the processor (110) can distinguish operational issue units using information on the occurrence time, processing time, processing order, procedure name, verification data, response requirements, documentation items, reporting targets, or reporting procedures included in the completed response case.

[0199] For example, in a completed response case regarding a temperature deviation of a drug subject to a clinical trial, the processor (110) can classify the occurrence of a temperature deviation, verification of temperature records, verification of acceptable storage conditions, calculation of deviation time, calculation of cumulative deviation time, verification of lot information, verification of whether the lot is used, verification of whether it is before or after administration, verification of stability data, determination of usability, determination of the need for subject safety observation, determination of the need for reporting, preparation of a deviation report, and establishment of measures to prevent recurrence into operational issue units.

[0200] As another example, in a completed response case regarding a missed visit, the processor (110) can classify the occurrence of a missed visit, verification of the allowed visit period, verification of whether a revisit is possible, verification of the missed evaluation item, verification of whether the sample is missing, review of the impact on key evaluation variables, review of the impact on subject safety, determination of whether there is a deviation from the clinical trial protocol, determination of the need for reporting, and establishment of documentation and preventive measures as operational issues.

[0201] As another example, in a completed response case regarding a sample processing error, the processor (110) can classify the occurrence of a sample processing error, verification of processing standards, verification of actual processing time, verification of storage conditions, verification of analyzability, verification of re-collectionability, review of the impact of key evaluation variables, reporting, and documentation into operational issue units.

[0202] The processor (110) can assign an issue type code, a parent issue group code, a time of occurrence, a time of processing, a stage of occurrence, a cause of occurrence, an affected target, verification data, candidate data requiring verification, conditions for response, results of action, reporting procedure information, documentation items, location information of supporting data, type of supporting data, and version information of supporting data for each operational issue unit.

[0203] Here, candidate data requiring verification is an item derived based on verification data actually obtained from completed response cases, and can be converted into data requiring verification to be applied to new clinical trial operational data when generating a response package thereafter.

[0204] The processor (110) can determine the relationship between operational issue units. In one embodiment, the processor (110) can determine at least one of the following relationships between operational issue units: a chronological relationship, a causal relationship, a conditional relationship, a relationship with the same affected subject, a relationship with the same clinical trial drug, a relationship with the same lot, a relationship with the same subject, a relationship with the same institution, a relationship with the same documented item, or a relationship with the same reporting procedure.

[0205] A chronological relationship may refer to a relationship where one operational issue unit occurs or must be processed before another. For example, if a temperature record check is performed after a temperature deviation occurs, and the deviation time calculation is performed after the temperature record check, a chronological relationship may exist between each operational issue unit.

[0206] A causal relationship may be a relationship in which one operational issue unit becomes the cause of another operational issue unit, or triggers the occurrence or processing of another operational issue unit. For example, if a review of the availability of a specific lot becomes necessary due to a deviation from storage conditions, a causal relationship may exist between the deviation from storage conditions and the review of availability.

[0207] A conditional relationship may be one in which subsequent operational issues arise or are processed only when specific conditions are met. For example, if the drug under investigation has already been administered to a subject, determining the necessity of observing subject safety may lead to a subsequent operational issue. On the other hand, if the drug under investigation has not yet been administered, lot isolation and review of availability may lead to a subsequent operational issue.

[0208] A common affected entity relationship may be a relationship in which multiple operational issue units are related to the same affected entity. The affected entity may be any one of the clinical trial drug, lot, subject, institution, specimen, document, reporting procedure, or evaluation item.

[0209] A same clinical trial target drug relationship may be a relationship in which multiple operational issue units relate to the same clinical trial target drug identifier. A same lot relationship may be a relationship in which multiple operational issue units relate to the same lot identifier. A same subject relationship may be a relationship in which multiple operational issue units relate to the same subject identifier. A same institution relationship may be a relationship in which multiple operational issue units relate to the same institution identifier.

[0210] A duplicate document item relationship may be a relationship in which multiple operational issue units relate to the same deviation report, monitoring record, clinical trial drug management record, specimen handling record, visit record, or safety review record.

[0211] A shared reporting procedure relationship may be a relationship in which multiple operational issue units are related to the same reporting target, reporting deadline, reporting form, reporting path, or reporting requirements. For example, if multiple operational issue units must be reported to the same reporting target within the same reporting deadline according to the same reporting form, the processor (110) may determine that a shared reporting procedure relationship exists between the multiple operational issue units.

[0212] The processor (110) can simultaneously assign multiple relationship types to a single pair of operational issue units. For example, “temperature deviation occurrence” and “corresponding lot isolation” can simultaneously have a chronological relationship, a cause relationship, and an identical lot relationship.

[0213] The processor (110) can determine the relationship between operational issue units based on at least one of the following: the time of occurrence, the time of processing, the order of processing, the phrase indicating the condition relationship, the phrase indicating the cause relationship, the result of the action, the identifier of the affected target, the identifier of the clinical trial target drug, the lot identifier, the subject identifier, the institution identifier, the documentation item, and the reporting procedure information included in the completed response case.

[0214] The chronological relationship can be determined based on the time of occurrence, the time of processing, or the order of processing. For example, if the time of occurrence of the first operational issue unit precedes the time of occurrence of the second operational issue unit, the processor (110) may determine that the first operational issue unit precedes the second operational issue unit. Additionally, if the completed response case includes sequence expressions such as “first,” “after,” “next,” “then,” or “finally,” the processor (110) may determine the chronological relationship based on said expressions.

[0215] Causal relationships can be determined based on phrases indicating causal relationships, causes of occurrence, or results of actions. For example, if a completed response case includes the sentence “due to temperature deviation, it was necessary to review the usability of the lot,” the processor (110) can determine that there is a causal relationship between “temperature deviation” and “review of usability.”

[0216] Conditional relationships can be determined based on phrases indicating conditional relationships, response requirements, medication status, information on subject safety effects, or the existence of safety data. For example, if a completed response case includes the content “isolate the lot if it is before medication, and review whether to observe subject safety if it is after medication,” the processor (110) can determine a conditional relationship between the condition “before medication” and “lot isolation” and a conditional relationship between the condition “after medication” and “review whether to observe subject safety.”

[0217] A same-affected-object relationship can be determined based on an affected-object identifier, a clinical trial drug identifier, a lot identifier, a subject identifier, or an institution identifier. For example, if a first operational issue unit and a second operational issue unit have the same lot identifier, the processor (110) can determine that a same-lot relationship exists between the two operational issue units.

[0218] Identical document item relationships can be determined based on a ditch report identifier, a document item name, a document template identifier, or record location information. For example, if multiple operational issue units are to be recorded in the same ditch report, the processor (110) can determine that an identical document item relationship exists between the multiple operational issue units.

[0219] A relationship of identical reporting procedures can be determined based on the reporting subject, reporting deadline, reporting form, reporting path, or reporting requirements. For example, if multiple operational issue units must be reported to the same reporting subject within the same reporting deadline, or must be reported using the same reporting form or reporting path, the processor (110) may determine that a relationship of identical reporting procedures exists between the multiple operational issue units.

[0220] In one embodiment, the processor (110) can verify a relationship candidate extracted by the artificial intelligence model (121) by comparing it with an issue chain extraction rule stored in the rule database (160). For example, if the artificial intelligence model (121) extracts a relationship between two operational issue units as a conditional relationship, the processor (110) can verify whether the relationship is supported by at least one of a phrase indicating a conditional relationship, a medication status, or subject safety impact information.

[0221] In another embodiment, the processor (110) can calculate the reliability of each relationship. The reliability of each relationship can be calculated based on at least one of the extraction reliability of the artificial intelligence model (121), the type of evidence used for determining the relationship, the number of sentences used for determining the relationship, the existence of multiple evidence location information supporting the same relationship, or the consistency with the rule database (160).

[0222] If the reliability of a relationship is less than a preset relationship reliability threshold, the processor (110) may store the relationship as a candidate relationship instead of storing it as a confirmed relationship. Additionally, the processor (110) may mark the relationship or the corresponding package item containing the relationship as an item requiring expert review.

[0223] The processor (110) can generate an issue chain relationship based on the result of determining the relationship between operational issue units.

[0224] The issue chain relationship may include subsequent operational issues that must be verified or processed following the initial operational issue, conditions for the occurrence of the said subsequent operational issues, materials required for verification, conditions required for response, and processing order.

[0225] In one embodiment, the issue chain relationship may be stored in a table format. For example, the issue chain relationship table may include a preceding operational issue, a subsequent operational issue, a relationship type, occurrence conditions, materials requiring verification, response requirements, processing order, reporting procedure information, source material location information, source material type, source material version information, and relationship reliability.

[0226] In another embodiment, the issue chain relationship can be stored in the form of a list. For example, a list in which subsequent operational issues are sequentially arranged for a single initial operational issue can be generated.

[0227] In another embodiment, issue chain relationships can be stored in a tree form. For example, a tree structure can be created where the initial operational issue serves as the root node, subsequent operational issues serve as child nodes, and branches based on occurrence conditions or response requirements. In the case of responding to temperature deviations, different subsequent operational issues may branch out depending on the "before administration" condition and the "after administration" condition.

[0228] In another embodiment, the issue chain relationship may be stored in the form of a graph. In this case, operational issue units may be represented as nodes, and the relationships between operational issue units may be represented as edges. Edges may include relationship type information such as chronological relationships, cause relationships, condition relationships, same affected subject relationships, same lot relationships, same subject relationships, or same reporting procedure relationships.

[0229] The processor (110) can store the generated issue chain relationship in a completed response case database (130), a response package database (170), or a separate issue chain relationship storage area. Additionally, the processor (110) can store the source material location information, source material type, and source material version information linked to each subsequent operational issue included in the issue chain relationship.

[0230] By creating an issue chain relationship in this way, the system (100) can convert individual measures included in completed response cases into structured response knowledge that can be reused in new clinical trials, rather than storing them in the form of simple documents or logs.

[0231] The processor (110) can generate a sequential response procedure sequence from the response procedures included in the issue chain relationship and the completed response case.

[0232] A sequential response procedure sequence may be data indicating the order of procedures to be performed for an operational issue. In one embodiment, the sequential response procedure sequence may include at least one of obtaining verification data, initial action, reporting, documentation, preventive measures, and follow-up observation.

[0233] Securing verification data may be a procedure for obtaining necessary data to verify the facts, scope of occurrence, affected subjects, or supporting data of an operational issue. For example, in the case of a temperature deviation, securing verification data may include a procedure for obtaining data regarding temperature records, storage conditions, time of deviation, cumulative time of deviation, lot information, and whether the drug was administered.

[0234] Initial measures may be actions that must be taken immediately after an operational issue is identified. For example, if a deviation from storage conditions occurs in the drug subject to a clinical trial, initial measures may include suspending the use of the lot, isolating it, withholding additional dosing, or requesting verification of stability data.

[0235] Reporting may be a procedure for notifying or reporting operational issues to internal departments, principal investigators, sponsors, institutions, safety officers, regulatory agencies, or other reporting subjects. The reporting procedure may include reporting subjects, reporting deadlines, reporting forms, reporting channels, and reporting requirements.

[0236] Documentation may be a procedure for recording in writing the occurrence of operational issues, verification data, basis for judgment, actions taken, reports, and follow-up actions. Items to be documented may include dropout reports, monitoring reports, records of management of the drug subject to clinical trial, specimen handling records, safety review records, subject records, or internal review records.

[0237] Preventive measures may be actions taken to ensure that the same or similar operational issues do not recur. For example, preventive measures may include training, revision of standard operating procedures, addition of checklists, enhanced temperature monitoring, addition of visit notification procedures, or supplementation of specimen handling procedures.

[0238] Follow-up monitoring may be a procedure to verify, over a certain period, the impact of operational issues on subject safety, primary endpoints, the quality of the investigational drug, or the reliability of the clinical trial results. For example, if there is a possibility of temperature deviation in the investigational drug that has already been administered, subject safety monitoring may be included as a follow-up item.

[0239] The processor (110) can classify the response procedures included in the completed response cases into procedure steps and combine them with the processing order of the issue chain relationship to generate a sequential response procedure sequence.

[0240] In one embodiment, the processor (110) can generate a sequential response procedure sequence in the following order.

[0241] First, the processor (110) extracts response procedure phrases for each operational issue unit. Next, the processor (110) classifies the extracted response procedure phrases into one of securing verification materials, initial action, reporting, documentation, recurrence prevention measures, and follow-up observation. Next, the processor (110) sorts the classified response procedures according to the processing order included in the issue chain relationship. Next, the processor (110) generates conditional branching procedures if a condition relationship exists. Next, the processor (110) links the source material location information, source material type, and source material version information corresponding to each procedure step.

[0242] Accordingly, the system (100) can structure the response contents scattered across completed response cases into a sequential response procedure sequence applicable to new clinical trials.

[0243] In one embodiment, the sequential response procedure sequence may include a conditional branch. The conditional branch may be structured such that different response procedures are selected depending on whether a specific condition is satisfied.

[0244] For example, in the case of a temperature deviation of a drug subject to clinical trials, the processor (110) may generate different procedures depending on the administration status. If the drug subject to clinical trials is in a pre-administration state, the sequential response procedure sequence may include isolating the lot, reviewing availability, verifying stability data, reporting, and documentation. On the other hand, if the drug subject to clinical trials has already been administered, the sequential response procedure sequence may include determining the need for subject safety observation, verifying safety evaluation items, reporting, and documentation.

[0245] As another example, in the case of a missed visit, the processor (110) may generate different procedures depending on whether a revisit is possible within the allowed visit period. If a revisit is possible within the allowed visit period, the sequential response procedure sequence may include scheduling the revisit, identifying and documenting the missed evaluation items. If a revisit is not possible within the allowed visit period, the sequential response procedure sequence may include reviewing the impact of key evaluation variables, determining whether there is a deviation from the clinical trial protocol, reporting, and taking measures to prevent recurrence.

[0246] As another example, in the case of a sample processing error, the processor (110) may generate different procedures depending on whether the sample can be analyzed or re-collected. If the sample can be analyzed, the sequential response procedure sequence may include securing and documenting the basis for the analysis possibility. If the sample cannot be analyzed but re-collected, the sequential response procedure sequence may include scheduling and documenting the re-collected schedule. If the sample cannot be analyzed and re-collected is also impossible, the sequential response procedure sequence may include reviewing the impact of key evaluation variables, reporting, and outputting items requiring expert review.

[0247] A sequential response procedure sequence including conditional branching can be used as a selection criterion for occurrence conditions, response requirements, or subsequent operational issues when generating a response package.

[0248] The processor (110) can generate a response package for each issue type based on an issue chain relationship and a sequential response procedure sequence.

[0249] A response package may be a structured response unit that combines issue chaining generated from completed response cases with a sequence of sequential response procedures. The response package may be used as an intermediate output to generate candidate response plans applicable to new clinical trial operational data.

[0250] In one embodiment, the response package may include at least one of an issue type code, a parent issue group code, occurrence conditions, subsequent operational issues, materials requiring verification, response requirements, sequential response procedure sequence, information on the drug subject to clinical trial, standardization code of the drug subject to clinical trial, reporting subjects, reporting procedure information, documentation items, measures to prevent recurrence, follow-up observation items, source data location information, source data type, and source data version information.

[0251] Issue type codes can indicate specific types of operational issues to which the response package applies. For example, issue type codes may indicate temperature deviations, storage condition deviations, missed visits, visit delays, specimen handling errors, medication errors, reporting delays, missing documentation, or deviations from the clinical trial protocol.

[0252] The parent issue group code can indicate the parent category to which the issue type code belongs. For example, temperature deviations and storage condition deviations may belong to the Clinical Trial Drug Management group, missed visits and visit delays may belong to the Visit Management group, and specimen handling errors and specimen storage condition deviations may belong to the Specimen Management group.

[0253] Occurrence conditions may indicate the conditions under which a response package is applied. For example, the occurrence condition for a temperature deviation response package may be when the investigational drug is exposed to a temperature outside of acceptable storage conditions. The occurrence condition for a missed visit response package may be when a subject fails to visit on the scheduled visit date or within the acceptable visit period.

[0254] Subsequent operational issues may represent issues that need to be verified or processed after the initial operational issue. For example, a temperature deviation response package may include verifying temperature records, calculating the time of deviation, verifying lot information, verifying stability data, verifying whether it is before or after administration, and determining the necessity of reporting as subsequent operational issues.

[0255] Data requiring verification may represent data that must be verified in order to apply the response package to new clinical trial operational data. For example, data requiring verification for a temperature deviation response package may include temperature records, acceptable storage conditions, actual storage temperature, time of deviation, cumulative time of deviation, lot information, dosing records, and stability data.

[0256] Response prerequisites may represent conditions that must be met to perform specific response procedures. For example, subject safety monitoring may satisfy a response prerequisite if the investigational drug has already been administered to the subject.

[0257] The sequential response procedure sequence may indicate the order of execution of procedures included in the response package. The reporting target may indicate the subject that must be reported upon the application of the response package. Reporting procedure information may include the reporting target, reporting deadline, reporting format, reporting path, or reporting requirements. Documentation items may indicate documents or items that must be recorded as a result of applying the response package. Recurrence prevention measures may indicate measures to prevent the recurrence of identical or similar issues. Follow-up items may indicate items that must be verified for a certain period following the occurrence of an operational issue.

[0258] Source material location information may indicate which source material each item included in the response package is based on. Source material type may indicate the source or confidence level of the corresponding source material. Source material version information may indicate the version of the source material referenced when creating the response package.

[0259] The processor (110) can generate one response package for one issue type. In another embodiment, the processor (110) can generate multiple response packages for one issue type. For example, for the same issue type of temperature deviation, a response package for temperature deviation before administration, a response package for temperature deviation after administration, a response package for temperature deviation when stability data exists, and a response package for temperature deviation when stability data does not exist can each be generated.

[0261] The processor (110) can standardize the generated corresponding package.

[0262] Standardization of response packages may be a procedure for organizing response packages generated from different completed response cases according to the same format, the same code system, or the same comparison criteria.

[0263] In one embodiment, the processor (110) can convert natural language items included in the corresponding package into standard codes. For example, items expressed differently, such as “temperature deviation,” “storage temperature exceeded,” and “allowable temperature range exceeded,” can be converted into the same issue type code.

[0264] In another embodiment, the processor (110) may standardize clinical trial drug information. The clinical trial drug information may include at least one of clinical trial drug identification information, candidate substance name, modality code, regulatory classification code, mechanism of action (MOA) code, target molecule identifier, direction of action code, site of action code, route of administration code, exposure characteristic code, storage conditions, preparation conditions, administration conditions, and lot information.

[0265] In another embodiment, the processor (110) may convert the reporting target, reporting procedure information, documentation items, follow-up items, or recurrence prevention measures included in the response package into standard items. For example, “PI reporting,” “Principal Investigator reporting,” and “Clinical Trial Principal Report” may be converted into the same reporting target code. Additionally, “email reporting,” “electronic system reporting,” and “reporting according to a defined reporting form” may be converted into standard codes regarding the reporting path or reporting form, respectively.

[0266] Standardizing response packages facilitates comparison between new operational profiles and response packages. Additionally, when calculating response package applicability scores, issue type gateway scores, problem situation match scores, operational environment adjustment scores, clinical trial drug similarity scores, and evidence reliability scores can be calculated in a consistent manner.

[0267] The processor (110) can link source data location information, source data type, and source data version information for each item included in the corresponding package.

[0268] Source material location information may indicate the location of the document that a specific item in the response package relies on as evidence. For example, source material location information may include a document identifier, document name, document type, page number, paragraph number, clause number, item name, table number, sentence number, or link information.

[0269] The type of supporting data may indicate the source or level of confidence of the supporting data that a specific item in the response package relies on. For example, the type of supporting data may be any one of regulatory agency documents, approved product labels, investigator's manuals, clinical trial outcome documents, safety data, clinical trial registration information, Q&A materials, standard operating procedures, internal review records, completed response cases, or company press releases.

[0270] Source data version information may indicate the version of the document that serves as the basis for a specific item in the response package. For example, the version of the clinical trial protocol, the version of the investigational medicinal product management manual, the version of the standard operating procedure, the revision date of the guideline, or the effective date of the regulation may be used as source data version information.

[0271] The processor (110) can connect each item of the corresponding package with the source data location information in a one-to-one or one-to-many relationship. For example, one reportable item may be based on both the clinical trial protocol provisions and the standard operating procedure provisions. In this case, the processor (110) can connect multiple source data location information to one reportable item.

[0272] The processor (110) may mark items for which source material location information does not exist as items lacking sufficient evidence. Additionally, if source material version information does not correspond to the current version, the processor (110) may mark the item as an item requiring expert review. Additionally, if the correspondence between the source sentence extracted from the source material location information and the corresponding package item is not confirmed, the processor (110) may mark the item as an item requiring expert review.

[0273] In this way, by linking the source data location information, source data type, and source data version information, each item in the response package can have traceable evidence and can subsequently be used to calculate the evidence reliability score.

[0274] The processor (110) can store the generated multiple corresponding packages in the corresponding package database (170).

[0275] The response package database (170) can store, for each response package, issue type code, top issue group code, occurrence condition, subsequent operation issue, data required for verification, response requirement condition, sequential response procedure sequence, information on the drug subject to clinical trial, standardization code of the drug subject to clinical trial, reporting target, reporting procedure information, documentation item, recurrence prevention measure, follow-up observation item, source data location information, source data type, source data version information, creation date and update date.

[0276] Additionally, the response package database (170) can store the history of calculating the applicability score of the response package, the history of selection, the history of correction, the history of exclusion, and the history of outputting items requiring expert review.

[0277] In one embodiment, the response package database (170) can index the response packages by issue type code. In another embodiment, the response package database (170) can index the response packages based on at least one of the top issue group code, standardization code of the drug subject to clinical trial, administration status code, subject safety impact code, source data type, or source data version information.

[0278] Response packages stored in the response package database (170) can be searched when new clinical trial operation data is entered. The processor (110) can search for candidate response packages in the response package database (170) based on candidate issue type codes, top issue group codes, clinical trial target drug information, administration status codes, subject safety impact codes, or standard operation codes included in the new operation profile.

[0279] The response packages stored in the response package database (170) may be updated when new completed response cases or revised / changed response basis data are entered. In this case, the processor (110) may store the response packages before the update and the response packages after the update separately, and store the package change history together.

[0280] In one embodiment, if the completed response case is a case regarding a temperature deviation of a drug subject to clinical trial, the processor (110) can generate a temperature deviation response package from the completed response case.

[0281] The above temperature deviation response package may include a temperature deviation code as an issue type code and a clinical trial drug management group code as a higher-level issue group code. The occurrence condition may be when the clinical trial drug is exposed to a temperature outside the permitted storage conditions.

[0282] Subsequent operational issues may include verifying temperature records, verifying acceptable storage conditions, calculating out-of-time, calculating cumulative out-of-time, verifying lot information, verifying whether it was before or after administration, verifying stability data, determining availability, determining the need for subject safety observation, and reporting and documentation.

[0283] The verification data may be temperature records, acceptable storage conditions, actual storage temperature, time of deviation, cumulative time of deviation, lot identifier, dosing records, and stability data actually obtained or reviewed in completed response cases. The processor (110) may generate verification required data for the temperature deviation response package based on the verification data. The verification required data may include temperature records, acceptable storage conditions, actual storage temperature, time of deviation, cumulative time of deviation, lot identifier, dosing records, and stability data that need to be verified in the future to apply the temperature deviation response package in a new clinical trial.

[0284] The sequential response procedure sequence may include securing temperature records, verifying acceptable storage conditions, calculating time out of bounds, calculating cumulative time out of bounds, isolating the relevant lot, verifying stability data, determining availability, and reporting and documentation. In the case of a post-administration condition, the sequential response procedure sequence may additionally include determining the necessity of subject safety observation and follow-up observation.

[0285] Information on reporting procedures may include the reporting subjects, reporting deadlines, reporting forms, reporting channels, and reporting requirements for at least one of the principal investigator, sponsor, drug management officer, safety officer, or regulatory authority. Documentation items may include at least one of a temperature deviation report, drug management record, monitoring record, or internal review record.

[0286] The source data location information may include storage condition provisions and temperature deviation response provisions of the Investigational Medicinal Product Management Manual, safety reporting provisions of the Clinical Trial Protocol, or deviation handling provisions of the Standard Operating Procedure. The source data type may be set to a type corresponding to the Investigational Medicinal Product Management Manual, the Clinical Trial Protocol, or the Standard Operating Procedure. The source data version information may include the version information of each document referenced when generating the corresponding package.

[0287] The processor (110) can generate a temperature deviation response package containing such items and store it in the response package database (170).

[0288] In another embodiment, if the completed response case is a case regarding a missed visit, the processor (110) may generate a missed visit response package. The missed visit response package may include verifying the allowed visit period, verifying whether a revisit is possible, verifying the missed evaluation items, reviewing the impact of key evaluation variables, determining whether there is a deviation from the clinical trial protocol, reporting, and documentation.

[0289] In another embodiment, if the completed response case is a case regarding a sample processing error, the processor (110) may generate a sample processing error response package. The sample processing error response package may include verifying sample processing standards, verifying actual processing time, verifying analyzability, verifying re-collectionability, reviewing the impact of key evaluation variables, reporting, and documentation.

[0290] In this way, the processor (110) can generate different response packages depending on the type of completed response case, and the generated response packages can be used to generate response measures for new clinical trial operation data.

[0291] According to FIG. 4, in one embodiment of the present invention, the processor (110) receives new clinical trial operation data (S400) and can generate a new operation profile from the new clinical trial operation data (S410). The processor (110) compares each of the plurality of corresponding packages stored in the corresponding package database (170) with the new operation profile (S420) and can calculate a corresponding package applicability score (S430).

[0292] The processor (110) can select a corresponding package as an applicable corresponding package if the corresponding package applicability score is greater than or equal to the applicable threshold value, the source data location information exists, and the source data version information corresponds to the current version (S440). Additionally, the processor (110) can derive an issue type corresponding to the selected corresponding package as an expected operational issue that may occur in a new clinical trial (S450).

[0293] The processor (110) determines the difference items between the new operation profile and the selected response package (S460), and can generate a corrected response package by correcting the difference items among the selected response packages (S470). Afterwards, the processor (110) generates a response plan based on the selected response package or the corrected response package (S480), and can output items requiring modification and items requiring expert review (S490).

[0294] The applicability score of the above response package may be calculated based on at least one of the issue type gateway score, problem situation match score, operating environment adjustment score, clinical trial target drug similarity score, and evidence reliability score.

[0295] The processor (110) can read new clinical trial operation data stored in the clinical trial operation data database (140) or receive new clinical trial operation data from a user terminal (190) or an external system.

[0296] The new clinical trial operational data may include at least one of the following: the clinical trial protocol, the investigational drug management manual, the management information for the investigational drug, the visit schedule, the administration schedule, the specimen management information, the role information by institution, and the documentation standards for the new clinical trial.

[0297] The clinical trial protocol may include subject selection criteria, exclusion criteria, dosing schedule, visit schedule, endpoints, safety endpoints, primary endpoints, discontinuation criteria, reporting criteria, protocol withdrawal handling criteria, and documentation criteria.

[0298] The investigational drug management manual may refer to documents regarding the storage, dispensing, dispensing, administration, retrieval, disposal, lot management, temperature record management, and response to temperature deviations of the investigational drug.

[0299] Clinical trial drug management information may include at least one of clinical trial drug identification information, candidate substance name, modality code, regulatory classification code, mechanism of action (MOA) code, target molecule identifier, direction of action code, site of action code, route of administration code, exposure characteristic code, storage conditions, preparation conditions, administration conditions, and lot information.

[0300] The processor (110) can analyze structured data, semi-structured data, or unstructured data included in the new clinical trial operation data. For example, the processor (110) can extract tables, items, clauses, schedules, visit-specific evaluation items, safety evaluation items, and reporting criteria from the clinical trial protocol. Additionally, the processor (110) can extract storage conditions, preparation conditions, temperature recording criteria, temperature deviation response criteria, and lot management criteria included in the investigational drug management manual.

[0301] The processor (110) can generate a new operational profile from new clinical trial operational data.

[0302] The new operation profile may be standardized data that allows the new clinical trial operation data to be compared with the corresponding package stored in the corresponding package database (170).

[0303] In one embodiment, the novel operational profile may include at least one of an issue type code, a parent issue group code, a drug management element, a dosing element, a visit element, a specimen management element, a reporting element, a documentation element, drug information, drug standardization code, modality code, regulatory classification code, mechanism of action (MOA) code, target molecule identifier, dosing status code, subject safety impact code, and a standard operational code comparable to a standard response case profile.

[0304] The management elements of the drug subject to clinical trial may include at least one of storage conditions, dispensing conditions, dispensing conditions, recall conditions, disposal conditions, lot management conditions, temperature recording standards, and temperature deviation response standards.

[0305] The administration elements may include at least one of the following: dosage, frequency of administration, timing of administration, administration interval, administration route, dosage adjustment criteria, discontinuation criteria, and pre-administration verification criteria.

[0306] Visit elements may include at least one of the following: visit name, time of visit, allowed visit period, evaluation items per visit, criteria for handling omission, and whether a revisit is possible.

[0307] Specimen management elements may include at least one of the following: type of specimen, time of collection, processing time, storage conditions, transportation conditions, analysis institution, criteria for determining the feasibility of analysis, and whether re-collection is possible.

[0308] The reporting element may include at least one of the reporting subject, reporting deadline, reporting format, reporting path, and reporting requirements. The above reporting element may be compared with reporting procedure information included in a completed response case or response package.

[0309] The documentation elements may include at least one of a dropout report, a monitoring report, a record of management of the drug subject to the clinical trial, a specimen handling record, a visit record, a safety review record, and an internal review record.

[0310] The processor (110) can convert natural language expressions included in the new clinical trial operational data into standard operational codes. For example, different expressions such as “visit allowance range,” “allowed visit period,” and “visit window” can be converted into the same standard operational code. Additionally, expressions such as “temperature deviation,” “storage condition deviation,” and “allowed storage temperature exceeded” can be converted into candidate issue type codes related to the management of the clinical trial drug.

[0311] The processor (110) can generate one or more candidate issue type codes using the clinical trial protocol provisions, investigational drug management manual provisions, dosing schedule, visit schedule, specimen management information, reporting deadline, documentation criteria, and investigational drug information included in the new clinical trial operation data.

[0312] Candidate issue type codes may represent types of operational issues that are likely to occur in new clinical trials. Candidate issue type codes may be generated prior to the calculation of the response package applicability score and may be confirmed as final anticipated operational issues or excluded based on the response package applicability score.

[0313] For example, if the new clinical trial operation data includes storage conditions, temperature recording standards, temperature deviation response standards, or stability data verification standards for the drug subject to the clinical trial, the processor (110) can generate candidate issue type codes regarding deviation from storage conditions or temperature deviation of the drug subject to the clinical trial.

[0314] As another example, if the new clinical trial operation data includes a visit schedule and an allowed visit period, the processor (110) can generate a candidate issue type code regarding a missed visit, a delayed visit, or an overdue allowed visit period.

[0315] As another example, if the new clinical trial operation data includes information on the time of sample collection, sample processing time, storage conditions, or analysis institution, the processor (110) can generate candidate issue type codes regarding sample processing errors, deviation from sample storage conditions, or sample transport errors.

[0316] As another example, if the new clinical trial operation data includes dosage, number of doses, interval between doses, route of administration, or discontinuation criteria, the processor (110) can generate candidate issue type codes regarding mis-administration, missed doses, delayed administration, or dosage errors.

[0317] The processor (110) can assign a parent issue group code to the generated candidate issue type code. The parent issue group code can correspond to any one of the management of clinical trial target drugs, administration management, visit management, specimen management, reporting management, documentation management, and clinical trial protocol deviation management.

[0318] The processor (110) can convert the new operation profile into a standard operation code that is comparable to a standard response case profile or response package.

[0319] The standard operating code may be a code that eliminates differences in the representation of different new clinical trial operating data and enables comparison with the corresponding package stored in the corresponding package database (170).

[0320] In one embodiment, the processor (110) can convert a phrase included in the new clinical trial operation data into a standard operation code by comparing it with a mapping rule stored in the rule database (160). In another embodiment, the processor (110) can use an artificial intelligence model (121) to analyze the context of the new clinical trial operation data and select a semantically corresponding standard operation code.

[0321] For example, expressions such as “pre-dose verification,” “pre-medication check,” “administration before check,” or “pre-dose verification” can be converted to the same pre-dose verification code. Expressions such as “sample centrifugation,” “sample processing,” or “sample processing” can be converted to the sample processing code.

[0322] The processor (110) can link source data location information, source data type, and source data version information to a new operational profile along with a standard operational code. For example, if a specific standard operational code is generated based on Section 5.2 of the clinical trial protocol or Section 3.1 of the investigational medicinal product management manual, the processor (110) can link the location information of the relevant section, the type of the relevant document, and the version information of the relevant document to a new operational profile.

[0323] When a new operation profile is created in this way, the processor (110) can calculate a corresponding package applicability score by comparing the new operation profile with a plurality of corresponding packages stored in the corresponding package database (170).

[0324] The response package applicability score may be a score indicating which of the multiple response packages stored in the response package database (170) is applicable to the new clinical trial operation data.

[0325] The processor (110) can calculate a response package applicability score based on at least one of the issue type match, occurrence stage match, affected target match, administration status match, operating environment match, clinical trial target drug similarity and evidence reliability between each of the new operation profile and the multiple response packages stored in the response package database (170).

[0326] The response package applicability score can be used as a criterion to determine whether a specific response package can be applied to new clinical trial operational data as is, after partial adjustments, or if expert review is required.

[0327] In one embodiment, the corresponding package applicability score may be calculated as a value between 0 and 1. In another embodiment, the corresponding package applicability score may be calculated as a value between 0 and 100.

[0328] The processor (110) can calculate an issue type gateway coefficient based on whether there is an issue type match between each of the multiple response packages stored in the new operation profile and the response package database (170).

[0329] The issue type gateway factor can be a gateway value that determines whether to maintain the response package as a target for applicability evaluation.

[0330] In one embodiment, the issue type gateway factor can be calculated as 1 when the candidate issue type code of the new operation profile and the issue type code of the corresponding package are the same.

[0331] If the candidate issue type code of the new operation profile and the issue type code of the response package are different but belong to the same top issue group, the issue type gateway factor can be calculated as 0.5.

[0332] If the candidate issue type code of the new operation profile and the issue type code of the response package are different and do not belong to the same top issue group, the issue type gateway coefficient may be calculated as 0.

[0333] For example, if the candidate issue type code of the new operation profile is temperature deviation and the issue type code of the response package is also temperature deviation, the issue type gateway factor can be calculated as 1.

[0334] If the candidate issue type code of a new operational profile is deviation from storage conditions and the issue type code of a response package is deviation from temperature, the two issue types may belong to the same higher-level issue group, "management of clinical trial drugs," even if they are not identical. In this case, the issue type gateway factor can be calculated as 0.5.

[0335] If the candidate issue type code of the new operation profile is Missed Visit and the issue type code of the response package is Temperature Out of Control, the two issue types are different and may not belong to the same parent issue group. In this case, the issue type gateway factor may be calculated as 0.

[0336] If the issue type gateway coefficient is 0, the processor (110) may not select the corresponding response package as an applicable response package. Additionally, the processor (110) may output the corresponding response package or at least some items included in the corresponding response package as items requiring expert review.

[0337] The processor (110) can calculate a problem situation match score based on at least one of the occurrence stage match, affected target match, medication status match, and subject safety effect match between each of the new operation profile and the plurality of response packages stored in the response package database (170).

[0338] Occurrence stage match can indicate the extent to which the issue occurrence stages included in the new operational profile match the occurrence stages included in the response package. For example, the screening stage, pre-treatment stage, in-treatment stage, post-treatment stage, follow-up stage, or termination stage may be used as occurrence stages.

[0339] The degree of agreement of affected subjects may indicate the extent to which the affected subjects of the new operational profile and the affected subjects of the response package match. The affected subjects may be any one of the investigational medicinal products, lots, subjects, institutions, visits, specimens, documents, or reporting procedures.

[0340] The dosing status concordance may indicate the extent to which the dosing status of the new operational profile matches the dosing status of the corresponding package. The dosing status may include any one of pre-dosing, during dosing, post-dosing, dosing hold, dosing discontinued, or dosing completed.

[0341] Subject safety impact consistency may indicate the degree of agreement between the subject safety impact information in the new operational profile and the subject safety impact information in the response package. The subject safety impact information may include any one of the following: no safety impact, potential safety impact, safety impact verification required, occurrence of an adverse event, or occurrence of a serious adverse event.

[0342] In one embodiment, the problem situation consistency score can be calculated as a sum of weights applied to the occurrence stage consistency, affected target consistency, medication status consistency, and subject safety effect consistency, respectively.

[0343] In another embodiment, the problem situation match score may be calculated by applying a preset deduction value when an essential item among the above items is inconsistent. For example, if a response package applicable only to the pre-medication state is compared with a new operation profile of the post-medication state, the processor (110) may apply a deduction value due to the medication state inconsistency.

[0344] The processor (110) can calculate an operating environment correction score based on whether there is a match between at least one of the clinical trial institution role, clinical trial target drug management conditions, lot range, visit schedule, specimen management conditions, reporting deadline, and documentation criteria between each of the new operating profile and the multiple response packages stored in the response package database (170).

[0345] The operating environment adjustment score may be a score indicating the degree of similarity between the operating environment of the completed response case where the response package was generated and the operating environment of the new clinical trial.

[0346] The role of a clinical trial institution may indicate whether the institution performs the role of a testing institution, central testing institution, analysis institution, drug storage institution, dispensing institution, or administration institution.

[0347] The management conditions for the drug subject to clinical trial may include at least one of storage conditions, dispensing conditions, dispensing conditions, recall conditions, disposal conditions, temperature recording standards, or lot management standards.

[0348] Lot scope may indicate whether the operational issue is limited to a specific lot, affects multiple lots, or affects the entire investigational drug.

[0349] The visit schedule may include the visit time, permitted visit period, visit interval, and evaluation items for each visit. Specimen management conditions may include the specimen collection time, processing time, storage conditions, transportation conditions, and criteria for determining analyzability.

[0350] The reporting deadline can indicate the time standard for when a report must be made after an operational issue occurs. The documentation standard can indicate the types, items, and deadlines for the documents required to record the operational issue.

[0351] The higher the operating environment correction score, the more the processor (110) can determine that the corresponding response package is more suitable for new clinical trial operating data.

[0352] The processor (110) can calculate a clinical trial drug similarity score based on at least one of the modality code match, regulatory classification code match, mechanism of action (MOA) code match, target molecule match score, direction of action match score, route of administration match score, and exposure characteristic match score between the clinical trial drug information included in the new operation profile and the clinical trial drug information included in the corresponding package.

[0353] Modality code concordance can indicate whether the modalities of the investigational drug in a novel clinical trial and the investigational drug corresponding to the corresponding package are identical or similar. For example, modalities such as antibody drugs, gene therapies, cell therapies, protein therapies, synthetic drugs, RNA therapies, or vaccines may be compared.

[0354] Regulatory classification code concordance can indicate whether the regulatory classification of the investigational drug in a new clinical trial and the investigational drug corresponding to the corresponding package are identical or similar.

[0355] Mechanism of Action (MOA) code concordance can indicate whether the mechanism of action (MOA) of the investigational drug in a novel clinical trial is identical or similar to that of the investigational drug corresponding to the corresponding package.

[0356] Target molecule correspondence scores can indicate whether target molecule identifiers, signaling pathway identifiers, or target series are identical or semantically corresponding.

[0357] The direction of action correspondence score can indicate whether the direction of action, such as inhibition, activation, supplementation, substitution, elimination, or regulation, is the same or corresponding.

[0358] The administration route correspondence score may indicate whether oral, intravenous, subcutaneous, intramuscular, topical, inhalation, ocular, intrathecal, or other administration routes are identical or corresponding.

[0359] The exposure characteristic match score can indicate whether exposure characteristics, such as single dose, repeated dose, continuous exposure, local exposure, systemic exposure, or long-term exposure, are the same or similar.

[0360] In one embodiment, the clinical trial drug similarity score can be calculated by applying weights to the modality code match, regulatory classification code match, mechanism of action (MOA) code match, target molecule match score, direction of action match score, route of administration match score, and exposure characteristic match score, respectively.

[0361] The processor (110) can calculate a reliability score based on at least one of the existence of source material location information included in the corresponding package, the type of source material, whether the source material version information corresponds to the current version, and whether there is a correspondence between the source sentence extracted from the source material location information and the item included in the corresponding package.

[0362] If the source data location information exists, the processor (110) can assign a higher source reliability score than if the source data location information does not exist.

[0363] If the type of evidence corresponds to regulatory agency documents, approved product labels, investigator's manuals, clinical trial results documents, safety data, clinical trial protocol provisions, investigational drug management manual provisions, or standard operating procedures, the processor (110) may assign a relatively high evidence reliability score.

[0364] On the other hand, if the type of supporting material corresponds to an internal review record, a completed response case, a company announcement, a press release about discontinued development, or a secondary summary of an unknown source, the processor (110) may assign a relatively low reliability score to the material or mark it as an item requiring expert review.

[0365] If the source data version information corresponds to the current version applied to the new clinical trial operational data, the processor (110) may maintain or increase the source reliability score. Here, the current version may refer to the approved version, implementation version, or valid version of the clinical trial protocol, investigational medicinal product management manual, standard operating procedure, clinical trial-related regulations, guidelines, or Q&A data actually applied to the new clinical trial operational data.

[0366] On the other hand, if the source data version information does not correspond to the current version, the processor (110) may lower the source reliability score or not select the corresponding response package as an applicable response package.

[0367] Additionally, the processor (110) can extract a source sentence from source data location information and determine whether there is a correspondence between the source sentence and an item included in a corresponding package. If a term, numerical value, condition, reporting target, reporting deadline, reporting form, reporting path, or documentation standard included in the source sentence matches an item in the corresponding package, the processor (110) can calculate a high source reliability score.

[0368] In one embodiment, the applicability score P of the corresponding package can be calculated by the following mathematical formula.

[0369] P = G × (w1 × C + w2 × O + w3 × D + w4 × R)

[0370] Here, G is the issue type gateway factor, C is the problem situation consistency score, O is the operating environment adjustment score, D is the clinical trial target drug similarity score, and R is the evidence reliability score. w1 through w4 may be weights applied to each score.

[0371] In one embodiment, the sum of w1 to w4 may be 1. In another embodiment, the sum of w1 to w4 may not be 1, and the corresponding package applicability score may be converted to a preset range through a normalization procedure.

[0372] The processor (110) can determine w1 to w4 using a set of weights stored in the rule database (160). For example, in the issue of managing a drug subject to clinical trials, the weights for the drug subject to clinical trials similarity score and the evidence reliability score may be set relatively high. In the issue of visit management, the weights for the occurrence stage consistency and the operating environment correction score may be set relatively high. In the issue of specimen management, the weight of the operating environment correction score may be set high to reflect the influence of specimen management conditions and key evaluation variables.

[0373] If the issue type gateway factor G is 0, the response package applicability score P can be calculated as 0. In this case, the processor (110) may not select the corresponding response package as an applicable response package.

[0374] 7.8. Criteria for Selecting a Response Package

[0375] The processor (110) can determine that a corresponding package is a candidate for a corresponding package applicable to new clinical trial operational data if the applicability score of the corresponding package is greater than or equal to a preset applicability threshold value, the location information of the supporting data included in the corresponding package exists, and the version information of the supporting data included in the corresponding package corresponds to the current version.

[0376] Applicable threshold values ​​can be stored in a rule database (160). Applicable threshold values ​​may be set differently depending on the issue type, top issue group, clinical trial stage, type of evidence, subject safety impact information, or user settings.

[0377] For example, applicable threshold values ​​may be set high for issues with a high potential impact on subject safety. Conversely, applicable threshold values ​​may be set low for issues with relatively low risk, such as supplementing documentation items.

[0378] The criteria for selecting a response package in this section may serve as criteria for determining a response package as an applicable candidate. The specific process of deriving anticipated operational issues and generating response measures using the selected response package is explained in the response measure generation example described later.

[0379] The processor (110) may not select the corresponding package as an applicable corresponding package if the applicability score of the corresponding package is less than the applicable threshold value.

[0380] Additionally, if the processor (110) does not have location information for supporting data included in the corresponding package, it may not select the corresponding package as an applicable corresponding package, or may mark the corresponding package or at least some items included in the corresponding package as items requiring expert review.

[0381] Additionally, if the version information of the supporting data included in the corresponding package does not correspond to the current version, the processor (110) may not select the corresponding package as an applicable corresponding package, or may mark the corresponding package or at least some of the items included in the corresponding package as items requiring expert review.

[0382] Additionally, if the processor (110) does not confirm a correspondence between the source sentence extracted from the source data location information and the item included in the corresponding package, it may mark the item as an item requiring expert review.

[0383] For example, if the reporting deadline of a response package is set based on a past version of the standard operating procedure and the reporting deadline has been changed in the current standard operating procedure applied to the new clinical trial operating data, the processor (110) may mark the corresponding reporting deadline item as an item requiring expert review.

[0384] The indication of items requiring expert review in this section may be intended to display the reasons for restricting automatic application during the response package selection or exclusion stage. The form in which items requiring expert review are provided to the user when outputting response plans is described in the response plan generation example described later.

[0385] The processor (110) can use the response package determined according to the response package applicability score and selection criteria to derive expected operational issues and generate response measures for new clinical trial operational data.

[0386] In one embodiment, the processor (110) may select and use one response package for one new operation profile. In another embodiment, the processor (110) may select and use multiple response packages for one new operation profile. For example, if a temperature deviation issue is associated with multiple response packages depending on the administration status, lot range, and subject safety impact, the processor (110) may use multiple response packages together.

[0387] In another embodiment, the processor (110) may use the corresponding package with the highest corresponding package applicability score among a plurality of corresponding packages corresponding to the same issue type code as the representative corresponding package, and output the remaining corresponding packages as reference corresponding packages.

[0388] The processor (110) can read out the issue type code, parent issue group code, occurrence condition, subsequent operational issue, materials requiring verification, response requirements, sequential response procedure sequence, reporting procedure information, documentation items, recurrence prevention measures, follow-up observation items, source material location information, source material type and source material version information included in the selected response package.

[0389] The processor (110) can derive the type of issue corresponding to the selected response package as an expected operational issue that may occur in a new clinical trial.

[0390] Anticipated operational issues may be operational issues that are likely to occur in light of the operating conditions included in the new clinical trial operational data, or that require follow-up verification or response if they occur.

[0391] For example, if the new clinical trial operation data includes a clinical trial drug requiring refrigerated storage, temperature recording standards, temperature deviation response standards, and lot management standards, the processor (110) can identify a temperature deviation or storage condition deviation as an expected operation issue.

[0392] As another example, if the new clinical trial operation data includes a strict visit allowance period and visit evaluation items linked to key evaluation variables, the processor (110) can identify a missed visit or a visit delay as an expected operation issue.

[0393] As another example, if the new clinical trial operation data includes sample processing time, storage conditions, and analysis institution information, and the applicability score of the sample processing error response package is greater than or equal to the applicable threshold value, the processor (110) can identify the sample processing error as an expected operation issue.

[0394] The processor (110) can connect selected response packages or correction response packages for each expected operational issue and generate a response plan corresponding to each expected operational issue.

[0395] The processor (110) can determine the difference items between the new operation profile and the selected corresponding package.

[0396] The above differences may include at least one of the following: differences in the protocol, differences in the investigational drug management manual, changes in regulations, differences in the version of the supporting data, differences in the administration status, differences in subject safety effects, or differences in the investigational drug.

[0397] Protocol differences may occur when the procedures, reporting subjects, reporting procedure information, documentation items, or follow-up items included in the selected response package differ from the provisions of the new clinical trial protocol.

[0398] Differences in the investigational medicinal product management manual may occur when the storage conditions, dispensing conditions, temperature recording standards, lot management standards, or temperature deviation response standards included in the selected response package differ from the provisions of the new investigational medicinal product management manual.

[0399] Changes in regulations may occur if clinical trial-related regulations, guidelines, Q&A materials, or standard operating procedures referenced at the time of creating the response package are revised or changed.

[0400] Differences in source data versions may occur when the source data version information included in the response package does not match the current version applied to the new clinical trial operation data.

[0401] Differences in dosing status may occur when the underlying dosing status of the corresponding package differs from that of the new clinical trial, such as when the corresponding package was generated based on the pre-dosing status but the post-dosing status is confirmed in the new operational profile.

[0402] Differences in subject safety impact may occur when a response package is generated under the premise of no or low subject safety impact, but a potential subject safety impact is identified in a new operational profile.

[0403] Differences in the investigational drug may occur when a corresponding package was created based on specific modalities, mechanisms of action (MOA), routes of administration, exposure characteristics, or storage conditions, but the information on the investigational drug for a new clinical trial differs from this.

[0404] The processor (110) can generate a corrected response package by correcting items that differ from the new clinical trial operation data among the selected response packages.

[0405] The corrected response package may be a response package that has been modified from the selected response package to align with the new clinical trial operational data. The corrected response package may be one in which at least one of the following has been corrected: the issue type code, parent issue group code, occurrence condition, subsequent operational issue, data requiring verification, response requirements, sequential response procedure sequence, reporting procedure information, documentation items, preventive measures, follow-up items, source data location information, source data type, or source data version information of the selected response package.

[0406] For example, if the reporting target for a temperature deviation in a selected response package is set to the person in charge of managing the clinical trial drug, but the new clinical trial protocol stipulates that it should also be reported to the clinical trial director and safety officer, the processor (110) can correct the reporting target item.

[0407] As another example, in the selected response package, the reporting deadline is set to within 24 hours, but in the current version of the standard operating procedure applied to the new clinical trial, the reporting deadline is changed to immediate reporting, and the processor (110) can correct the reporting deadline item in the reporting procedure information.

[0408] As another example, in the case where the documentation item in the selected response package is set only to the temperature deviation report, but the new clinical trial operation data specifies that it should also be recorded in the monitoring report and internal review record, the processor (110) can correct the documentation item.

[0409] As another example, if a selected corresponding package is generated based on a clinical trial drug having a specific mechanism of action (MOA) or route of administration, but the clinical trial drug of a new clinical trial is different from this, the processor (110) can determine whether to correct the item based on the clinical trial drug similarity score and evidence reliability score.

[0410] The processor (110) can generate a response plan for new clinical trial operation data based on a selected response package or a correction response package.

[0411] The response plan may include at least one of the following: materials requiring verification, initial measures, reporting subjects, information on reporting procedures, documentation items, measures to prevent recurrence, follow-up observation items, items requiring correction, and items requiring expert review.

[0412] Data requiring verification may be data that must be verified before implementing countermeasures applied to new clinical trial operational data. For example, data requiring verification for temperature deviation countermeasures may include temperature records, acceptable storage conditions, actual time of deviation, cumulative time of deviation, lot information, administration status, and stability data.

[0413] Initial measures may be actions that must be performed first when operational issues occur or are anticipated. For example, in the case of a deviation from storage conditions of a drug subject to a clinical trial, initial measures may include isolating the lot, withholding use, withholding additional dosing, or requesting a stability review.

[0414] Information on reporting subjects and reporting procedures may indicate to whom, when, and by what form and channel operational issues or response results must be reported. Reporting subjects may include at least one of the following: the principal investigator, sponsor, institution, safety officer, drug administrator, monitor, data administrator, or regulatory authority. Information on reporting procedures may include reporting deadlines, reporting forms, reporting channels, and reporting requirements.

[0415] Documentation items may be documents or items in which the results of the implementation of response measures are to be recorded. Documentation items may include at least one of a dropout report, a monitoring report, a record of management of the drug subject to clinical trial, a specimen handling record, a visit record, a safety review record, or an internal review record.

[0416] Preventive measures may be actions intended to prevent the same or similar operational issues from recurring. For example, preventive measures may include training, adding checklists, revising standard operating procedures, strengthening temperature monitoring, supplementing visit notification procedures, or supplementing specimen handling procedures.

[0417] Follow-up items may be items that must be observed for a certain period to determine the impact of operational issues on subject safety, primary endpoints, the quality of the drug subject to the clinical trial, or the reliability of the clinical trial results.

[0418] The processor (110) can apply the selected response package to the new clinical trial operation data because the applicability score of the response package is greater than or equal to the applicable threshold value, but if there is a difference between the new operation profile and the selected response package, the item corresponding to the difference among the items included in the response plan can be output as an item requiring modification.

[0419] For example, if the reporting target of the selected response package differs in some way from the reporting target of the new clinical trial protocol, the processor (110) may output the reporting target item as an item requiring modification.

[0420] As another example, if the reporting deadline, reporting form, or reporting path of the selected response package differs from the current version of the standard operating procedure applied to the new clinical trial, the processor (110) may output the different items among the reporting procedure information as items requiring correction.

[0421] As another example, if the documentation items of the selected corresponding package differ from the documentation criteria of the new investigational drug management manual or standard operating procedure, the processor (110) may output the documentation items as items requiring modification.

[0422] As another example, if the follow-up observation item of the selected response package does not sufficiently correspond to the safety evaluation item of the new clinical trial, the processor (110) may output the follow-up observation item as an item requiring modification.

[0423] Items requiring modification can be used to generate a correction response package. The processor (110) can output the content before modification, the content after modification, the reason for modification, the location information of related supporting materials, the type of supporting materials, and the version information of supporting materials together for each item requiring modification.

[0424] If the processor (110) determines that the applicability score of a corresponding package is less than the applicable threshold value, it may not select the corresponding package as an applicable corresponding package, and may output the corresponding corresponding package or at least some items included in the corresponding corresponding package as items requiring expert review.

[0425] Additionally, if the processor (110) does not have location information for supporting data included in the corresponding package, it may output the corresponding corresponding package or the corresponding item as an item requiring expert review.

[0426] Additionally, if the version information of the supporting data included in the corresponding package does not correspond to the current version, the processor (110) may output the corresponding corresponding package or the corresponding item as an item requiring expert review.

[0427] Additionally, if the processor (110) does not confirm a correspondence between the source sentence extracted from the source data location information and the item included in the response plan, the item may be output as an item requiring expert review.

[0428] Items requiring expert review can be output along with countermeasures through a user terminal (190). In one embodiment, items requiring expert review may include an issue type, a reason for the need for review, a related response package, location information of supporting materials, type of supporting materials, version information of supporting materials, inconsistency items, and recommended review content.

[0429] For example, if the applicability score of the temperature deviation response package is above the threshold value but stability data does not exist, the processor (110) may output “assessment of usability due to absence of stability data” as an item requiring expert review.

[0430] As another example, in a similar completed response case, only temperature deviation before administration is dealt with, but in a new clinical trial, if the condition after administration has already been confirmed, the processor (110) can output “necessity for observing subject safety after administration” as an item requiring expert review.

[0431] The processor (110) can output the generated response plan to a user terminal (190), a management server, a clinical trial operating system, or a document generation system.

[0432] The response plan can be output in the form of a table, list, checklist, flowchart, report, electronic document, or screen display data.

[0433] In one embodiment, response measures may be output for each anticipated operational issue. For each anticipated operational issue, the processor (110) may output together an applied response package, a response package applicability score, materials requiring verification, initial measures, reporting targets, reporting procedure information, documentation items, recurrence prevention measures, follow-up observation items, items requiring modification, and items requiring expert review.

[0434] In another embodiment, response measures may be output separately for each person in charge. For example, items such as checking storage conditions, lot isolation, and checking temperature records may be output to the person in charge of managing the clinical trial drug, items such as observing and reporting subject safety may be output to the person in charge of the clinical trial, and items such as writing dropout reports and monitoring records may be output to the person in charge of documentation.

[0435] In another embodiment, the response plan may be output along with source data location information, source data type, and source data version information. In this case, the user can verify which clinical trial protocol provision, investigational medicinal product management manual provision, standard operating procedure provision, guideline, or internal review record each item of the response plan is based on.

[0436] According to FIG. 5, in one embodiment of the present invention, the processor (110) may receive a new completed response case or a revised / changed response basis data (S500). The revised / changed response basis data may include at least one of a revised clinical trial-related regulation, a revised guideline, a revised Q&A data, a revised standard operating procedure, a revised clinical trial protocol, or a revised investigational drug management manual.

[0437] The processor (110) identifies a response package to be updated related to a new completed response case or a revised / changed response basis data (S510), and can compare the new completed response case with the response package to be updated (S520). Additionally, the processor (110) can compare the revised / changed response basis data with the source data location information and source data version information of the response package to be updated (S530).

[0438] The processor (110) can determine whether at least one of the occurrence conditions, subsequent operational issues, materials requiring verification, response requirements, reporting targets, documentation items, sequential response procedure sequences, recurrence prevention measures, and follow-up observation items included in the response package to be updated needs to be changed (S540). If there are items that need to be changed, the processor (110) can create a response package to be updated (S550) and create a package change history (S560). Subsequently, the processor (110) can store the response package to be updated and the package change history in the response package database (170) (S570).

[0439] The package change history may include at least one of the version before the update, the version after the update, identification information of the basis for the revised / changed response, the changed item, the reason for the change, and the date and time of the update.

[0440] The processor (110) can perform a response package update procedure when at least one of a new completed response case, a revised clinical trial-related regulation, a revised guideline, a changed Q&A material, a changed standard operating procedure, a changed clinical trial protocol, or a changed investigational drug management manual is entered.

[0441] New completed response cases may be clinical trial operational issue response cases that occur or are processed additionally after the creation of an existing response package. For example, a new temperature deviation case, a new missed visit case, a new specimen handling error case, or a new medication error case may be entered as a new completed response case.

[0442] Revised clinical trial regulations or revised guidelines may affect the reporting subjects, reporting deadlines, reporting forms, reporting routes, documentation items, safety observation items, or follow-up items included in the response package.

[0443] Modified standard operating procedures, modified clinical trial protocols, or modified investigational medicinal product management manuals may change response procedures, reporting procedure information, or documentation standards applicable to a specific clinical trial or a specific institution.

[0444] When the update input occurs, the processor (110) can identify the update target corresponding package among a plurality of corresponding packages stored in the corresponding package database (170).

[0445] The processor (110) can identify a response package to be updated related to a new completed response case or a revised / changed response basis data.

[0446] In one embodiment, the processor (110) can identify a response package to be updated by comparing the issue type code, parent issue group code, clinical trial target drug information, occurrence stage, affected target, administration status, subject safety impact information, or reporting procedure information included in the new completed response case with the response package stored in the response package database (170).

[0447] For example, if the new completed response case is a case regarding temperature deviation, the processor (110) can identify a response package for temperature deviation, a response package for storage condition deviation, or a response package belonging to a clinical trial drug management group in the response package database (170) as a response package to be updated.

[0448] In another embodiment, the processor (110) can identify the response package to be updated by comparing the document identifier, clause number, version information, or item name of the revised / changed response basis material with the source material location information and source material version information included in the response package.

[0449] For example, if the temperature deviation response clause of the investigational drug management manual is changed, the processor (110) can identify a temperature deviation response package that includes the said clause as source data location information as a response package to be updated.

[0450] In another embodiment, if the reporting deadline clause or reporting form clause of the standard operating procedure is changed, the processor (110) can identify a plurality of corresponding packages that refer to the relevant reporting procedure information as corresponding packages to be updated.

[0451] The processor (110) can compare a new completed response case with at least one of the issue type, top issue group, occurrence condition, subsequent operation issue, verification material, verification required material, response required condition, sequential response procedure sequence, reporting procedure information and action result included in the response package to be updated.

[0452] In one embodiment, the processor (110) can determine whether a subsequent operational issue not included in the existing response package is identified in a new completed response case. For example, if the existing temperature deviation response package did not include a subject safety observation item, but in a new completed response case, subject safety observation is performed due to a temperature deviation after medication, the processor (110) can identify the subject safety observation item as an item requiring update.

[0453] In another embodiment, the processor (110) can determine whether a response procedure different from the existing response package was performed in a new completed response case. For example, in the existing response package, only internal reporting was to be performed when a temperature deviation occurred, but in a new completed response case, the clinical trial manager and safety officer were also reported, the processor (110) can determine whether there is a need to change the information regarding the reporting subjects or reporting procedures.

[0454] In another embodiment, the processor (110) can determine whether the response procedure of the existing response package was appropriate by analyzing the result of the action taken in a new completed response case. For example, if the same issue occurs repeatedly despite taking action according to the existing response procedure, the processor (110) can determine the need to supplement the recurrence prevention measures.

[0455] The processor (110) can store the result of the comparison between the new completed response case and the update target response package as update candidate information.

[0456] The processor (110) can compare the revised clinical trial-related regulations, revised guidelines, changed Q&A materials, changed standard operating procedures, changed clinical trial protocols, or changed investigational medicinal product management manuals with at least one of the source data location information and source data version information included in the update target response package.

[0457] In one embodiment, the processor (110) can compare the document identifier of the revised / changed response basis material with the document identifier of the basis material location information included in the response package to be updated. If the document identifiers match, the processor (110) can determine which item of the response package the changed clause, changed paragraph, or changed item of the document affects.

[0458] In another embodiment, the processor (110) can compare the version information of the revised / changed response basis data with the version information of the basis data included in the response package to be updated. If the version information of the basis data of the response package does not correspond to the current version, the processor (110) can determine the corresponding response package as the response package to be updated.

[0459] In another embodiment, the processor (110) can determine whether there is a semantic change by comparing the basis sentence before the change with the basis sentence after the change. For example, if the reporting deadline, reporting subject, reporting form, reporting path, documentation item, safety observation item, or follow-up item is changed, the processor (110) can determine that the change affects the response package item.

[0460] The processor (110) can store the result of comparing the revised / changed response basis data and the update target response package as basis change information.

[0461] The processor (110) can determine whether at least one of the occurrence conditions, subsequent operational issues, materials requiring verification, response requirements, reporting targets, reporting procedure information, documentation items, sequential response procedure sequence, recurrence prevention measures, and follow-up observation items included in the response package to be updated needs to be changed, based on the results of comparison with the new completed response case and the results of comparison with the revised / changed response basis data.

[0462] In one embodiment, if a new subsequent operation issue that is not included in an existing response package is repeatedly identified, the processor (110) may determine that it is necessary to add the subsequent operation issue to the response package.

[0463] In another embodiment, the processor (110) may determine that it is necessary to change the reporting procedure information of the response package if the reporting target, reporting deadline, reporting form, or reporting path is changed by the revised / changed response basis data.

[0464] In another embodiment, the processor (110) may determine that it is necessary to change the sequential response procedure sequence if the result of the action of a new completed response case differs from the sequential response procedure sequence of the existing response package and the different procedure is associated with a more appropriate result of the action.

[0465] In another embodiment, the processor (110) may determine that it is necessary to change the source data location information or the corresponding item if the source data location information of the existing corresponding package no longer exists or if the source data version information does not correspond to the current version.

[0466] If there are items that need to be changed, the processor (110) can update the update target corresponding package to create an update corresponding package.

[0467] The update response package may be a response package in which some items of the response package prior to the update have been changed, added, deleted, or rearranged.

[0468] In one embodiment, the processor (110) may add subsequent operational issues identified in a new completed response case to an existing response package. For example, if an item “determination of necessity for subject safety observation” not included in the existing response package is identified in a new completed response case, the processor (110) may add the item to subsequent operational issues and sequential response procedure sequences.

[0469] In another embodiment, the processor (110) may change the reporting subject, reporting deadline, reporting form, reporting path, or documentation item according to the revised / changed response basis data.

[0470] In another embodiment, the processor (110) can reflect the source data location information, source data type, and source data version information of the revised / changed response source data in the update response package.

[0471] After the processor (110) creates an update response package, it can assign new version information to the update response package. Additionally, the processor (110) can store the creation date and time or the update date and time of the update response package.

[0472] The processor (110) can store a package change history for the update response package in the response package database (170), which includes at least one of the pre-update version, post-update version, identification information of the revised / changed response basis data, changed items, reason for change, and update date and time.

[0473] The version prior to the update can display the version information of the corresponding package prior to the change. The version after the update can display the version information of the updated corresponding package.

[0474] Identification information of revised / changed response basis materials may include information identifying clinical trial-related regulations, guidelines, Q&A materials, standard operating procedures, clinical trial protocols, investigational medicinal product management manuals, or new completed response cases that caused the update of the response package.

[0475] Changed items may refer to items within the response package subject to update that have been changed, added, deleted, or rearranged. Reasons for change may include reflecting new completed response cases, revisions to regulations or guidelines, changes to standard operating procedures, changes to clinical trial protocols, changes to the investigational medicinal product management manual, or updates to supporting data versions.

[0476] The update date may indicate the time when the update response package is created or stored in the response package database (170).

[0477] By storing the package change history, the system (100) can track the changes to the response package. Additionally, the user can check which version of the response package a specific response plan was created based on, which revised / changed response basis data the response package was changed according to, and which items were changed.

[0478] After the update response package is stored in the response package database (170), the processor (110) can use the update response package when calculating the score of applicability of the response package to new clinical trial operation data.

[0479] In one embodiment, the processor (110) stores both the existing response package and the updated response package, but may set the latest version of the updated response package as the default search target.

[0480] In another embodiment, the processor (110) may select either an existing response package or an updated response package according to the source data version information applied to specific new clinical trial operation data. For example, if a specific clinical trial is conducted according to a past version of the clinical trial protocol or the investigational medicinal product management manual, the processor (110) may select a response package corresponding to that version.

[0481] In another embodiment, the processor (110) may reflect only the items among the change items of the update response package that affect the new clinical trial operation data in the creation of the correction response package.

[0482] In this way, through the response package update procedure, the system (100) can maintain the up-to-dateness and traceability of the response package by reflecting new completed response cases and revised / changed response basis data.

[0483] In one embodiment, the system (100) may further include a development case database (180) that stores a plurality of development cases.

[0484] The development case database (180) can be distinguished from the completed response case database (130), which stores completed response cases that occurred during the operation of the clinical trial. The completed response case database (130) stores response cases for clinical trial operation issues such as temperature deviation, missed visits, sample handling errors, medication errors, reporting delays, missing documentation, or deviations from the clinical trial protocol, whereas the development case database (180) can store development cases related to the drug or candidate substance subject to the clinical trial.

[0485] The development case database (180) may be an optional configuration of the system (100) illustrated in FIG. 1. The development case database (180) and the operational risk reflection configuration of claim 4 may not be illustrated in a separate drawing, and specific processing details regarding the development case-based operational risk classification and reflection of countermeasures may be described in sentences below.

[0486] The development case database (180) may store a development case profile for each development case. The development case profile may include at least one of a modality code, a regulatory classification code, a mechanism of action (MOA) code, a target molecule identifier, a signaling pathway identifier, a direction of action code, a site of action code, a route of administration code, an exposure characteristic code, development stage information, safety event information, toxicity event information, information on the reason for development discontinuation, information on the reason for clinical hold, source data location information, source data type, and source data version information.

[0487] Development cases may include any one of the following: approved product cases, clinical trial progress cases, clinical trial result cases, development discontinuation cases, clinical hold cases, safety cases listed on the approved product label, similar mechanism cases listed in the investigator's data booklet, or review cases listed in regulatory agency documents.

[0488] Safety event information may include information regarding adverse events, serious adverse events, death, life-threatening events, hospitalization, discontinuation of medication, dose reduction, dose adjustment, discontinuation of treatment, or adverse events of special interest.

[0489] Toxic event information may include information regarding non-clinical toxicity, clinical toxicity, organ toxicity, immunotoxicity, genotoxicity, reproductive toxicity, local toxicity, systemic toxicity, or unexpected toxic reactions.

[0490] Information on the reasons for discontinuation of development may include insufficient efficacy, safety issues, toxicity issues, clinical hold, requests for supplementary information from regulatory authorities, route of administration issues, dosage setting issues, manufacturing or quality issues, lack of commercial viability, or other reasons for discontinuation.

[0491] Information on reasons for clinical hold may include a hold by a regulatory agency, insufficient safety data, insufficient non-clinical data, insufficient manufacturing quality data, the need to supplement the clinical trial plan, insufficient criteria for discontinuation of treatment, or insufficient subject protection measures.

[0492] The processor (110) can generate a candidate substance profile based on information on the target drug for clinical trials included in the new clinical trial operation data.

[0493] A candidate substance profile may be data representing the characteristics of the investigational drug or candidate substance of a novel clinical trial. The candidate substance profile may include at least one of the following: a candidate substance name, investigational drug identification information, a modality code, a regulatory classification code, a mechanism of action (MOA) code, a target molecule identifier, a signaling pathway identifier, a direction of action code, a site of action code, a route of administration code, an exposure characteristic code, storage conditions, preparation conditions, and administration conditions.

[0494] The processor (110) can generate a candidate substance profile from a clinical trial protocol, a manual for managing investigational drugs, an investigator's manual, safety data, or candidate substance description data.

[0495] For example, if the new clinical trial operation data contains information that a specific candidate substance is an antibody drug, inhibits a specific target molecule, is administered intravenously, and has systemic exposure characteristics, the processor (110) can use that information to generate a candidate substance profile.

[0496] As another example, if the new clinical trial operation data contains information that a specific candidate substance is a cell therapy, is administered locally to a specific site of action, and has a mechanism of action (MOA) related to an immune response, the processor (110) can standardize the information to generate a candidate substance profile.

[0497] The processor (110) can generate a standardized profile by converting the candidate substance profile into the same standard tag system as the development case profile.

[0498] A standardized profile may be a data structure designed to enable comparison between a candidate profile and a development case profile. The standardized profile may include modality codes, regulatory classification codes, mechanism of action (MOA) codes, target molecule identifiers, signaling pathway identifiers, direction of action codes, site of action codes, route of administration codes, and exposure characteristic codes.

[0499] In one embodiment, the processor (110) can convert natural language expressions included in the candidate substance profile into standard tags. For example, expressions such as “antibody,” “monoclonal antibody,” “mAb,” or “antibody drug” can be converted into the same modality code.

[0500] In another embodiment, the processor (110) can convert expressions related to a mechanism of action (MOA) into standard mechanism of action (MOA) codes. For example, expressions such as “inhibition of target molecule A,” “A inhibitor,” and “A signaling pathway blockade” can be converted into identical or corresponding mechanism of action (MOA) codes and direction of action codes.

[0501] In another embodiment, the processor (110) can convert administration routes and exposure characteristics into standard codes. For example, “IV,” “intravenous injection,” and “intravenous administration” can be converted into intravenous administration codes, and “local injection,” and “local administration” can be converted into local administration codes.

[0502] The processor (110) can calculate a case similarity score by comparing the standardization profile with the development case profile stored in the development case database (180).

[0503] The processor (110) can calculate the modality gateway coefficient, mechanism agreement score, application environment correction score and evidence availability correction score based on whether there is a semantic correspondence between the standardization profile and the development case profile.

[0504] The modality gateway factor may be a coefficient indicating whether the modality of a candidate substance and the modality of a development case are identical or similar. For example, if the candidate substance and the development case belong to the same modality, the modality gateway factor may be calculated as 1. If the candidate substance and the development case belong to different modalities but have the same regulatory classification or similar functional characteristics, the modality gateway factor may be calculated as a value greater than 0 and less than 1. If the candidate substance and the development case belong to different modalities and have low semantic correspondence, the modality gateway factor may be calculated as 0.

[0505] The mechanism agreement score may be a score indicating the extent to which the mechanism of action (MOA), target molecule, signaling pathway, and direction of action of a candidate substance match the mechanism of action (MOA), target molecule, signaling pathway, and direction of action of a development case.

[0506] The application environment adjustment score may be a score indicating the extent to which the candidate substance's site of action, route of administration, exposure characteristics, development stage, and clinical application environment correspond to the application environment of the development case.

[0507] The evidence usability adjustment score may be a score indicating the presence of location information of evidence included in the development case, the type of evidence, version information of the evidence, the confidence level of the evidence, and the applicability to the candidate substance.

[0508] In one embodiment, the case similarity score S can be calculated by the following mathematical formula.

[0509] S = M × (v1 × A + v2 × E + v3 × B)

[0510] Here, M is the modality gateway factor, A is the mechanism consistency score, E is the application environment adjustment score, and B is the evidence availability adjustment score. v1 to v3 may be weights applied to each score.

[0511] In one embodiment, the sum of v1 to v3 may be 1. In another embodiment, the sum of v1 to v3 may not be 1, and the processor (110) may normalize the calculated case similarity score to a preset range.

[0512] The processor (110) can determine v1 to v3 using a set of weights stored in the rule database (160). For example, if safety event prediction is important, the weight of the evidence availability correction score may be set relatively high. If the search for similar development cases based on mechanism of action (MOA) is important, the weight of the mechanism consistency score may be set relatively high.

[0513] The processor (110) can rank multiple development cases based on case similarity scores.

[0514] In one embodiment, the processor (110) can calculate a case similarity score for each of a plurality of development cases stored in a development case database (180) and sort the development cases in descending order of the calculated case similarity scores.

[0515] The processor (110) can select the top N development cases among the ranked development cases as a group of development cases to be analyzed. Here, N may be a pre-set positive integer.

[0516] In another embodiment, the processor (110) may select a development case whose case similarity score is greater than or equal to a preset similarity threshold value as a group of development cases to be analyzed.

[0517] In another embodiment, the processor (110) may select a group of development cases to be analyzed by considering the case similarity score and the type of supporting data together. For example, even if the case similarity score is high, if the location information of the supporting data does not exist or the type of supporting data corresponds to secondary summary data of unknown source, the processor (110) may exclude the development case from the group of development cases to be analyzed or mark it as a case requiring additional verification.

[0518] The development case group subject to analysis can be used as an analysis target to derive operational risks of new clinical trials.

[0519] The processor (110) can extract at least one of safety events, toxicity events, reasons for discontinuation of development, and reasons for clinical hold from the group of development cases to be analyzed.

[0520] Safety events may include adverse events, serious adverse events, death, life-threatening events, hospitalization, discontinuation of treatment, dose reduction, dose adjustment, or adverse events of special concern observed during clinical trials or post-approval use.

[0521] Toxic events may include toxic reactions observed in non-clinical or clinical trials. Toxic events may include organ toxicity, immunotoxicity, genotoxicity, reproductive toxicity, local toxicity, systemic toxicity, neurotoxicity, hepatotoxicity, cardiotoxicity, or unexpected toxic reactions.

[0522] Reasons for discontinuation of development may include safety issues, toxicity issues, insufficient efficacy, requirements for supplementation from regulatory agencies, manufacturing quality issues, administration route issues, or lack of commercial viability.

[0523] Reasons for a clinical hold may include reasons for which the regulatory agency has suspended the initiation or progress of the clinical trial. For example, insufficient safety data, insufficient non-clinical toxicology data, insufficient manufacturing quality data, the need to supplement the clinical trial plan, or inadequate subject protection measures may be grounds for a clinical hold.

[0524] The processor (110) can assign event name, event type, severity, stage of occurrence, frequency of occurrence, development discontinuation correlation, clinical hold correlation, source data location information, source data type and source data version information to each extracted event.

[0525] The processor (110) can classify a safety event or a toxicity event as an operational risk based on at least one of a case similarity score, the severity of the event, the stage of occurrence of the event, the relevance to development discontinuation, the relevance to clinical hold, the location information of the evidence, and the type of evidence.

[0526] In one embodiment, the processor (110) may classify safety events or toxicity events as predictable operational risks, operational risks requiring further verification, or operational risks with low applicability.

[0527] A processor (110) may classify a safety event or toxicity event as a predictable operational risk if the case similarity score is greater than or equal to a first risk classification threshold value, the safety event or toxicity event is associated with at least one of death, life-threatening, hospitalization, clinical hold, discontinuation of development, discontinuation of administration, or dose reduction, and the location information of the supporting data for the safety event or toxicity event corresponds to at least one of regulatory agency documents, approved product labels, clinical trial data, clinical trial results documents, safety data, or discontinuation records.

[0528] The processor (110) may classify the safety event or toxicity event as an operational risk requiring further verification if the case similarity score is greater than or equal to the similarity threshold value, the case similarity score is less than the first risk classification threshold value, or if the location information of the source data for the safety event or toxicity event corresponds to clinical trial registration information, development discontinuation press releases, company announcements, completed response cases, internal review records, or secondary summary data of unknown source, or does not exist.

[0529] The processor (110) may classify the safety event or toxicity event as an operational risk with low applicability if the case similarity score is below the similarity threshold value.

[0530] Predictable operational risks may be risks that need to be reflected in observation, reporting, discontinuation, or dose adjustment criteria in advance during the operation of a new clinical trial. Operational risks requiring further verification may be risks that require additional confirmatory data, additional safety reviews, additional non-clinical studies reviews, or expert reviews before being reflected in the new clinical trial. Operational risks with low applicability may be risks that may not be directly reflected in the response plan due to low similarity to the new clinical trial.

[0531] When the processor (110) generates a response plan or a corrective response package, it can map the operational risk classification results to the response plan or corrective response package.

[0532] In one embodiment, the processor (110) can map the predictable operational risk to at least one of the subject safety observation items, discontinuation criteria, dosage adjustment criteria, reporting subjects, reporting procedure information, or clinical investigator data collection reflection review items.

[0533] For example, if a specific serious adverse event is identified in a similar development case, and the adverse event is described in a document corresponding to a highly reliable type of evidence, and the case similarity score between the candidate substance and the similar development case is high, the processor (110) may reflect the adverse event in the subject safety observation item or the discontinuation criteria.

[0534] In another embodiment, the processor (110) may map the operational risk requiring additional verification to at least one of additional verification data, additional stability review, additional non-clinical trial review, items requiring expert review, or items requiring supplementary review for clinical trial plan approval.

[0535] In another embodiment, if the reason for discontinuation of development or the reason for clinical hold is related to safety or toxicity, the processor (110) may map the reason for discontinuation of development or the reason for clinical hold to at least one of the regulatory agency report review items, the clinical trial plan approval supplement review items, or the risk summary review items of the investigator's data booklet when generating a response plan or generating a corrective response package.

[0536] When an operational risk is reflected in a response plan, the processor (110) can output the location information of the supporting data, the type of supporting data, and the version information of the supporting data for the operational risk.

[0537] In one embodiment, if the investigational drug of a new clinical trial has a specific modality code, mechanism of action (MOA) code and target molecule identifier, the processor (110) can search for development cases having the same or similar modality, mechanism of action (MOA) or target molecule in the development case database (180).

[0538] The processor (110) can calculate a case similarity score for each of the searched development cases and select the top N development cases as a group of development cases to be analyzed.

[0539] If a life-threatening safety event is identified in one of the development cases among the group of development cases being analyzed, and the said safety event is listed in a regulatory agency document or clinical trial data booklet, and the case similarity score is greater than or equal to the first risk classification threshold value, the processor (110) may classify the said safety event as a predictable operational risk.

[0540] On the other hand, if a toxic event is identified in another development case but the location information of the source data for the toxic event corresponds only to company announcements or secondary summary data of unknown source, the processor (110) may classify the toxic event as an operational risk requiring additional verification.

[0541] If the case similarity score of another development case is less than the similarity case threshold value, the processor (110) may classify the safety event or toxicity event identified in that development case as an operational risk with low applicability.

[0542] The processor (110) can reflect predictable operational risks in the subject safety observation items, discontinuation criteria, dosage adjustment criteria, or reporting procedure information of the response plan, and output operational risks requiring additional verification as items requiring expert review.

[0543] The processor (110) can output location information of supporting materials for each item included in the response plan.

[0544] Source material location information may be information indicating which source material was used as the basis for generating a specific item of the response plan. Source material location information may include document identifiers, document names, document types, page numbers, paragraph numbers, sentence numbers, table numbers, item names, section names, clause numbers, or link information.

[0545] For example, if the reporting target or reporting deadline item of the response plan is based on a specific clause of the clinical trial protocol, the processor (110) can output the document name, version information, clause number, and page number of the said clinical trial protocol as source material location information.

[0546] As another example, if the item for checking storage conditions of the temperature deviation response plan is based on the storage condition clause of the investigational drug management manual, the processor (110) can output the location information of the said clause together.

[0547] As another example, if the subject safety observation item is based on a specific safety event listed in the clinical trial data or safety data, the processor (110) can output the section name, page number, and sentence number of the document as source data location information.

[0548] The processor (110) can determine the type of document corresponding to the source material location information and generate a source material type.

[0549] The types of supporting materials may include any one of regulatory agency documents, approved product labels, investigator's manuals, clinical trial results documents, safety data, clinical trial registration information, Q&A materials, standard operating procedures, internal review records, completed response cases, or company press releases.

[0550] In one embodiment, the processor (110) can determine the type of source material using a document identifier, document name, document source, document authoring organization, document metadata, or notation information within the document.

[0551] For example, if the source of the document is a regulatory agency and the name of the document corresponds to a guideline or an examination report, the processor (110) can determine that the document is a regulatory agency document.

[0552] As another example, if the document name corresponds to Investigator's Brochure or Investigator's Brochure, the processor (110) can determine that the document is Investigator's Brochure.

[0553] As another example, if the document is an internal meeting minutes or an internal review record, the processor (110) can determine that the document is an internal review record.

[0554] The processor (110) can use the type of supporting data to calculate the reliability score of the supporting data, classify operational risks, output items requiring expert review, or determine whether to select a response package.

[0555] The processor (110) can extract a sentence of grounds corresponding to a countermeasure or counter package item from the location information of grounds data.

[0556] Source sentences can be extracted from sentences, paragraphs, clauses, tables, or items within a document identified by the location information of the source material.

[0557] In one embodiment, the processor (110) can extract a sentence corresponding to a sentence number as a source sentence when the source data location information includes a sentence number.

[0558] In another embodiment, when the source material location information includes only a clause number or item name, the processor (110) can extract the sentence that is semantically most similar to the countermeasure item among the multiple sentences included in the clause or item as the source sentence.

[0559] In another embodiment, if the source data location information includes a table number or a table item, the processor (110) can extract a cell, row, or column related to a countermeasure item in the table as a source sentence or source data.

[0560] The processor (110) can store the extracted basis sentences by connecting them to each item of the response plan.

[0561] The processor (110) can determine whether there is a correspondence between the terms, standardization tags, numerical information, condition information, reporting procedure information, or documentation criteria included in the basis sentence and the items included in the response plan or response package.

[0562] For example, if the item of data requiring verification of the response plan is “temperature record” and the basis sentence includes the content “the storage temperature of the drug subject to clinical trial must be continuously recorded,” the processor (110) can determine that the basis sentence and the item of data requiring verification correspond to each other.

[0563] As another example, if the item subject to reporting in the response plan is “clinical trial manager” and the basis sentence includes content stating that it must be reported to the clinical trial manager, the processor (110) can determine that the basis sentence and the item subject to reporting correspond.

[0564] As another example, if the reporting deadline item of the response plan is “within 24 hours” and the same reporting deadline is included in the basis sentence, the processor (110) can determine that the basis sentence and the reporting procedure information correspond.

[0565] On the other hand, if an item of a response plan includes a specific reporting deadline but the basis sentence does not include information regarding the said reporting deadline, the processor (110) may determine that there is no correspondence between the basis sentence and the output item.

[0566] Additionally, if an item of the response plan includes observation of subject safety but the supporting sentence does not include information regarding observation of subject safety or related safety events, the processor (110) may output the item as an item requiring expert review.

[0567] The processor (110) may use a natural language processing model, a sentence embedding model, a rule-based matching model, or a combination thereof to determine whether there is a correspondence between a source sentence and an output item.

[0568] The processor (110) may output the item as an item requiring expert review if the source data location information does not exist, if the correspondence between the source sentence extracted from the source data location information and the item included in the response plan is not confirmed, or if the severity, stage of occurrence, or development discontinuation correlation of the event confirmed from multiple source data location information is inconsistent with each other.

[0569] For example, if the response plan includes an item “need to observe the safety of the subject” but there is no location information for the supporting data for that item, the processor (110) can output that item as an item requiring expert review.

[0570] As another example, if a specific safety event is described as mild in one source document but as a serious adverse event in another source document, the processor (110) may output an item requiring expert review due to the discrepancy in the severity of the event.

[0571] As another example, in one development case, a specific toxic event is identified only in the non-clinical stage, but in another development case, the same or similar event is described as being related to discontinuation of administration in the clinical stage, the processor (110) may output an item requiring expert review due to a discrepancy in the stage of occurrence or the relationship to discontinuation of development.

[0572] Items requiring expert review may include the item name, reason for the need for review, location information of relevant supporting documents, type of supporting documents, version information of supporting documents, details of inconsistencies, relevant response package, relevant operational risks, and recommended review contents.

[0573] In this way, the system (100) can prevent unfounded output or inaccurate response plans from being used as is by verifying the location information of the source material, the type of source material, the version information of the source material, and the correspondence with the source sentence regarding the response plan generated by the artificial intelligence model (121).

[0574] In one embodiment, the completed response case may be a case where the drug subject to the clinical trial was exposed to a temperature outside the permitted storage conditions.

[0575] The above completed response cases may include the occurrence of temperature deviation, verification of temperature records, verification of acceptable storage conditions, calculation of deviation time, calculation of cumulative deviation time, verification of lot information, verification of whether the lot was used, verification of whether it was before or after administration, verification of stability data, determination of the necessity of observing subject safety, verification of reporting procedure information, and documentation procedures.

[0576] The processor (110) can analyze the above completed response case to extract the occurrence of temperature deviation as the initial operational issue, and can extract temperature record verification, deviation time calculation, lot information verification, stability data verification, verification of whether before and after administration, reporting procedure information verification and documentation as subsequent operational issues.

[0577] The temperature records, lot information, medication records, and stability data actually obtained in the above completed response cases may be verification data. Based on the above verification data, the processor (110) can derive the verification data required for the temperature deviation response package to be applied to the new clinical trial.

[0578] The processor (110) can link subsequent operational issues that must be performed after a temperature deviation occurs into an issue chain.

[0579] For example, the processor (110) can determine a chronological relationship between the occurrence of a temperature deviation and the verification of the temperature record. Additionally, it can determine a chronological relationship between the verification of the temperature record and the calculation of the deviation time. A causal relationship or a conditional relationship can be determined between the calculation of the deviation time and the verification of stability data.

[0580] Additionally, the processor (110) can determine the same lot relationship between operational issue units containing the same lot identifier. If the lot has already been administered, the processor (110) can determine the conditional relationship between the post-administration state and the determination of the need for subject safety observation.

[0581] Additionally, if the reporting target, reporting deadline, reporting form, or reporting path related to the response to temperature deviation is the same, the processor (110) may determine that there is an identical reporting procedure relationship between the related operational issue units.

[0582] Accordingly, the processor (110) can create an issue chain in which, after a temperature deviation occurs, the temperature record is checked, the deviation time is calculated, the cumulative deviation time is calculated, the lot information is checked, whether it is before or after administration is checked, the stability data is checked, the lot is isolated, the subject's safety is observed, and the report and deviation report are written as subsequent operational issues.

[0583] The processor (110) can generate a temperature deviation response package.

[0584] The temperature deviation response package may include an issue type code, a parent issue group code, occurrence conditions, subsequent operational issues, materials requiring verification, response requirements, a sequential response procedure sequence, reporting procedure information, documentation items, preventive measures, follow-up observation items, location information of supporting materials, type of supporting materials, and version information of supporting materials.

[0585] For example, data requiring verification may include temperature records, permissible storage conditions, actual storage temperature, time out of reach, cumulative time out of reach, lot identifier, dispensing records, and stability data.

[0586] The sequential response procedure sequence may include securing temperature records, verifying acceptable storage conditions, calculating out-of-time, calculating cumulative out-of-time, lot isolation, verifying stability data, determining availability, reporting, and documentation.

[0587] In the case of a post-medication state, the sequential response procedure sequence may additionally include items for determining the necessity of observing the subject's safety and follow-up observation.

[0588] The processor (110) can calculate the applicability score of the response package by comparing the investigational drug management manual, stability data, dosing schedule, reporting procedure information and documentation standards included in the new clinical trial operation data with the temperature deviation response package.

[0589] If the applicability score of the response package is greater than or equal to the applicable threshold value, and the source data location information and source data version information correspond to the current version, the processor (110) can select a temperature deviation response package.

[0590] When the processor (110) is in a pre-administration state, it can generate a response plan including isolating the lot, reviewing availability, verifying stability data, reporting, and documentation.

[0591] On the other hand, in the case of a post-medication state, the processor (110) can generate a response plan including subject safety observation, verification of safety evaluation items, reporting, documentation, and items requiring expert review.

[0592] In one embodiment, a completed response case may be a case where the subject did not visit on the scheduled visit date or did not visit within the allowed visit period.

[0593] The above completed response cases may include missing visits, verification of the permitted visit period, verification of the possibility of revisits, identification of missing evaluation items, verification of missing specimens, review of the impact on key evaluation variables, review of the impact on subject safety, determination of deviations from the clinical trial protocol, verification of reporting procedure information, and documentation procedures.

[0594] The processor (110) can extract a missed visit or a delayed visit as an initial operational issue, and can extract subsequent operational issues such as verifying the allowed visit period, verifying whether a revisit is possible, verifying the missed evaluation items, reviewing the impact of key evaluation variables, determining whether there is a deviation from the clinical trial protocol, and verifying and documenting reporting procedure information.

[0595] The processor (110) can create an issue chain in which the allowed visit period after a missed visit, the possibility of a revisit, the missed evaluation items, the missing specimens, the impact of major evaluation variables, the record of deviation from the clinical trial protocol, and the reporting and prevention of recurrence measures are linked as subsequent operational issues.

[0596] For example, if a revisit is possible within the allowed visit period, the processor (110) can link the revisit schedule adjustment and the supplementation of missing evaluation items to subsequent operational issues.

[0597] On the other hand, if a revisit is not possible within the allowed visit period, the processor (110) can review the impact of key evaluation variables, determine whether there is a deviation from the clinical trial protocol, and link reporting and documentation to subsequent operational issues.

[0598] In addition, regarding the omission of visits, if the reporting subject, reporting deadline, reporting form, or reporting path is the same, the processor (110) can determine the same reporting procedure relationship between the related operational issue units.

[0599] The processor (110) can generate a response plan for missing or delayed visits by reflecting the visit schedule, allowed visit period, evaluation items, key evaluation variables, safety evaluation items, reporting procedure information and documentation standards of the new clinical trial operation data.

[0600] The response plan may include the scheduled visit date, actual visit date, allowable visit period, omitted evaluation items, omitted specimens, relevance to key evaluation variables, and the possibility of a revisit as data requiring verification.

[0601] Initial measures may include contacting the subject, scheduling a follow-up visit, verifying missing evaluation items, and determining the feasibility of a follow-up visit.

[0602] Reporting procedure information may include the reporting subject, reporting deadline, reporting form, reporting route, and reporting requirements for at least one of the clinical trial principal investigator, monitor, sponsor, or data manager.

[0603] Documentation items may include visit records, clinical trial protocol deviation reports, monitoring reports, or internal review records.

[0604] The processor (110) may output the item as an item requiring expert review if there is a possibility of influence on the major evaluation variable or if revisiting is not possible.

[0605] In one embodiment, a completed response case may be a case where the specimen was not processed according to the time or conditions specified in the clinical trial protocol or laboratory manual.

[0606] The above completed response cases may include sample processing errors, verification of processing standards, verification of actual processing time, verification of storage conditions, verification of transportation conditions, verification of analyzability, verification of re-collection possibility, review of the impact on key evaluation variables, recording of deviations from the clinical trial protocol, and procedures for measures to prevent recurrence.

[0607] The processor (110) can extract sample processing errors as the initial operational issue, and can extract verification of processing standards, verification of actual processing time, verification of analyzability, verification of re-collection possibility, review of the impact of key evaluation variables, and reporting and documentation as subsequent operational issues.

[0608] In the above completed response case, the actual time of sample collection, the actual time of processing, the records of storage conditions, the results of the analysis institution's review, or the results of the review of the possibility of re-collection may be verification data. Based on the above verification data, the processor (110) can derive the necessary verification data for the sample processing error response package to be applied to the new clinical trial.

[0609] The processor (110) can create an issue chain in which, following a sample processing error, the laboratory manual standard is checked, the actual processing time is checked, storage conditions are checked, the possibility of analysis is checked, the impact of key evaluation variables is reviewed, the possibility of re-collection is checked, and reporting and documentation are linked as subsequent operational issues.

[0610] For example, if the actual processing time exceeds the allowed processing time but meets the criteria for analyzability, the processor (110) can link the securing and documentation of the basis for analyzability to subsequent operational issues.

[0611] On the other hand, if the actual processing time exceeds the allowed processing time and the possibility of analysis is not confirmed, the processor (110) can connect the confirmation of the possibility of re-sampling, the review of the impact of key evaluation variables, and the output of items requiring expert review to subsequent operational issues.

[0612] The processor (110) can generate a response plan for sample processing errors by reflecting the sample management conditions, analysis purpose, key evaluation variables, whether re-collection is possible, reporting procedure information, and documentation standards of the new clinical trial operation data.

[0613] The response plan may include the time of sample collection, the actual time of processing, the allowable processing time, storage conditions, transportation conditions, analysis institution standards, data for determining the feasibility of analysis, and whether re-collection is possible as data requiring verification.

[0614] Initial measures may include checking the status of the sample, contacting the analysis institution, reviewing the feasibility of analysis, and confirming whether re-collection is possible.

[0615] The reporting procedure information may include the reporting subject, reporting deadline, reporting form, reporting route, and reporting requirements for at least one of the clinical trial principal investigator, monitor, analysis agency, sponsor, or data manager.

[0616] Documentation items may include specimen handling records, specimen drop reports, clinical trial protocol drop reports, monitoring reports, or internal review records.

[0617] If the sample is related to the main evaluation variable and the possibility of analysis is not confirmed, the processor (110) may output the item as an item requiring expert review.

[0618] In one embodiment, a completed response case may be a case in which the drug subject to the clinical trial is administered differently from the dosage, frequency, timing, route of administration, or subject specified in the clinical trial protocol.

[0619] The above completed response cases may include the occurrence of medication errors, verification of medication records, identification of the subject, verification of the dosage, verification of the time of administration, verification of the route of administration, assessment of the impact on subject safety, assessment of the necessity of reporting, and documentation procedures.

[0620] The processor (110) can extract the occurrence of a medication error as the initial operational issue and can extract the verification of medication records, verification of the subject, verification of the actual amount of medication, verification of the time of administration, determination of the impact on the subject's safety, determination of the need for additional observation, verification of reporting procedure information, and documentation as subsequent operational issues.

[0621] The processor (110) can generate an issue chain in which, following a medication error, the medication record is verified, the subject is identified, the actual dosage is verified, the dosage is compared with the dosage based on the clinical trial protocol, the impact on the subject's safety is determined, the need for additional observation is determined, and reporting and documentation are linked as subsequent operational issues.

[0622] For example, if the actual dosage exceeds the dosage based on the clinical trial protocol, the processor (110) can determine the impact on subject safety and the need for additional observation as subsequent operational issues.

[0623] On the other hand, if the medication error corresponds to a delay in the timing of medication administration and the likelihood of impact on subject safety is low, the processor (110) can determine whether there is a deviation from the clinical trial protocol, and report and document it as a subsequent operational issue.

[0624] Additionally, if a medication error is related to the potential impact on the safety of the subject, the processor (110) can determine the conditional relationship between the medication error and the follow-up observation item.

[0625] The processor (110) can generate a response plan for medication errors by reflecting the medication schedule, dosage criteria, medication route, discontinuation criteria, dosage adjustment criteria, safety evaluation items, reporting procedure information and documentation criteria of the new clinical trial operation data.

[0626] The response plan may include subject identifiers, medication records, actual dosage, dosage based on the clinical trial protocol, time of administration, route of administration, concomitant drug information, and safety evaluation data as materials requiring verification.

[0627] Initial measures may include checking the subject's condition, withholding additional medication, reporting to the principal investigator, verifying safety assessment items, and requesting a medical review if necessary.

[0628] Information on reporting procedures may include the reporting subjects, reporting deadlines, reporting forms, reporting routes, and reporting requirements for at least one of the clinical trial principal investigator, sponsor, safety officer, monitor, or regulatory authority.

[0629] Documentation items may include medication records, clinical trial protocol deviation reports, adverse event records, monitoring reports, or internal review records.

[0630] The processor (110) may include subject safety observation items or items requiring expert review in response measures if there is a possibility of subject safety impact or if the actual dosage exceeds the clinical trial protocol standards.

[0631] In one embodiment, a completed response case may be a case where the procedures, schedule, tests, administration, evaluation, or reporting criteria set forth in the clinical trial protocol were not observed.

[0632] The above completed response cases may include the occurrence of deviation from the clinical trial protocol, identification of the type of deviation, identification of the time of deviation, identification of the scope of impact, review of the impact on key endpoints, review of the impact on subject safety, determination of the need for reporting, and documentation and establishment of measures to prevent recurrence.

[0633] The processor (110) can identify the occurrence of a deviation from the clinical trial protocol as an initial operational issue, and identify the type of deviation, identify the scope of impact, review the impact on key evaluation variables, review the impact on subject safety, identify information on reporting procedures, and identify documentation and preventive measures as subsequent operational issues.

[0634] The processor (110) can create an issue chain in which the scope of impact after deviation from the clinical trial protocol, the impact on key endpoints, the impact on subject safety, reporting, documentation, and preventive measures are linked as subsequent operational issues.

[0635] For example, if a deviation from the clinical trial protocol is related to a major evaluation variable, the processor (110) can link the review of the major evaluation variable impact and the verification of data processing criteria to subsequent operational issues.

[0636] On the other hand, if the deviation from the clinical trial protocol is related to subject safety, the processor (110) can connect the review of the impact on subject safety, additional safety observation, and determination of the need for reporting to subsequent operational issues.

[0637] Additionally, if the deviation from the clinical trial protocol is related to a specific reporting subject, reporting deadline, reporting form, or reporting path, the processor (110) can link the reporting procedure information to the issue chain.

[0638] The processor (110) can generate a response plan for deviations from the clinical trial protocol by reflecting the clinical trial protocol clauses, documentation standards, reporting procedure information, agency role information, and standard operating procedures of the new clinical trial operation data.

[0639] The response plan may include the date of withdrawal, type of withdrawal, relevant clinical trial protocol provisions, affected subjects, subject identifiers, relevance to key endpoints, impact on subject safety, and reporting criteria as data requiring verification.

[0640] Initial measures may include confirming the withdrawal, securing relevant data, reporting to the principal investigator, reviewing the scope of impact, and determining the need for additional measures.

[0641] Information on reporting procedures may include the reporting subjects, reporting deadlines, reporting forms, reporting channels, and reporting requirements for at least one of the clinical trial principal investigator, sponsor, monitor, institution, or regulatory authority.

[0642] Documentation items may include clinical trial protocol withdrawal reports, monitoring reports, subject records, internal review records, or records of preventive measures.

[0643] The processor (110) can output the item as requiring expert review if the deviation from the clinical trial protocol is related to the primary evaluation variable or subject safety.

[0644] According to the present invention, by extracting the issue chain relationship between the initial operational issue and the subsequent operational issue from completed response cases, it is possible to systematically derive the issues that need to be subsequently identified or processed when an issue occurs during the operation of a clinical trial.

[0645] In addition, according to the present invention, by generating a response package based on issue chain relationships and sequential response procedure sequences, standardized response plans can be generated based on completed response cases without having to create new response plans for clinical trial operation issues every time.

[0646] In addition, according to the present invention, by calculating a score of applicability of the corresponding package between the new clinical trial operation data and the corresponding package, a corresponding package applicable to the new clinical trial can be selected quantitatively or semi-quantitatively.

[0647] In addition, according to the present invention, by generating candidate issue type codes from new clinical trial operational data and calculating an issue type gateway coefficient based on whether the candidate issue type codes match the issue type codes of each corresponding package, it is possible to more systematically derive expected operational issues that may occur in new clinical trials.

[0648] In addition, according to the present invention, even if the applicability score of the corresponding package is greater than or equal to the applicable threshold value, if there exists a difference in the protocol, a difference in the management manual for investigational medicinal products, a change in regulations, a difference in the version of supporting data, a difference in the administration status, a difference in subject safety effects, or a difference in the investigational medicinal product between the new clinical trial operational data and the corresponding package, the item corresponding to the said difference can be output as an item requiring modification.

[0649] In addition, according to the present invention, by distinguishing items requiring expert review using source data location information, source data type, source data version information, and whether there is a correspondence between source sentences and output items, the possibility of unfounded output from an artificial intelligence model can be reduced.

[0650] In addition, according to the present invention, by verifying whether the source data location information and source data version information included in the corresponding package correspond to the current version applied to the new clinical trial operational data, it is possible to prevent the inappropriate application of response measures based on past versions of regulations, clinical trial protocols, investigational medicinal product management manuals, or standard operating procedures.

[0651] In addition, according to the present invention, an updated response package is created based on a new completed response case or revised / changed response basis data, and the updated response package and package change history are stored in a response package database (170), thereby maintaining the up-to-dateness and traceability of the response package.

[0652] In addition, according to the present invention, safety events, toxicity events, reasons for discontinuation of development, or reasons for clinical hold identified in similar development cases are classified as operational risks using a development case database (180), and by reflecting these in a response plan or a correction response package, a clinical trial operation response plan that takes into account the characteristics of the drug subject to the clinical trial can be generated.

[0653] In addition, according to the present invention, by mapping predictable operational risks to subject safety observation items, discontinuation criteria, dosage adjustment criteria, reporting subjects, reporting procedure information, or review items for inclusion in the investigator's manual, and mapping operational risks requiring additional verification to additional confirmation data, additional safety review, additional non-clinical study review, items requiring expert review, or review items for supplementary review for clinical trial plan approval, safety or toxicity information of similar development cases can be converted into clinical trial operational action items.

[0654] In addition, according to the present invention, for various clinical trial operational issues such as temperature deviation, omission of visits, sample processing errors, medication errors, and deviation from the clinical trial protocol, response measures can be generated by utilizing issue chain relationships based on completed response cases and sequential response procedure sequences.

[0655] Although embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the above embodiments. A person skilled in the art to which the present invention pertains may modify, substitute, add, or delete parts of the components within the scope of the technical scope of the present invention, and such modification, substitution, addition, or deletion may be included within the scope of the rights of the present invention. Explanation of the symbols

[0656] 100: AI-based system for identifying anticipated clinical trial operational issues and generating response strategies 110: Processor 120: Memory 121: Artificial Intelligence Model 130: Completed Response Case Database 140: Clinical Trial Operational Data Database 150: Basis for Response Database 160: Rule Database 170: Response Package Database 180: Development Case Database 190: User terminal

Claims

Claim 1 One or more processors; memory storing instructions and artificial intelligence models executed by said processors; a database of completed response cases; a database of clinical trial operational data; a database of response grounds; a rule database; and response package database; wherein the completed response case database stores completed response cases, which are completed clinical trial operational issue response cases, wherein the completed response case includes at least one of the following: an initial operational issue that occurred in a specific clinical trial; a subsequent operational issue identified or processed after the initial operational issue; a phrase indicating the occurrence time, processing time, processing sequence, stage of occurrence, cause of occurrence, conditional relationship of the initial operational issue or the subsequent operational issue; a phrase indicating the causal relationship; an affected subject; an affected subject identifier; clinical trial drug information; clinical trial drug identifier; lot identifier; subject identifier; institution identifier; administration status; subject safety impact information; verification data; response requirements; response procedures; action results; reporting subjects; reporting procedure information; documentation items; preventive measures; follow-up observation items; source data location information; source data type; and source data version information, wherein the reporting procedure information includes at least one of the reporting subjects; reporting deadline; reporting form; reporting path; and the clinical trial drug information includes clinical trial drug identification information, candidate substance name, modality code, regulatory classification code, It includes at least one of a mechanism of action (MOA) code, target molecule identifier, direction of action code, site of action code, route of administration code, exposure characteristic code, storage conditions, preparation conditions, administration conditions, and lot information; the clinical trial operational data database stores new clinical trial operational data including at least one of a clinical trial protocol, investigational medicinal product management manual, investigational medicinal product management information, visit schedule, administration schedule, specimen management information, institutional role information, and documentation standards for the new clinical trial; and the corresponding basis database includes clinical trial-related regulations, guidelines, Q&A materials, standard operating procedures, internal review records,It stores at least one of the clinical trial protocol provisions, investigational drug management manual provisions, investigator's manual, approved product label, and safety data; the rule database stores at least one of the issue classification rule, issue chain extraction rule, response procedure extraction rule, response package application rule, applicability score calculation rule, risk classification rule, weight set, mapping rule, and top issue group definition information; the top issue group defined by the top issue group definition information corresponds to any one of the management of the investigational drug, administration management, visit management, specimen management, reporting management, documentation management, and clinical trial protocol deviation management; the processor extracts the initial operational issue, subsequent operational issue, occurrence conditions of the subsequent operational issue, verification data, data requiring verification, response requirements, response procedures, action results, reporting procedure information, source data location information, source data type, and source data version information from the completed response case using the artificial intelligence model; and converts the initial operational issue and the subsequent operational issue into a standard response case profile, wherein the standard response case profile includes an issue type code, a top issue group code, an occurrence stage code, an occurrence cause code, an affected target code, and a clinical trial Including at least one of the target drug code, standardization code of the drug subject to clinical trial, modality code, regulatory classification code, mechanism of action (MOA) code, target molecule identifier, administration status code, subject safety impact code, reporting target code, reporting procedure code, documentation code, source data location information, source data type, and source data version information, and based on at least one of the occurrence time, processing time, processing sequence, phrase indicating conditional relationship, phrase indicating causal relationship, action result, affected target identifier, clinical trial drug identifier, lot identifier, subject identifier, institution identifier, documentation item, and reporting procedure information included in the completed response case, the initial operational issue and the subsequent operational issue are classified into respective operational issue units.An issue chain relationship is created by determining at least one of the following relationships among the above operational issue units: chronological relationship, causal relationship, conditional relationship, relationship of the same affected subject, relationship of the same clinical trial drug, same lot relationship, same subject relationship, same institution relationship, same documentation item relationship, or same reporting procedure relationship; wherein the issue chain relationship includes a subsequent operational issue to be identified or processed following the above initial operational issue, the occurrence condition of the said subsequent operational issue, data required for verification, response requirements, and processing order; a sequential response procedure sequence is created from the issue chain relationship and the said response procedure, including at least one of securing verification data, initial action, reporting, documentation, recurrence prevention measures, and follow-up observation; and based on the issue chain relationship and the said sequential response procedure sequence, for each issue type, at least one of an issue type code, parent issue group code, occurrence condition, subsequent operational issue, data required for verification, response requirements, sequential response procedure sequence, clinical trial drug information, clinical trial drug standardization code, reporting subject, reporting procedure information, documentation item, recurrence prevention measures, follow-up observation item, source data location information, source data type, and source data version information is included. A plurality of response packages are generated, the plurality of response packages are stored in the response package database, and a new operational profile is generated from the new clinical trial operational data, wherein the new operational profile includes at least one of an issue type code, a parent issue group code, a clinical trial drug management element, a dosing element, a visit element, a specimen management element, a reporting element, a documentation element, clinical trial drug information, a clinical trial drug standardization code, a modality code, a regulatory classification code, a mechanism of action (MOA) code, a target molecule identifier, a dosing status code, a subject safety impact code, and a standard operational code comparable to the standard response case profile, and the degree of issue type congruence between the new operational profile and each of the plurality of response packages stored in the response package database,A response package applicability score is calculated based on at least one of the following: consistency of occurrence stage, consistency of affected subjects, consistency of administration status, consistency of operating environment, similarity of the clinical trial drug, and reliability of evidence; if the response package applicability score is greater than or equal to a preset applicability threshold, location information of the evidence included in the response package exists, and version information of the evidence included in the response package corresponds to the current version applied to the new clinical trial operational data, one or more response packages applicable to the new clinical trial operational data are selected from among multiple response packages stored in the response package database; issue types corresponding to the selected response package are derived as expected operational issues that may occur in the new clinical trial; items among the selected response packages that differ from the new clinical trial operational data are corrected to generate a corrected response package; and a response plan for the new clinical trial operational data is generated based on the selected response package or the corrected response package, wherein the response plan includes data requiring verification, initial measures, reporting subjects, reporting procedure information, documentation items, recurrence prevention measures, follow-up observation items, items requiring correction, and items requiring expert review Including at least one of the above, and if the applicability score of the response package is greater than or equal to the applicability threshold value, but there exists a protocol difference, a difference in the investigational medicinal product management manual, a change in regulations, a difference in the version of the evidence data, a difference in the administration status, a difference in subject safety impact, or a difference in the investigational medicinal product between the new operational profile and the selected response package, the item corresponding to the difference among the items included in the response plan is output as an item requiring modification, and if the applicability score of the response package is less than the applicability threshold value, or if the location information of the evidence data included in the response package does not exist, or if the version information of the evidence data included in the response package does not correspond to the current version applied to the new clinical trial operational data,An AI-based system for deriving anticipated clinical trial operation issues and generating response plans, which, when no correspondence is confirmed between a supporting sentence extracted from the location information of the supporting data and an item included in the corresponding package or the corresponding plan, does not select the corresponding package as an applicable corresponding package and outputs the corresponding package or at least some items included in the corresponding package as items requiring expert review. Claim 2 In claim 1, the processor calculates an issue type gateway coefficient based on whether there is an issue type match between the new operation profile and each of the plurality of corresponding packages stored in the corresponding package database, wherein the issue type gateway coefficient is calculated as 1 when the issue type code of the new operation profile and the issue type code of the corresponding package are identical, and as 0 when the issue type code of the new operation profile and the issue type code of the corresponding package are different but belong to the same upper issue group.It is calculated as 5, and is calculated as 0 if the issue type code of the new operational profile and the issue type code of the response package are different and do not belong to the same upper issue group; a problem situation match score is calculated based on at least one of the match in occurrence stage, match in affected subjects, match in administration status, and match in subject safety impact between each of the multiple response packages stored in the new operational profile and the response package database; an operational environment adjustment score is calculated based on whether there is a match in at least one of the clinical trial site role, clinical trial drug management conditions, lot scope, visit schedule, specimen management conditions, reporting deadline, and documentation standards between each of the multiple response packages stored in the response package database and the new operational profile; and a clinical trial drug similarity score is calculated based on at least one of the match in modality code, regulatory classification code, mechanism of action (MOA) code, target molecule match score, direction of action match score, route of administration match score, and exposure characteristic match score between the clinical trial drug information included in the new operational profile and the clinical trial drug information included in the response package. An AI-based system for deriving expected clinical trial operation issues and generating response plans, which calculates an evidence reliability score based on at least one of the existence of source data location information included in the response package, the type of source data, whether the source data version information corresponds to the current version, and whether there is a correspondence between a source sentence extracted from the source data location information and an item included in the response package; wherein the applicability score of the response package is calculated by multiplying the issue type gateway coefficient by a weighted sum to which weights corresponding to the problem situation match score, the operating environment correction score, the clinical trial target drug similarity score, and the evidence reliability score are respectively applied; and when the issue type gateway coefficient is 0, the response package is not selected as an applicable response package but is output as an item requiring expert review. Claim 3 In claim 1, when a new completed response case or revised / changed response basis data is input, the processor comprises, wherein the revised / changed response basis data includes at least one of revised clinical trial-related regulations, revised guidelines, revised Q&A data, revised standard operating procedures, revised clinical trial protocols, or revised investigational medicinal product management manuals, and among a plurality of response packages stored in the response package database, an update target response package related to the new completed response case or the revised / changed response basis data, and compares the new completed response case with at least one of the issue type, top issue group, occurrence condition, subsequent operational issue, verification data, data requiring verification, response requirement, sequential response procedure sequence, reporting procedure information, and action result included in the update target response package, and compares the revised / changed response basis data with at least one of the basis data location information and basis data version information included in the update target response package, and based on the comparison result, the occurrence condition, subsequent operational issue, data requiring verification, response requirement, reporting target, An AI-based clinical trial operation expected issue derivation and response plan generation system that determines whether at least one of reporting procedure information, documentation items, sequential response procedure sequence, recurrence prevention measures, and follow-up observation items needs to be changed; if an item requiring change exists, updates the response package subject to update to create an updated response package; and stores a package change history for the updated response package in the response package database, which includes at least one of a pre-update version, a post-update version, identification information of revised / changed response basis data, changed items, reasons for change, and update date. Claim 4 In claim 1, the invention further comprises a development case database storing multiple development cases, wherein the development case database stores a development case profile for each development case, the profile including at least one of a modality code, regulatory classification code, mechanism of action (MOA) code, target molecule identifier, signaling pathway identifier, direction of action code, site of action code, route of administration code, exposure characteristics code, development phase information, safety event information, toxicity event information, reason for discontinuation of development, reason for clinical hold information, and source data location information; the processor generates a candidate substance profile based on the clinical trial subject drug information included in the novel clinical trial operation data, converts the candidate substance profile into the same standard tag system as the development case profile to generate a standardized profile including a modality code, regulatory classification code, mechanism of action (MOA) code, target molecule identifier, signaling pathway identifier, direction of action code, site of action code, route of administration code, and exposure characteristics code, and based on whether there is a semantic correspondence between the standardized profile and the development case profile, modality gateway coefficient, mechanism concordance score, application environment Calculate a correction score and an evidence availability correction score, calculate a case similarity score by multiplying the weighted sum of the modality gateway coefficient, the mechanism consistency score, the application environment correction score, and the evidence availability correction score, respectively, and rank multiple development cases based on the case similarity score, and select the top N development cases among the ranked development cases as the group of development cases to be analyzed, where N is a pre-set positive integer, extract at least one of a safety event, a toxicity event, a reason for discontinuation of development, and a reason for clinical hold from the group of development cases to be analyzed, and the case similarity score is greater than or equal to a first risk classification threshold value set to a value greater than the similarity threshold value, and the safety event or toxicity event is associated with at least one of death, life-threatening, hospitalization, clinical hold, discontinuation of development, discontinuation of administration, or dose reduction, andIf the location information of the supporting data for the above safety event or toxicity event corresponds to at least one of regulatory agency documents, approved product labels, investigator's manuals, clinical trial outcome documents, safety data, or discontinuation records, the above safety event or toxicity event is classified as a foreseeable operational risk; if the above case similarity score is greater than or equal to the above similarity threshold value and the above case similarity score is less than the above first risk classification threshold value, or if the location information of the supporting data for the above safety event or toxicity event corresponds to clinical trial registration information, discontinuation press releases, company announcements, completed response cases, internal review records, or secondary summary data of unknown source, or if such data does not exist, the above safety event or toxicity event is classified as an operational risk requiring further verification; if the above case similarity score is less than the above similarity threshold value, the above safety event or toxicity event is classified as an operational risk with low applicability; and when generating the above response plan or generating the above correction response package, the above foreseeable operational risk is mapped to at least one of subject safety observation items, discontinuation criteria, dose adjustment criteria, reporting targets, or investigator's manual reflection review items, and when generating the above response plan or the above correction When generating a response package, the aforementioned operational risk requiring additional verification is mapped to at least one of additional verification data, additional safety review, additional nonclinical study review, items requiring expert review, or items requiring supplementary review for clinical trial plan approval; and if the aforementioned reason for discontinuation of development or reason for clinical hold is related to safety or toxicity, when generating the aforementioned response plan or the aforementioned corrective response package, the aforementioned reason for discontinuation of development or reason for clinical hold is mapped to at least one of items requiring review for regulatory agency reporting, items requiring supplementary review for clinical trial plan approval, or items requiring review of the risk summary in the investigator's manual; and for each item included in the aforementioned response plan, source data location information is output together; and if the aforementioned source data location information does not exist, orAn AI-based system for deriving anticipated clinical trial operational issues and generating response plans that outputs an item requiring expert review when a correspondence between a supporting sentence extracted from the location information of the supporting data and an item included in the response plan is not confirmed, or when the severity, stage of occurrence, or relevance to development discontinuation of events confirmed from multiple location information of supporting data is inconsistent with one another.

Images