System and method for autonomously generating and modifying dynamic electronic trial master file in real time

Abstract

The present disclosure relates to a system and method for autonomous real-time generation and modification of a dynamic hierarchical file structure of an electronic test master file (eTMF). A method includes receiving, by an event processor, an event message from a messaging system, the messaging system integrating a plurality of applications configured to automatically generate a message in real time in response to an event. The method further includes determining, by the event processor, one or more hierarchical file structure placeholders associated with the respective activity for each of the event messages based on the metadata of each of the event messages. The method further includes automatically modifying, by the event processor, the hierarchical file structure in real time to contain the one or more hierarchical file structure placeholders. The method further includes storing one or more documents received from a computer system of the first site in the one or more hierarchical file structure placeholders.

Description

Technical Field

[0001] This disclosure generally involves the autonomous, real-time generation and modification of dynamic electronic test master files. Background Technology

[0002] The electronic trial master file (eTMF) serves as a central repository for various documents collected during clinical trials. Regulatory agencies such as the U.S. Food and Drug Administration (FDA) and corresponding agencies in other countries regularly review the eTMF to ensure that clinical trials are conducted in a compliant and regulated manner.

[0003] Traditional methods of creating eTMFs are largely manual and labor-intensive. Typically, these processes begin by defining the structure of the eTMF. This involves generating a documentation plan that outlines the relevant document types required for the experiment. The team then manually identifies appropriate documents based on the nature of the research and creates a hierarchical structure for these documents, populated with placeholders.

[0004] Such methods require significant manual intervention, including printing and scanning documents. Once printed and manually completed, the documents are scanned back into the system for review and then placed in the correct location within the eTMF folder structure. This process is repeated for each document, resulting in an extremely time-consuming and inefficient workflow. As a further example, continuous quality control reviews are required as regulations and / or research change over time (e.g., the addition or removal of pilot sites).

[0005] In traditional methods, document planning is based on industry-standard reference models, such as the Clinical Data Exchange Standards Consortium (CDISC) Trial Master Document Reference Model, which is currently the standard in the field (“TMF Reference Model”). This plan specifies the structure and type of the required documents, which are then manually entered into the eTMF. Despite advancements in digital technology, a significant amount of paper is still used because documents are typically printed, filled out manually, and then scanned back into the system. Therefore, overall, the current state of eTMF management is characterized by manual operation, requiring substantial human resources for reviewing, classifying, and properly archiving documents. Summary of the Invention

[0006] Given the shortcomings of the traditional methods mentioned above, it is necessary to provide technical solutions to address the technical problems, namely, to provide more efficient and automated solutions to simplify the eTMF process and improve the efficiency of clinical trial document management.

[0007] The disclosed embodiments provide a system, a computer-implemented method, and a computer-readable medium for autonomously generating and modifying a dynamic hierarchical file structure of an electronic trial master file (eTMF) in real time. The method includes receiving event messages from a messaging system by an event processor. The messaging system integrates multiple applications configured to automatically generate messages in real time in response to events. Each event corresponds to a user interaction at a first site via a user interface of the computer system and one of the applications, based on an activity associated with the clinical trial. Each event message corresponding to each user interaction contains metadata associated with the corresponding activity. The method further includes the event processor determining one or more hierarchical file structure placeholders associated with the corresponding activity for each event message based on the metadata of each event message. The method further includes the event processor automatically modifying the hierarchical file structure in real time to include the one or more hierarchical file structure placeholders. The method further includes storing one or more documents received from the computer system at the first site within the one or more hierarchical file structure placeholders.

[0008] An embodiment may include one or more of the following features.

[0009] Metadata associated with the corresponding activity may include one or more of the following: a site identifier associated with the site, a milestone identifier corresponding to the activity, and role assignments for users associated with the activity. This method may include notifying one or more users of the first site in real time when placeholders are created in the hierarchical file structure. This method may include automatically validating one or more documents received from a computer system according to a predefined format or compliance standard before storing them in the hierarchical file structure placeholders.

[0010] Determining one or more hierarchical file structure placeholders by the event handler may include dynamically applying one or more predefined templates from a template library, which specify the expected document type for activities associated with the clinical trial. Automatic, real-time modification of the hierarchical file structure may include dynamically adjusting the structure to meet region-specific requirements associated with the first site.

[0011] This method may include tracking the completion status of hierarchical file structure placeholders across multiple sites and generating quantitative metrics in real time for completed or pending placeholders at each site. The method may include generating an alert when one or more placeholders remain unpopulated beyond a predetermined compliance period, wherein the alert is sent to a designated user. The method may include automatically replacing existing placeholders in the hierarchical file structure with updated placeholders in response to receiving an event message indicating a change in the corresponding activity.

[0012] Hierarchical file structure placeholders can be organized into packages, each package associated with a specific milestone or activity in a clinical trial. This method can include associating one or more documents with placeholder-specific metadata, where the metadata includes document type, author, version, and applicable milestone identifier. The hierarchical file structure can be configured to logically divide placeholders into regions and sections corresponding to study phases, trial sites, or geographic regions.

[0013] The disclosed embodiments further provide a real-time autonomous creation and management system for dynamic electronic trial master files (eTMFs) for clinical trials. The system includes multiple integrated applications configured to manage various aspects of the clinical trial and generate signals based on clinical trial activities. The system further includes an event processor configured to receive and process signals to create and dynamically update the eTMF in real time. The system further includes a document repository configured to store and manage documents across multiple studies. The system further includes a document generation module configured to generate documents using templates and contextual data and populate the dynamic eTMF in real time. The system further includes a large language model (LLM) configured to interpret signals and assist in the automatic generation and organization of the dynamic eTMF. The system further includes an integration module configured to retrieve necessary documents from external and internal repositories.

[0014] An embodiment may include one or more of the following features.

[0015] The user interface can be configured to prompt users to upload temporary documents when needed. Event handlers can dynamically update the eTMF in real time based on received signals. Signals can be in the form of messages sent via a messaging system connected to the integrated application. This system can include modules configured to group placeholders into packages, each containing a set of placeholders required for a specific scenario within the clinical trial. A document generation module can use predefined templates to dynamically create documents based on contextual data. The document repository can include advanced data security measures such as encryption, access control, and audit trails. The Large Language Model (LLM) can continuously learn and improve its processes based on the data it processes and the feedback it receives. The integration module can support seamless data flow and real-time updates between different components of the clinical trial process. Placeholders and generated documents can be organized in a structured and logical manner to ensure compliance with regulatory requirements and audit readiness, and consistency with the organization of the reference model.

[0016] The disclosed embodiments provide a method for autonomously creating and managing an electronic trial master file (eTMF) for a clinical trial. The method includes integrating multiple applications to manage various aspects of the clinical trial and generating and receiving signals based on clinical trial activities. The method further includes receiving and processing signals to automatically generate and dynamically update a dynamic eTMF in real time. The method further includes generating documents using predefined templates and contextual data to populate the dynamic eTMF. The method further includes using a large language model (LLM) to interpret the signals and assist in the automatic generation and organization of the dynamic eTMF. The method further includes dynamically updating the eTMF in real time based on the received signals.

[0017] An embodiment may include one or more of the following features.

[0018] This method may include prompting users to upload temporary documents when needed. The method may further include grouping placeholders into packages, each package containing a set of placeholders required for a specific scenario within the clinical trial. The method may further include implementing advanced data security measures, such as encryption, access controls, and audit trails, in the document repository. The method may further include continuously learning and improving the LLM process based on the data processed and the feedback received. The method may further include storing and managing documents across multiple studies in the document repository. The method may further include retrieving necessary documents from external and internal repositories.

[0019] Disclosed embodiments may include creating an eTMF structure containing placeholders for artifacts based on received events (e.g., in the form of signals or event messages) and / or existing document plans. Placeholders in the eTMF structure may be populated based on analysis of key documents such as study protocols, FDA Form 1572, and clinical trial protocols. Content in the dynamic eTMF structure (e.g., e-learning certificates) may be created in real-time and automatically archived based on incoming signals (e.g., event messages). Content in the dynamic eTMF structure (e.g., master file annotation list) may be created in real-time and automatically archived based on the content of other artifacts already archived in the eTMF data store. For example, the configuration of application functionality in a site monitoring application may be automated based on artifact content archived in the eTMF data store, such as monitoring plans.

[0020] The disclosed embodiments provide a non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform the methods described herein. Attached Figure Description

[0021] Figure 1 This is a diagram of a system that autonomously generates and modifies dynamic electronic test master files in real time based on a disclosed embodiment.

[0022] Figures 2-1 to 2-2 This is a table showing examples of the results of analyzing documents using machine learning algorithms.

[0023] Figures 3-1 to 3-3 Describes the event handling limit screen that allows input of parameters used in event handling.

[0024] Figure 4 It is a sequence diagram depicting the message delivery of events related to arranging a visit.

[0025] Figures 5 to 7 It is a pattern used by event handlers to perform event processing via a messaging system.

[0026] Figure 8 It is a list of examples of message system events (or "topics") used by the event handler in event processing.

[0027] Figure 9 This is a flowchart of a method for autonomously generating and modifying dynamic electronic test master files in real time, based on a disclosed embodiment.

[0028] Where appropriate, reference numerals may be reused in the accompanying drawings to indicate corresponding or similar elements. Furthermore, some modules shown in the drawings may be combined into a single function. Detailed Implementation

[0029] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the invention. However, those skilled in the art will understand that embodiments of the invention can be practiced without these specific details. In other instances, some well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the invention.

[0030] Each clinical study uses an Electronic Trial Master File (eTMF) as its official documentation. Typically, the eTMF is a compilation of necessary documents collected during the clinical trial process, designed to facilitate clinical trial operations and document regulatory compliance. The document plan allows specifying the types of relevant documents (i.e., “artifacts”) that users should provide at each stage of the clinical trial. The document plan also allows defining specific levels within the eTMF structure, such as hierarchical file structures and metadata for documents at the product, trial, country, and trial site levels. For example, at the trial site level, the licenses of the principal investigators (PIs) participating in each site of the clinical study need to be collected. Based on the document plan, a hierarchical file structure and document “placeholders” for all active stages within the file structure are created. Contributors and authors create and import documents and associate them with placeholders. The eTMF system names, archives, and secures the documents.

[0031] To track the completeness of the eTMF, placeholders are typically used to indicate which documents need to be collected. Generally, "completeness" can be defined as the ratio between the placeholders and the completed documents (i.e., those uploaded and stored in the eTMF). Depending on the research stage, research type (e.g., interventional or observational studies, drug or device studies), regulatory requirements, and research risk level, different documents may need to be collected for different studies, countries, and sites, based on a reference model.

[0032] Configuring an eTMF typically takes weeks or even months and may require significant vendor support. Once configured, the eTMF requires ongoing manual maintenance throughout the clinical trial to keep it in a relatively static state, ensuring its timeliness and up-to-dateness. For example, sites may be added or removed, and adverse events (AEs) that need to be recorded in the eTMF may occur.

[0033] The embodiments disclosed herein introduce a novel eTMF method for clinical trials. Unlike most traditional, static, manual, and labor-intensive methods, the disclosed method provides a dynamic eTMF system configured to autonomously create and populate itself using various signals and data sources. This can begin with creating placeholders in a dynamic eTMF structure, which are then populated with actual documents. By utilizing the system's data and signals (e.g., in the form of messages), the dynamic eTMF can autonomously create placeholders and generate (and / or receive) and organize these documents in real time, thereby reducing the need for manual input and supervision. For example, at the initiation of a clinical trial, the system can automatically identify and create placeholders for key documents (e.g., study protocols) based on predefined criteria and signals.

[0034] This autonomous, dynamic approach aims to simplify the eTMF creation process, making it simpler and more convenient for users. Without detailed demonstrations and extensive manual setup, the system is offered as a ready-to-use solution. This significantly streamlines the startup process for new research, allowing users to focus on their core activities rather than the tedious administrative tasks of setting up and managing eTMFs.

[0035] Using initial data points, such as trial phase and whether a medical device is involved, the system can automatically begin generating an eTMF. Other signals can be collected in real time to dynamically refine and build the eTMF structure. Users can also be prompted to provide specific documents, such as the study protocol, which contains crucial information about study execution, participant criteria, and other procedural details. By integrating this information, the system can automate more aspects of dynamic eTMF management, thereby ensuring the comprehensiveness and orderliness of the trial documentation repository.

[0036] The system also integrates various applications, including those for electronic informed consent, adverse events, and research management, into a unified framework. Each application generates signals and data, which are processed to automatically and dynamically populate the eTMF in real time. This integration enables seamless data flow and real-time updates between different components of the clinical trial process.

[0037] Furthermore, the system employs a Large Language Model (LLM) to interpret complex data inputs and generate the necessary document structure. The LLM processes signals and data to create artifacts, ensuring that the dynamic eTMF comprehensively and accurately reflects the trial's needs in real time. Using LLM for data interpretation and document generation is an advanced approach to extracting clinical trial-relevant data from a variety of data sources.

[0038] The system also includes a centralized document repository, which allows documents to be stored once and reused in multiple studies. This repository ensures that documents such as medical licenses are always up-to-date and can be cited in any relevant study, thereby reducing redundancy and optimizing storage.

[0039] Overall, the autonomous eTMF generation and modification system offers a powerful and innovative technical solution to the challenges of clinical trial document management. By employing advanced automation technologies, integrating various applications, utilizing large language models, event-driven processing, and contextual analysis, the system provides specific technical improvements that enhance the efficiency, accuracy, and reliability of eTMF management.

[0040] Figure 1 This is a diagram of a system 100, referred to as an autonomous eTMF generation and modification system, according to a disclosed embodiment. The system architecture of the autonomous eTMF generation and modification system 100 includes various components and modules designed to automatically create and dynamically manage eTMFs in real time. In the illustrated example, system 100 has two main components: an application layer 105 and a dynamic eTMF system 110.

[0041] Application layer 105 represents various applications and services that contribute information and data to the dynamic eTMF system 110 in real time. The dynamic eTMF system 110 includes the eTMF itself and other components discussed in detail below. As system 100 receives this information in real time, it continuously and automatically processes the information to determine its impact on the structure and content of the dynamic eTMF system 110. In an embodiment, the dynamic eTMF itself may include an eTMF file structure 112 and an eTMF data storage 114. Application layer 105 provides information through "events," upon which the dynamic eTMF system 110 can take action.

[0042] An "event" refers to an interaction initiated by a user on a user interface at a clinical trial site or related institution and on a computer system, in response to an actual activity or milestone event associated with the clinical trial. This interaction generates electronic signals, messages, or notifications—called "event messages"—containing metadata related to the actual activity or milestone event, and is transmitted to an event handler for further automated processing. An "event" can correspond to, but is not limited to, activities such as confirming a site visit, submitting or approving documents, creating a subject record, or confirming a regulatory milestone. While events can originate from actual activities (e.g., completing a site visit, screening patients), the term specifically refers to computer-mediated interactions that record, process, and transmit data generated by user actions.

[0043] In addition to the event itself, contextual information exists, allowing system 100 to take actions such as utilizing a list of predefined data in the form of packages 115 and templates 120 as input to the dynamic eTMF system 110. The autonomous eTMF generation and modification system 100 effectively integrates various applications and related user cases to automate and streamline the collection and management of clinical trial information. Each application plays a specific role in the clinical trial process and provides data to the dynamic eTMF system 110.

[0044] These applications contain the information required by the dynamic eTMF system 110 in different formats and through various mechanisms. For example, some data is stored in a database, while other information may be available in the form of directly searchable files. The challenge lies in enabling system 100 to efficiently extract information and integrate it into the dynamic eTMF system 110. By using innovative methods and mechanisms to generalize and streamline these processes, system 100 can help ensure the accurate and efficient collection of all necessary data.

[0045] The disclosed approach aims to create efficient, automated systems where each application can seamlessly participate in the construction and management of the dynamic eTMF system 110. This involves designing streamlined processes that facilitate easy integration and data retrieval from various applications. By doing so, the system can minimize human intervention, reduce errors, and help ensure the comprehensiveness and timeliness of the eTMF.

[0046] Master Data Management Application

[0047] One application in application layer 105 is master data management application 125, which is responsible for the overall definition of research, including the sites and countries where the research is conducted. It also manages research-related users, including site-level and sponsor-level users. Master data management application 125 generates signals such as research creation, site addition or removal, and user addition or removal. These signals are important for maintaining an up-to-date and accurate eTMF structure.

[0048] For example, the functionality of the master data management application 125 can be used to define site teams. For instance, suppose a clinical research coordinator is added for a study. In this case, this person's CV must be provided to the dynamic eTMF system 110, possibly along with other types of data, such as a medical license. When the master data management application 125 notifies the dynamic eTMF system 110 of this addition (and provides contextual information about the associated study and site), the dynamic eTMF system 110 can access predefined packages 115 that specify the required document placeholders. In embodiments, these packages 115 include contextual information, as a specific site in a particular country may require a different set of placeholders. The event handler 130 matches the incoming event information against the package metadata to determine which package to apply.

[0049] In this embodiment, when a new clinical trial site is added or removed, or when a user joins or leaves a study, the master data management application 125 automatically and dynamically updates the eTMF (via the dynamic eTMF system 110) to reflect these changes. This automation ensures that the eTMF maintains a reliable and comprehensive repository of all trial-related documentation.

[0050] The Master Data Management application 125 is just one example of the many components that make up the Autonomous eTMF Generation and Modification System 100. By integrating these applications, System 100 can automatically generate and organize necessary documents, significantly reducing manual intervention and improving the efficiency and accuracy of clinical trial management. This approach not only ensures compliance but also streamlines the entire process, making it more user-friendly and efficient.

[0051] e-learning management application

[0052] Another application in application layer 105 is e-learning management application 160. In clinical studies, all personnel at the site level must undergo specific training. e-learning management application 160 generates signals indicating when these personnel are in training and sends these signals to event handler 130, prompting the creation of placeholders for corresponding training certificates. Once training is complete, system 100 can utilize a combination of signals and APIs to retrieve the actual training certificates. These certificates are then archived into appropriate placeholders and marked as satisfactory.

[0053] The process of using actual documents to fulfill placeholders is an integral part of maintaining the integrity of the eTMF. For example, when a user completes training, event handler 130 must retrieve the training certificate. System 100 is designed to understand how to handle various events and know what actions should be taken to obtain the necessary documents. Once a document is archived into a placeholder, the placeholder is marked as complete, ensuring the eTMF is up-to-date and complete. The system's use of placeholders and automated document processing, such as the e-learning management application 160, embodies a key aspect of the autonomous eTMF generation and modification system 100.

[0054] Bad Event Handler

[0055] Another application in application layer 105 is the adverse event processor 135. This application is used to record and manage any adverse events (or safety incidents) that occur during clinical trials. When an adverse event occurs, such as contamination or clarification, all relevant parties must be notified and the necessary processes to resolve the issue must be initiated. The adverse event processor 135 ensures that these adverse events are fully recorded and communicated. This may include a detailed description of the nature of the event, actions taken to mitigate its impact, and communications sent to other sites. Adverse events are an important component of clinical research because they help understand potential risks and ensure that research complies with regulatory requirements for safety and reporting.

[0056] For example, if a subject develops an unexpected illness due to the drug being tested, the adverse event processor 135 will log all necessary details. This logging is required for regulatory compliance because it allows agencies such as the FDA to review how such events are mitigated and communicated. System 100 helps ensure that information generated by this module is correctly and automatically added to the eTMF via the dynamic eTMF system 110. It is also important that the logged documents are distributed to all relevant sites. In an embodiment, system 100 can automatically distribute individual artifacts (e.g., adverse event reports) from the site reporting the safety incident to other trial sites within a specific study.

[0057] Electronic informed consent management application

[0058] Another application in application layer 105 is the electronic informed consent management application 140, which manages patient informed consent forms. This application generates signals and information that can be processed by event handler 130 to be automatically fed to the dynamic eTMF system 110.

[0059] Temporary document processor

[0060] System 100 includes a temporary document processor 145 for receiving temporary documents. For example, a user might be prompted to upload a research proposal. The system then uses machine learning algorithms to analyze the document to extract necessary information, which is subsequently used to populate relevant sections of the eTMF. This automated analysis and population process reduces the need for manual data entry and ensures that the eTMF always contains accurate and up-to-date information. In an embodiment, a user interface can be configured to prompt the user to upload a temporary document when specific information is needed but cannot be automatically generated by the system. This proactive approach ensures that all necessary documents are collected, even if manual user input is required.

[0061] In an embodiment, when a research program becomes available, machine learning algorithms in document analyzer 150 can be used to extract information about the trial, such as the trial phase, whether by an application providing event information about availability or by directly importing it into dynamic eTMF system 110 (e.g., via temporary document processor 145). This extracted information is then sent to event processor 130 to determine, for example, whether placeholders for safety reports and regulatory approvals are required in the eTMF. In some cases, event processor 130 may also send events (e.g., in the form of signals or messages) to one or more applications in application layer 105 based on the extracted information. For example, information such as the phase of a trial can be sent to e-learning management application 160 to facilitate the implementation of the required training. In an embodiment, the extracted information can be sent to external applications and / or systems. For example, information extracted from the “activity schedule” section of a research program can be sent to an electronic data capture (EDC) application.

[0062] As described above, system 100 utilizes package 115 and template 120 to efficiently build and manage eTMF. Package 115 can be described as a collection of placeholders. These placeholders are predefined slots within the eTMF used to indicate where a specific document needs to be placed. For example, package 115 might be created to contain placeholders for documents required when adding a new research site. The package can contain placeholders for various documents associated with the new site.

[0063] Similarly, another package 115 can be designed for situations where a new principal investigator is assigned to a site. This package will contain placeholders for the necessary documents associated with the new investigator. By grouping these placeholders into packages, the system ensures that all required documents are systematically organized and easily accessible within the eTMF.

[0064] Templates play a crucial role in generating eTMF content. When documents need to be created based on stored data, templates are used to format and organize that information. For example, in the context of e-learning, if data shows that a user has completed training, a template can be used to generate certificates or other relevant documents. This template, combined with event handler 130, creates actual documents such as PDFs, which are then archived into appropriate placeholders within the eTMF.

[0065] Document Library

[0066] In one embodiment, system 100 may include a document repository 175 for managing and storing documents. Document repository 175 is used to maintain a centralized storage system that can be used across multiple studies. This ensures that documents frequently used in different studies only need to be stored once and reused as needed, thereby improving efficiency and consistency.

[0067] For example, suppose a hospital is conducting multiple clinical studies led by the same principal investigator. The physician's license remains unchanged across these studies. By storing the physician's license in a document repository, the system can easily access and use the document for any study involving that researcher. This avoids the redundancy of repeatedly collecting the same document for each new study.

[0068] In this embodiment, document repository 175 can be connected to both external and internal repositories, enabling integration and data retrieval from various sources. This integration allows the system to automatically retrieve necessary documents from these repositories, further reducing the need for manual intervention. External repositories may include regulatory databases or other institutional repositories, while internal repositories may be part of an organization's document management system.

[0069] Figures 2-1 to 2-2 This is a table showing examples of the results of analyzing documents using machine learning algorithms. Figure 2-2 yes Figure 2-1 (Continued from the previous image). In this example, the analysis of the research protocol document provides several potential placeholders for the eTMF, such as: Protocol - Artifact 01.01.03; Protocol Revision - Artifact 01.01.04; Informed Consent Form (ICF) - Artifact 05.01.02; Case Report Form (CRF) - Artifact 01.04.01; FDA Form 1572 (Investigator Statement) - Artifact 05.02.03; and Authorization Record - Artifact 02.01.06.

[0070] Generating placeholders based on the analysis of documents such as research protocols can lead to the analysis of other documents, such as the document corresponding to the generated placeholder. For example, if the analysis of the research protocol generates a placeholder for FDA Form 1572, then the analysis of that form (see...) Figures 2-1 to 2-2The second row of the table can generate more placeholders, such as: 01.01.01 CV for principal investigator; 01.01.02 GCP training certificate; 02.01.02 list of associate investigators; 02.01.01 list of site contacts; 05.02.04 financial disclosure form; 05.02.01 IRB approval; 05.02.02 IRB correspondence; 02.01.06 authorization record; 06.02.01 site initiation report; 02.01.07 training record; 05.01.01 clinical trial agreement; and 05.03.02 FDA correspondence.

[0071] The system aims for a high degree of automation, making the eTMF largely autonomous. Various information sources from clients (i.e., users), such as initial sales orders, can trigger the eTMF creation process. Each document in the eTMF, such as the study protocol, provides specific information for the trial documentation. The system is designed to ensure that all necessary documents are automatically generated and organized, significantly reducing manual workload and improving the efficiency of clinical trial management.

[0072] In this embodiment, system 100 incorporates a machine learning model / algorithm, such as a large language model (LLM), to generate the structure of the eTMF using signals and artifacts. Various types of machine learning models can be used in the system, but a typical implementation uses an LLM integrated with event processor 130. This integration is designed to identify signals that guide the creation and organization of artifacts within the eTMF.

[0073] For example, signals such as "one study," "five sites," "US only," or "Class I medical diagnostic device" are extracted from various inputs. These signals form the basis for determining the specific requirements of the eTMF. Each signal represents a piece of information that helps to construct the overall structure and content of the eTMF. The system uses these signals to guide the generation of artifacts, which are the individual documents and data points that populate the eTMF.

[0074] Research Management Applications

[0075] Another application in application layer 105 is the research management application 155, also known as the Clinical Trial Management System (CTMS). This application helps to oversee the entire clinical study. In the research management application 155, various aspects of the study are managed, including, for example, the selection of study sites, the visibility of these sites, and the definition of study milestones. These milestones are particularly important for the dynamic eTMF system 110 because they determine the timeliness of submission of various documents. Since clinical studies typically last for several years, not all documents need to be submitted at the start of the study. When a milestone is reached, the research management application 155 sends an event to the event handler 130, which then updates the eTMF with the necessary placeholders and the documents that need to be prepared and uploaded.

[0076] During the research execution phase, the research management application 155 is also responsible for site monitoring. Pharmaceutical companies and life science institutions collaborate with research sites to ensure compliance with research protocols. This includes regular site visits to verify compliance and the generation of relevant documentation during these visits. Each site visit is recorded in the eTMF, including details such as the visit date, participants, and visit results. These site monitoring activities generate multiple documents, which are integrated into the eTMF by sending events to the event processor 130. In this embodiment, a separate site monitoring application 165 may exist.

[0077] The research management application 155 is responsible for generating the majority of the thousands of documents added to a typical eTMF during the clinical research process. These numerous documents are crucial for maintaining regulatory compliance and ensuring the integrity of the clinical trial process. By automating the generation and management of these documents, System 100 significantly reduces manual workload and improves the efficiency and accuracy of trial documentation.

[0078] Table generation

[0079] In one embodiment, system 100 may include a form generation module 170 to automatically generate forms to be stored in the eTMF based on information from other sources, such as electronic data capture (EDC) systems. This feature enables the system to automatically generate the necessary forms, thereby further reducing manual workload and ensuring the accuracy and consistency of documents.

[0080] For example, in clinical research, various patient forms need to be created and completed. These forms are designed to collect specific data from patients during their visits, such as blood pressure, weight, height, and other relevant health indicators. The study design outlines the required data points, and the resulting predefined forms are an integral part of the clinical trial process. When patients come for their visits, the system uses these predefined forms to ensure that all necessary data is collected. These forms are tailored to the specific requirements of the study, ensuring the accurate collection of all relevant information.

[0081] In this embodiment, system 100 can automatically generate these patient forms based on data received from other sources, following a predefined table design. By automatically creating these forms, the system helps ensure that no important data points are missed and that the table format always conforms to research specifications. This automation not only improves the efficiency of data collection but also enhances the overall quality and reliability of clinical trial documentation. Notably, in addition to the forms containing patient data, blank forms are typically submitted for storage in the eTMF for documentation of their structure and editing checks.

[0082] Event handlers / Event processing

[0083] Figures 3-1 to 3-3 The event handling definition screen depicts the parameters that can be used in the event handling process. Figure 3-2 and Figure 3-3 yes Figure 3-1 (Continued from previous diagram). As shown in this example user interface, a user can configure event handling for specific incoming events. In this example, the event refers to the notification sent by the master data management application 125 indicating that a research team member has been added. The user can define the event handler 130 to use when it receives a notification of this specific event.

[0084] To define event handling, the user enters an event name, such as "Add a new PIT to the research event," and a trigger selected from dropdown menus, including: "Add Country," "Complete e-learning," "Add Protocol," "Add Site," and "Add Research Team Member." Additionally, the user enters the "eTMF context" to specify which products (e.g., drugs), studies, countries, and / or sites this handling applies to. For example, the required event handling for a particular event may differ across different types of studies, such as one type of study investigating pediatric drugs versus another investigating adult drugs. Furthermore, the handling requirements for the same event may differ for a specific site in a specific country; for instance, adding research team members might require different document sets.

[0085] Users specify "roles" associated with event information, where each event can provide additional information. In this example, we focus on the case where a research team member is designated as the principal investigator. In this embodiment, each event pre-registers metadata about the event and the additional information it will provide with the system. The user interface in this example displays the specific information registered for the "Add Research Team Member" event.

[0086] The user specifies the action to be performed for a specific event and role. In this case, our target is the "application package," which refers to a set of placeholders. Based on the selected action, the user can specify the "eTMF package," which is the name of an available package.

[0087] The "Attribute Mapping" section of the user interface allows control over how various attributes or metadata (pre-registered) associated with an event are mapped to the attributes or metadata available in the package for the placeholder. For example, "Doctor's Name" can be directly mapped, but this would not be the case if the event used "Research Team Member Name". The "Placeholder Name" section of the user interface refers to the placeholders derived from the package and how they are named. In this example, the "Doctor's Name" value (e.g., "James") from an incoming event (e.g., "James Johnson") will be populated into the "Doctor's Name" attribute (i.e., metadata) of the placeholder. This value will then be used as part of the placeholder name (e.g., "Research Team Member Resume James Johnson").

[0088] In an embodiment, specific artifacts or documents exist within the eTMF, which include a list of other artifacts or documents available within the eTMF. For example, "FileNote" is an artifact or document present in every region of the eTMF to record decisions or clarify information related to that region. Additionally, there exists a "FileNote Master List Artifact (01.01.05)". In this case, this artifact is automatically generated whenever a FileNote is added to a given region. When receiving a... Figure 1 This automation occurs when a temporary document is displayed and "Document Analysis" is performed. This analysis creates a "FileNote Add" event, which can have a handler to update the FileNote master file list. Therefore, adding a document or placeholder can trigger other processes in eTMF.

[0089] Figure 1 The connections shown can represent bidirectional data flow between the dynamic eTMF system 110 and the application layer 105, at least in some cases. This is because documents can enter the eTMF, containing information that other applications may need. For example, a monitoring plan (artifact 01.01.08) contains information required by the Clinical Trial Management System (CTMS). In this case, the dynamic eTMF system 110 can issue an event indicating that the monitoring plan is available, thereby allowing the CTMS to respond by automatically configuring the monitoring system.

[0090] A monitoring plan is a document that outlines how monitoring will be conducted during a clinical trial, including strategies for validating raw data. This artifact may include any evidence of plan execution, such as the plan itself, reports, checklists, and other relevant documents. Typically, the monitoring plan is one of the plans managed by the clinical operations department (clinops). However, the document is not usually stored in the CTMS. Instead, it serves as a reference when setting parameters such as monitoring visit rates and visit types. The monitoring plan is usually created before the study can be performed in the CTMS, and there may not be a convenient repository outside the eTMF to store the document. In some cases, a link may be provided in the CTMS to facilitate easy access to the document in the eTMF.

[0091] Since CTMS is configured based on the content of the monitoring plan, and clinical research assistants (CRAs) frequently refer to the monitoring plan, recreating documents based on the settings in CTMS may not be desirable. Therefore, in practice, users can utilize a combination of document tracking and editing functions to create monitoring plans. Thus, as described above, according to the disclosed embodiments, configuring the system for studies in CTMS based on documents stored in the eTMF may be more efficient.

[0092] Use of TMF reference model

[0093] As described above, a typical implementation uses an LLM (Liquid Management Model), integrated with an event processor 130. This integration aims to identify signals that guide the creation and organization of artifacts in the eTMF. Part of this process involves mapping signals to specific artifacts. In some cases, if there is a direct correlation between signals and artifacts, this mapping may be straightforward. However, in other cases, the mapping may be more complex, requiring machine learning / artificial intelligence techniques to accurately interpret and apply the signals. Using an LLM enables the system to process these signals and generate the necessary artifacts, thereby ensuring the integrity and correct structure of the eTMF.

[0094] To facilitate this process, System 100 may rely on a detailed TMF reference model, which contains all available artifacts and the corresponding signals required to generate them. For example, System 100 may use a reference model maintained by the Clinical Data Exchange Standards Consortium (CDISC), which is an industry standard. This structured approach ensures that the system can accurately and efficiently create eTMFs based on the provided data.

[0095] Typical considerations for generating eTMF

[0096] As mentioned above, traditional eTMF systems require manual creation of a placeholder eTMF structure after customization, which can take weeks. In contrast, the embodiments described herein allow for the automatic configuration of a functional eTMF based on various input parameters that progressively populate the structure.

[0097] For example, the system considers the trial phase. Early trials (Phase I) may require less documentation than later trials (Phase III), especially in terms of security reporting and regulatory approvals. This distinction ensures that the eTMF can be tailored to the specific requirements of each trial phase, thereby streamlining the documentation process.

[0098] Trial type is another key parameter. For example, interventional trials for new drugs typically require a more comprehensive documentation set than observational studies because observational studies may involve fewer direct participant involvement and less regulatory review. For non-drug trials, such as medical device trials or behavioral interventions, certain documents, such as investigator's brochures, may not be applicable. By identifying these differences, the system ensures that only relevant documents are included in the eTMF, thereby improving efficiency.

[0099] Regulatory requirements vary across regions, such as the FDA in the US, the EMA in the EU, and the PMDA in Japan. Specific documentation required will depend on the jurisdiction; for example, some studies, such as non-interventional studies or post-market surveillance, may not require such stringent documentation. The system automatically adapts to these regional differences, ensuring compliance with local regulations without requiring manual customization.

[0100] The risk level of a trial also affects documentation requirements. For example, low-risk trials investigating approved therapies in new populations may have fewer documentation requirements. Conversely, high-risk trials involving new drugs or gene therapies require more comprehensive documentation, including additional safety reports. By tailoring the eTMF structure to the risk level, the system ensures that all necessary documentation is included without adding unnecessary paperwork to low-risk trials.

[0101] In summary, the autonomous eTMF generation and modification system described in this paper represents a significant improvement over traditional methods by automating the creation and population of eTMF structures. By considering parameters such as trial phase, trial type, regulatory requirements, and risk level, the system ensures that the eTMF meets the specific needs of each clinical trial. This automated and dynamic approach improves efficiency, accuracy, and compliance, providing a robust technical solution to the challenges of clinical trial documentation management.

[0102] Generation of real-time event-driven eTMF structures

[0103] In this embodiment, when defining a study, the user first sees a minimal eTMF structure created. Subsequently, placeholders and content are added based on various events from other applications integrated with the eTMF. This dynamic, event-driven approach helps ensure that the eTMF evolves in real time, reflecting the latest developments and requirements of clinical trials.

[0104] Handle events through event handlers

[0105] Refer again Figure 1 Event handler 130 is configured to process a set of events generated by research management application 155, such as Figure 8 The events listed herein. In embodiments, these events may include milestone events, such as “REPORT_INITIALIZED,” “VISIT_CONFIRMED,” and “CONFIRMATION_LETTER_INITIALIZED.” These events are published by a messaging system 132 (e.g., Kafka) and processed by an event handler 130 to dynamically create placeholders, manage document workflows, and update trial progress in real time. For example, when the “VISIT_CONFIRMED” event occurs, placeholders for confirmation letters and related visit documents will be automatically generated in the eTMF file structure 112. This streamlined process helps ensure that all relevant artifacts are prepared and managed according to pre-defined protocols and regulatory requirements. By utilizing, for example, Figure 8 The predefined event types in the research management application 155 shown in the system support comprehensive coverage of trial activities and documentation requirements, minimize delays caused by manual operations, and help ensure that all operations are ready throughout the trial lifecycle.

[0106] like Figure 4 As shown, the interactions between the various system components highlight how an event-driven architecture accelerates milestone-driven placeholder generation and document management workflows. For example, when a user initializes a visit document through the research management application 155, a "Visit document initialized" message is published to the messaging system 132 (e.g., Kafka), as... Figure 4 The first sequence is shown. This event triggers event handler 130 to apply the appropriate package corresponding to the visit type category and dynamically create the necessary placeholders in eTMF file structure 112. Additionally, when the user finalizes or approves the visit document (e.g., via a "Finalize" or "Approve" operation in CTMS), such as Figure 4 As shown in the second sequence, the finalized document is uploaded to a cloud service (e.g., S3), and a "View document changes" message is published to messaging system 132. Event handler 130 retrieves the finalized document from S3, archives it into dynamic eTMF system 110 (using associated placeholders), and overwrites previous documents where applicable. This interaction helps ensure real-time updates of eTMF placeholders while maintaining auditing and version control capabilities.

[0107] Figures 5 to 7 This is the mode by which event handler 130 performs event handling via message system 132. Event handler 130 uses messages such as "apply-package" ( Figure 5 ) and the "add-doc-to-PH" message ( Figure 6 This model facilitates advanced integration between milestones, placeholders, and document completion workflows. For example, when a "REPORT_SUBMITTED" event occurs for a visit report, event handler 130 ensures that the corresponding placeholder in the eTMF file structure is dynamically updated and the document is accurately positioned. In embodiments, for example, Figure 7 The document confirmation mode, using the "return what was received" pattern, confirms that a document has been successfully stored and associated with the correct placeholder. This robust event-driven mechanism ensures real-time updates for large-scale distributed experiments, thereby improving data integrity and operational efficiency. This is achieved by managing parameters such as "VISIT_START_DATE_REACHED" and "CONFIRMATION_LETTER_INITIALIZED" (see...). Figure 8 By identifying dependencies between events, the system implements fine-grained placeholders that reflect the actual progress of the experiment, thereby helping to ensure that the documentation workflow remains accurate and ready for auditing even in experiments with complex timeliness and diverse documentation requirements.

[0108] In this embodiment, the use of dynamic event dependencies enables the system to handle phased documentation workflows more efficiently. For example, a “VISIT_CONFIRMED” event triggers the generation of placeholders for visit scheduling and initial documentation, while subsequent events such as “VISIT_START_DATE_REACHED” dynamically add placeholders for security reports, monitoring logs, and site-specific regulatory filings. By managing these interrelated dependencies across multiple trial phases, the system ensures that placeholders are created on demand, rather than pre-created, reducing documentation clutter and providing focused compliance tracking for specific phases. This phased approach aligns with operational requirements and ensures that documentation is updated in sync with trial progress.

[0109] Event-driven architecture facilitates collaborative management of the eTMF across multiple trial stakeholders. For example, when document placeholders (e.g., via “file-to-eTMF” messages) are populated, the system updates other integrated systems in real time, such as the research management application 155 or the adverse event handler 135. These updates ensure synchronization between systems and provide stakeholders with access to the latest information. Additionally, event triggers such as “VISIT_START_DATE_REACHED” or “CONFIRMATION_LETTER_FINALIZED” automatically notify and update relevant teams, promoting seamless collaboration and reducing delays in document workflows. This integrated, event-driven approach enhances transparency and coordination among trial personnel, regulatory teams, and external partners, helping to ensure the integrity of the eTMF and its real-time alignment with trial milestones.

[0110] In this embodiment, the system utilizes machine learning techniques to enhance event processing and document management workflows. For example, event processor 130 can use machine learning algorithms to analyze historical and real-time event data (see [link]). Figure 8 This allows for the prediction of potential delays in placeholder creation or document submission. If a specific event type, such as "VISIT_CONFIRMED," has historically caused delays in the creation of confirmation placeholders, the system pre-allocates resources to address this gap. Furthermore, machine learning integration ensures that placeholders requiring document localization (e.g., language-specific versions for global sites) are dynamically tagged and adjusted in future workflows. The application of machine learning facilitates continuous learning and improvement of system workflows, enabling adaptive automation that evolves in sync with experimental conditions and regulatory requirements.

[0111] Document planning and placeholder creation

[0112] Figure 5 An example of a “apply-package” message pattern is provided, which is used to create eTMF placeholders when a relevant event is detected in the study management application 155. For example, this message may include attributes such as placeholder category, study context, event type, and application source, helping to ensure consistency between the created placeholders and the triggered clinical trial milestones. Once the message is processed by the event handler 130, a hierarchical placeholder structure is dynamically built in the eTMF file structure 112. When finalized or updated documents need to be archived into these placeholders, the event handler utilizes the “add-doc-to-PH” (or “file-to-eTMF”) message (see [link to event handler]). Figure 6 This ensures that documents are placed accurately. This approach reduces human error and supports version control by replacing outdated files where applicable.

[0113] In this embodiment, the system optimizes document plan creation and placeholder generation by adjusting the eTMF structure to align with the experimental phase, such as... Figure 4 The workflow is illustrated in the documentation. For early-stage trials (e.g., Phase I), the system can limit placeholders to basic security and protocol documentation; while for later-stage trials (e.g., Phase III), the system can generate additional placeholders for complex regulatory documents, expanded site reports, and final study results. This phase-specific approach ensures that the eTMF evolves dynamically as the trial progresses, minimizing placeholder clutter in early stages while handling the increasing demands of later-stage trials.

[0114] To meet a wider range of testing needs, the system includes a rich library of event types that can trigger placeholder generation and document updates. Figure 8 This is a list of examples of message system events (or "topics") used by event handler 130 in event processing. These include events such as "REPORT_INITIALIZED", "VISIT_START_DATE_REACHED", and "CONFIRMATION_LETTER_INITIALIZED". These events enable fine-grained tracking of experiment progress, and each event is pre-configured to generate a specific placeholder via event handler 130. This is achieved by referencing events such as "apply-package" (…). Figure 5 ) and "add-doc-to-PH" Figure 6 The system provides sponsors with customizable document management methods based on unique trial workflows, including various modes such as [list of modes]. These updates ensure timely fulfillment of document requirements while supporting multi-layered tracking and compliance monitoring during trial execution.

[0115] In the embodiment, placeholder creation uses methods such as Figures 4 to 8 The messaging framework shown is progressively adjusted based on trial milestones. For example, during the trial initiation phase, placeholders for site initiation files and principal investigator credentials are generated based on initial events such as "SITE_ADDED" or "PI_ASSIGNED". Later in the trial, as milestones such as "PATIENT_ENROLLED" or "SITE_CLOSEOUT" are achieved, additional placeholders, such as patient safety reports, end-of-visit reports, or final monitoring records, are progressively created. This progressive approach ensures that placeholders remain relevant to the current trial phase, reducing the complexity of managing unnecessary placeholders early on. By dynamically adjusting the eTMF structure in response to milestones, the system minimizes documentation clutter and improves organizational efficiency throughout the long trial period.

[0116] In this embodiment, a time-based process is integrated into the system to ensure that placeholders are created and filled in in accordance with the critical risk points of the experiment. For example... Figure 4 As shown, the message workflow ensures that placeholders associated with high-risk milestones such as "PATIENT_SCREENED" or "VISIT_REPORT_FINALIZED" are dynamically generated as the event occurs. This timing mechanism prevents premature creation of placeholders (which could lead to the eTMF being prematurely flooded with unnecessary placeholders) while helping to ensure that the correct documents are declared as applicable at the correct time during clinical trials. This approach mitigates common problems in trial documentation management, such as the illusion of incomplete documentation in long-term studies due to prematurely anticipating too many documents.

[0117] In this embodiment, the event handler dynamically adjusts placeholder generation and documentation workflows based on phase transitions. For example, during the transition from Phase I to Phase II, milestone events such as “PHASE_I_COMPLETE” can trigger automatic updates to the eTMF structure, including new placeholders for expanded regulatory filings, mid-term security analyses, and updated scenarios. These adaptive triggers ensure that placeholders are created and documentation is managed only when necessary, reducing redundancy and keeping eTMF operations aligned with the evolving needs of the trial phases. This feature reduces manual adjustments during phase transitions, ensures regulatory compliance, and maintains operational efficiency across multiple phases of the trial.

[0118] In this embodiment, the system employs role-based dynamic document management, utilizing, for example... Figure 4 The event handler 130 dynamically generates placeholders associated with each role's responsibilities when adding roles such as "Research Coordinator" or "Primary Researcher" for a specific site, and sends notifications for specific tasks to designated personnel. By automatically assigning document management tasks based on roles, the system reduces administrative overhead and ensures that experimenters focus on actionable matters relevant to their specific sites and roles.

[0119] In this embodiment, the system facilitates seamless interaction with external document repositories to efficiently populate placeholders. For example, when triggered by an event like “TRAINING_ASSIGNED,” the event handler creates a matching placeholder for the assigned person. Upon a subsequent “TRAINING_COMPLETED” event, the corresponding training credentials (e.g., training course certificates) can be downloaded from an external source (e.g., a third-party training institution) via a secure API. The system then archives these documents into appropriate pre-created placeholders within the dynamic eTMF system 110. This integration significantly reduces the manual workload of document retrieval and improves synchronization with external collaborators, making it easier to maintain a complete and compliant eTMF.

[0120] Document processing

[0121] In this embodiment, the document validation layer ensures that the document conforms to predefined criteria after placeholder padding. For example, when via, for instance... Figure 4 When uploading documents in the workflow shown, the system cross-checks the document format, content, and consistency against the relevant placeholder metadata. For example, verification errors such as missing signatures or incomplete attachments can be flagged, and notifications can be sent to relevant personnel via the research management application 155. This verification process helps ensure that placeholders are not marked as complete unless the uploaded document complies with regulatory and operational standards. By integrating document verification directly into the placeholder filling workflow, the system helps prevent incomplete or non-compliant documents from disrupting trial timeliness or jeopardizing regulatory preparation. This verification layer, combined with an event-driven workflow, establishes a robust mechanism to ensure the quality and accuracy of submitted documents while maintaining traceability throughout the eTMF lifecycle.

[0122] The system can employ intelligent document classification using machine learning models and event-driven signals. For example, when placeholders are populated via a "file-to-eTMF" message (see...). Figure 6 The system analyzes document metadata, such as type, author, and related milestones, and categorizes them into the corresponding directories in eTMF structure 112.

[0123] Figure 4 The bidirectional message flow shown illustrates how the system provides a continuous, audit-ready feedback loop from placeholder creation to document archiving. For example, as... Figure 4 As shown in the second sequence, once a document is successfully stored in the eTMF system, the system generates a "return" message, which is routed through the messaging system 132 to confirm the document identifier (FSI-ID) and document version. This message flow helps ensure that the research management application 155 database reflects the latest document status. Real-time confirmation of document archiving status enables seamless collaboration between site personnel, regulatory teams, and trial sponsors, helping to ensure that placeholders for high-priority trial milestones are populated in a timely manner.

[0124] The sequence diagrams and patterns described herein demonstrate the system's ability to provide integrated feedback on the status of research documents and placeholders. For example, when a milestone triggers a file upload (e.g., "REPORT_SUBMITTED" or "VISIT_START_DATE_REACHED"), the feedback loop ensures that placeholders are accurately updated and marked as populated, thereby reducing inefficiencies in document tracking and submission. Event handler 130 can interactively retrieve, validate, and store documents, enabling trial sponsors to continuously monitor compliance, while the feedback mechanism ensures end-to-end consistency between messaging system 132, research management application 155, and dynamic eTMF system 110.

[0125] The event workflow described herein enhances collaboration between integrated applications by facilitating seamless communication of placeholder states and document updates. When an event like “TRAINING_ASSIGNED” or “TRAINING_COMPLETED” occurs in the research management application 155, the event handler dynamically updates the eTMF while synchronizing with related applications such as the bad event handler 135. Furthermore, as for example… Figures 4 to 8 As shown, the fine-grained reporting built into messaging system 132 enables stakeholders to track document status with high precision. For example, when a visit report is updated, the system sends a "Return Received Content" message (see...). Figure 7 Real-time status updates are generated in the form of a report, containing detailed information about the document's current status, placeholder assignments, and version history. This feedback loop helps ensure that document updates are reflected in all integrated applications, including the research management application and the user interface. This enhanced reporting capability gives trial sponsors and site administrators a clear understanding of the document workflow, thereby improving overall operational transparency and reducing the risk of incomplete or outdated content.

[0126] In this embodiment, the document version control function tracks updates in real time. For example, when a placeholder in a visit report is filled with a new version of the document, the system overwrites the old version and can archive it for compliance purposes. For example, according to... Figure 7 This model, which sends back confirmation messages, provides users with traceable evidence of the version control process, helping to ensure that regulatory bodies can access the accurate document history during audits. This extended version control system enhances oversight, ensures compliance with regulatory standards, and reduces the risk of errors associated with outdated or duplicate documents.

[0127] For example, Figure 4The document version control and linking workflow shown in the second sequence enables the system to maintain real-time consistency between placeholders and uploaded documents. For example, after the event handler retrieves and archives the finalized document into the eTMF system, the system automatically verifies the document version and generates an "FSI-ID and Version Return" message, such as... Figure 4 As shown. This integration prevents recurring errors and ensures that experimenters can directly access the latest version of archived documents via hyperlinks generated by the research management application 155 (as described below in "Hyperlinks to eTMF"). These improvements enhance the system's ability to maintain comprehensive, easily accessible, accurate, and regulatory-compliant version-controlled documents.

[0128] In this embodiment, the system integrates workflow-based version control and detailed audit trails for event tracking, such as... Figure 4 As shown, each uploaded, updated, or replaced document can record timestamps, relevant milestones, and event metadata. By maintaining a complete history of placeholder interactions, such as creation time, population time, and linking time to the final document, the system ensures that regulatory agencies have traceable evidence of compliance during audits. This audit trail, combined with a dynamic feedback verification mechanism, provides trial sponsors with a comprehensive understanding of trial operations, thereby helping to ensure compliance and enabling rapid response to external audits or internal inquiries.

[0129] Risk- and prediction-based event handling

[0130] The system supports multi-tiered event processing based on risk-based milestones, leveraging specific event types (e.g., see...). Figure 8 This includes its integration with placeholder workflows. For example, when a high-risk event such as “ADVERSE_EVENT_REPORTED” is triggered, the system creates placeholders for security documents at the research and site levels. Parallel workflows ensure that the required placeholders are created simultaneously at all sites, helping to prioritize critical events impacting patient safety and regulatory claims. This multi-layered integration combines milestone scheduling with compliance-critical tasks, enhancing system reliability and enabling it to better handle complex trial operations.

[0131] Localization strategy execution and workflow customization

[0132] The system equips event processor 130 with localized policy execution capabilities that meet region-specific milestone requirements. For example, placeholders generated for German clinical trials may contain specific metadata and documentation requirements tailored to national and / or regional laws and regulations, which are specified during placeholder creation (e.g., see...). Figure 5 ) and during the document archiving workflow (for example, see Figure 6This process is automated. Similarly, for US sites, placeholders conform to FDA-specific regulatory standards, helping to ensure compliance tasks are performed in the applicable jurisdictions. By dynamically aligning placeholder packages with geographic regulations, the system ensures global compliance while avoiding redundant or irrelevant documentation. This precision reduces manual configuration, enhances eTMF's adaptability to multi-regional trials, thereby improving setup efficiency and minimizing compliance errors.

[0133] Using, for example Figures 4 to 8 The workflows, patterns, and messaging system topics illustrated demonstrate how system implementations dynamically support region-specific workflow customization based on localized regulatory requirements. For example, events like "SITE_ADDED" or "PI_ASSIGNED" trigger the creation of placeholders that conform to applicable jurisdictional standards. This regional customization ensures global compliance, simplifies auditing processes for international trials, and reduces the manual adjustments required by trial coordinators to address different regulatory environments.

[0134] Advanced site monitoring, metrics, and notifications

[0135] The system allows sponsors to view the overall population of all participating sites to track placeholder progress. This is achieved using event messaging workflows and population confirmation messages (see, for example, see...). Figure 4 Sponsors can access a centralized dashboard to see detailed information on filled, pending, or overdue placeholders for each site. For example, sponsors can view performance at a glance and identify sites with incomplete critical documentation such as ethics committee approvals or safety reports. The dashboard aggregates data into meaningful insights and highlights areas requiring intervention, enabling sponsors to allocate resources effectively and ensure trial milestones are not delayed.

[0136] In an embodiment, a real-time upgrade framework triggered by a specific event (e.g., see...) Figure 8 The system alerts sponsors and regulatory teams when critical placeholders are at risk of breaching compliance deadlines. For example, if a placeholder associated with “REGULATORY_SUBMISSION” remains unfilled within a specified time, the system escalates the issue to higher-level management, such as the trial manager or compliance officer. These escalations include contextual metadata from events such as “MILESTONE_DUE_DATE_REACHED,” enabling recipients to take action based on critical contextual information. The framework minimizes regulatory delays in high-priority areas and strengthens overall compliance management during sensitive trial phases.

[0137] Advanced site monitoring metrics and workflow integration (e.g., see...) Figure 4This can be used to provide detailed feedback on site-specific document status and long-term performance. For example, for each site, the system can track metrics such as placeholder fill rate, document submission delays, and report revision frequency. Events such as “VISIT_CONFIRMED” or “REPORT_SUBMITTED” generate and update these site-specific metrics in real time, enabling trial managers to identify problem sites and proactively address performance issues. This detailed understanding allows sponsors to assess trial health at the site level, optimize resource allocation, and improve overall compliance and operational efficiency across all participating sites.

[0138] In this embodiment, the system features automated milestone-specific notifications designed to alert trial personnel to tasks related to upcoming or overdue milestones. For example, when the “PATIENT_SCREENED” event is triggered, the system sends a targeted notification to the relevant site administrator, reminding them to fill in placeholders for patient informed consent forms and screening safety reports. These notifications, generated via event handlers and routed using workflows, can be role-specific and may contain contextual data such as document deadlines and procedural requirements. By proactively notifying personnel of milestone-specific document requirements, the system ensures timely filling of placeholders, reduces delays, and supports seamless compliance.

[0139] The system provides notification and monitoring capabilities, enabling sponsors to compare placeholder filling metrics across different trial sites. For example, the system can highlight trial sites with faster turnaround times for documents such as visit reports or training certificates, while flagging underperforming sites for intervention. This cross-site comparison capability provides actionable insights, allowing sponsors to identify best practices at high-performing sites and apply them to improve lagging sites. It also facilitates dynamic resource allocation, helping to ensure critical trial activities proceed as planned regardless of geographic location.

[0140] Leveraging the query-driven workflow supported by the aforementioned messaging scheme, the system allows sponsors to generate on-demand compliance reports. These reports extract real-time data from all integrated components regarding placeholder status, document version control, milestone consistency, and event-driven updates. For example, during a regulatory audit, sponsors can query placeholders associated with specific milestones such as “SITE_SELECTED” or “SITE_CLOSED_OUT” and retrieve the corresponding archived history. This functionality ensures sponsors remain audit-ready while eliminating manual searches and delays during regulatory inspections.

[0141] In this embodiment, the system integrates an automated risk assessment function to evaluate site compliance, leveraging event-driven workflows and metric tracking. For example, the event processor analyzes past event data, such as delays in "VISIT_CONFIRMED" or "REPORT_SUBMITTED" events, and scores sites based on their historical performance and the timeliness of placeholder filling. Low-compliance sites are flagged for additional monitoring or intervention during future milestone events, such as document submissions or regulatory approvals. This automated risk assessment helps sponsors proactively address potential issues at underperforming sites, thereby improving overall compliance and helping to ensure smooth workflows across all pilot sites.

[0142] Using comprehensive data tracked through placeholders and document submission patterns, the system can provide insights into the performance of external vendor activities and regulatory interactions, as managed via messaging system 132. For example, the system can track metrics of vendor-submitted documents, such as laboratory test results or ethics committee review metrics, to assess the timeliness and completeness of submissions. Sponsors can use this insight to identify high-performing vendors or address delays caused by underperforming partners. This transparency improves coordination among stakeholders and ensures consistent, timely submission of regulatory documents across all sites.

[0143] As described above, the integrated message flow enables the system to track placeholders and record the filling process from initialization to completion. For example, when a placeholder corresponding to a milestone (e.g., "VISIT_CONFIRMED") is dynamically created via an "apply-package" message (see, for example, see...), Figure 5 ), via subsequent messages sent by event handler 130, such as “file-to-eTMF” messages (e.g., see Figure 6 The system updates the status of placeholders in real time. Since populated placeholders are associated with uploaded documents, discrepancies can be flagged to help ensure that all necessary artifacts required for compliance are processed. By monitoring placeholder status via a two-way message flow, the system considers dependencies between site-specific activities, establishing better accountability and efficiency in the processing of experimental documents at each research site.

[0144] In one embodiment, the system may include a high-priority notification framework to alert users to critical placeholder activities. For example, when a deadline-related placeholder, such as “REGULATORY_APPROVAL_REQUIRED” or “ETHICS_COMMITTEE_APPROVAL_OUTSTANDING”, approaches its due date, the system notifies the responsible personnel. These notifications contain actionable details, such as the placeholder, a description of the relevant milestone, and expected documentation. This high-priority notification system helps ensure timely submission of critical compliance documentation, thereby reducing the risk of delays that could impact regulatory timeliness.

[0145] In this embodiment, the system integrates a visualization dashboard that uses placeholder population data and event feedback to display compliance gaps in real time. For example, the dashboard highlights placeholders marked as overdue and their associated milestones and documents. Users can delve into site-specific details, viewing document submission timeliness and population metrics for key milestones such as “REGULATORY_APPROVAL” or “PATIENT_SCREENED.” This visualization tool provides trial sponsors with actionable insights into eTMF progress, enabling faster resolution of compliance issues and helping to ensure timely compliance with regulatory requirements.

[0146] The system's event-driven architecture is highly valuable in situations where clinical trials must adjust parameters based on interim results, such as subject recruitment or dosing strategies. For example, when an adaptive trial milestone such as "TRIAL_PROTOCOL_AMENDED" is detected, the event processor 130 can dynamically update placeholders to reflect new documentation requirements, including updated trial protocols or additional regulatory submissions. This real-time adaptability helps ensure that the eTMF evolves efficiently to meet changing trial needs, thereby reducing administrative overhead and helping to ensure continued compliance with regulatory standards.

[0147] By integrating predictive algorithms into its event-driven workflow, event processor 130 can predict potential bottlenecks during placeholder filling. For example, using data from past events, such as “REPORT_INITIALIZED” and “VISIT_START_DATE_REACHED”, the system identifies patterns indicating delays in document preparation or submission at specific milestones. If a delay is predicted, event processor 130 can trigger advance notification, suggesting actions to mitigate risk or allocate resources to high-risk workflows. This predictive capability helps ensure smoother trial progress by addressing potential delays before they occur, thereby improving operational efficiency and reducing regulatory risks associated with incomplete trial documentation.

[0148] In one embodiment, the system can integrate e-learning progress tracking with event workflows (e.g., see...). Figure 4 For example, when a training milestone such as “TRAINING_COMPLETED” is reached in the e-learning management application 160, the event processor 130 automatically updates the corresponding placeholder for the training certificate in the eTMF. As the training-related placeholders are populated, the event processor can provide status feedback to the e-learning management application, thus helping to ensure that the two systems are synchronized. Notifications can be sent to site administrators and sponsors, enabling them to track training compliance in real time. This integration helps ensure that staff at all sites meet the required qualifications before participating in clinical trial activities, thereby reducing regulatory risk and helping to ensure readiness for inspections.

[0149] Figure 9 This is a flowchart of a method for autonomously generating and modifying dynamic electronic trial master files in real time according to a disclosed embodiment. Method 600 includes: receiving event messages from a messaging system via an event processor, the messaging system integrating multiple applications configured to automatically generate messages in real time in response to events (610). Each event corresponds to a user interaction at a first site based on an activity associated with the clinical trial via a user interface of a computer system and one of the applications. Each event message corresponding to each user interaction contains metadata associated with the corresponding activity. The method further includes: determining one or more hierarchical file structure placeholders associated with the corresponding activity for each event message based on the metadata of each event message via the event processor (620). The method further includes: automatically modifying the hierarchical file structure in real time via the event processor to include the one or more hierarchical file structure placeholders (630). The method further includes: storing one or more documents received from the computer system of the first clinical trial site in the one or more hierarchical file structure placeholders (640).

[0150] The disclosed embodiments provide technical solutions to the technical problems.

[0151] The autonomous electronic trial master file (eTMF) system is a technical solution to address the challenges of efficiently managing and organizing clinical trial documentation. Traditional eTMF systems heavily rely on manual operation, consuming significant manpower and leading to inefficiency, delays, and increased error risks. The disclosed embodiments solve these problems by innovatively integrating automation technology to autonomously create and populate eTMFs using signals and data. This system constructs a technical solution to address these issues from multiple perspectives.

[0152] The system automates the document management process, which traditional workflows often require significant manual intervention. Manual operations in traditional eTMF systems are prone to human error, inconsistencies, and inefficiencies, leading to delays and increased costs in clinical trials. The autonomous eTMF generation and modification system mitigates these problems by dynamically creating placeholders for required documents using signals and data, and automatically populating these placeholders as data and documents become available. This automation significantly reduces the need for human intervention, minimizing errors and inconsistencies, and ensuring timely updates. This structured automation represents a significant technological advancement compared to traditional manual systems.

[0153] The system integrates multiple applications, such as electronic informed consent, adverse event management, and research management, into a unified framework. Clinical trials involve numerous aspects, each managed by different applications, and integrating data from these diverse sources into a unified eTMF is a complex and time-consuming process. The autonomous eTMF generation and modification system processes signals from these applications to automatically generate and organize necessary documentation, ensuring completeness and up-to-date records. This integration enables seamless data flow and real-time updates between different components of clinical trial processing, representing a significant technological advancement in clinical trial management.

[0154] The system employs a Large Language Model (LLM) to generate signals and extract information required by the eTMF and other applications. Extracting and processing relevant information from diverse and unstructured data sources is a significant challenge in eTMF management, as traditional systems lack the ability to effectively interpret and utilize this data. LLM can interpret complex data inputs and map them to the specific documents required by the eTMF, enabling the system to autonomously generate and organize documents based on predefined criteria, ensuring accuracy and completeness. This capability goes beyond general data processing, providing a sophisticated approach to clinical trial document management.

[0155] The system utilizes templates and event-driven processing to dynamically create documents based on contextual data. Clinical trials require the creation of various documents at different stages, such as patient forms, training certificates, and adverse event reports. Manually generating these documents is inefficient and error-prone. For example, when a patient completes training, the system can automatically generate a training certificate using stored data and predefined templates. This approach ensures accurate and timely document creation, reducing delays and human error. Dynamically creating documents using templates is a unique technical feature that enhances the reliability and efficiency of the eTMF system.

[0156] The system includes a centralized document repository that allows documents to be stored once and reused across multiple studies. In traditional systems, managing documents across multiple studies and ensuring their reusability is challenging, often leading to repetitive work and increased storage requirements. For example, the principal investigator's license can be stored in the repository and cited in any study involving that investigator, reducing redundancy and ensuring that the latest version of the document is always used. This centralized approach optimizes storage and ensures consistency, thereby enhancing the overall management of clinical trial documents.

[0157] Furthermore, the system ensures compliance and audit readiness by maintaining a comprehensive and up-to-date eTMF. Ensuring compliance and maintaining audit readiness are challenges faced by clinical trials, and traditional eTMF systems often struggle to provide real-time access to complete and accurate documentation. By automating the document management process and maintaining a comprehensive library, the autonomous eTMF generation and modification system enhances compliance and audit readiness, and provides real-time access to accurate and complete documentation.

[0158] For example, if an adverse event requires the creation of new documents across test sites, the system ensures that placeholders for these reports are created simultaneously at all relevant locations. This synchronization mechanism minimizes delays in multi-application workflows and ensures that all test components operate collaboratively in accordance with regulatory requirements.

[0159] Furthermore, the system employs event-driven processing to respond to specific signals and triggers in the clinical trial workflow. For example, when a new site is added or a new principal investigator is designated, the event processor 130 identifies the necessary documents and creates appropriate placeholders. This event-driven approach ensures that the eTMF is dynamically updated in real time, reflecting the latest changes and additions. It significantly reduces the latency between event occurrence and document update, representing a clear technological improvement over batch processing or manual updates.

[0160] The system also utilizes contextual analysis to interpret the data and signals it receives. For example, it can analyze research protocols to determine the types of documents required, such as patient informed consent forms, safety reports, and monitoring visit records. This contextual interpretation enables the system to make informed decisions regarding document requirements and organization, ensuring that the eTMF comprehensively and accurately reflects research needs. This capability goes beyond simple data collection, providing a nuanced and intelligent approach to eTMF management.

[0161] Furthermore, the system is designed to be scalable and flexible, capable of handling eTMFs of varying sizes and complexities. It can adapt to different study designs, regulatory requirements, and document types. This scalability and flexibility ensures the system can be used for a wide range of clinical trials, from small studies to large multisite trials. This scalability and adaptability enhance the system's practicality and applicability, making it suitable for the specific technical characteristics of various clinical trial scenarios.

[0162] User prompts and guidance are another feature of the system. When manual input is required, such as uploading specific documents, the system provides clear prompts and guidance to the user. For example, if a medical license is required, the system will prompt the user to upload the document and guide them through the entire process. This user-friendly approach reduces the possibility of errors and ensures that all necessary documents are collected in a timely manner. The integration of user prompts and guidance is a specific technical feature that enhances the overall user experience and efficiency of the system.

[0163] Data security and integrity are also critical in the autonomous eTMF generation and modification system. The system employs robust data security measures to protect sensitive clinical trial information. These include encryption, access controls, and audit trails to ensure all data is secure and its integrity is maintained. By ensuring all data is secure and traceable, the system meets stringent regulatory requirements and provides users with peace of mind. Implementing advanced security measures is a specific technological improvement for addressing the data protection needs in clinical trials.

[0164] Finally, the system is designed to continuously learn and improve from the data it processes. Utilizing machine learning algorithms, it can optimize its processes, improve accuracy, and adapt to new research and documentation. This continuous learning capability ensures the system remains up-to-date and can respond to evolving clinical trial requirements. This represents a specific technological advancement that enhances the system's long-term effectiveness and reliability.

[0165] Therefore, the autonomous eTMF generation and modification system offers a robust and innovative technical solution to the challenges of clinical trial document management. By utilizing advanced automation technologies, integrating various applications, leveraging large language models, event-driven processing, contextual analysis, scalability, user prompts, data security, and continuous learning, the system provides specific technical improvements to enhance the efficiency, accuracy, and reliability of eTMF management. This comprehensive approach ensures that eTMFs are always up-to-date, complete, and compliant with regulatory standards, offering significant advancements compared to traditional methods. Unlike abstract results or generic processes, the disclosed embodiments provide specific and detailed implementation plans that directly address technical challenges and offer substantial improvements to related technologies.

[0166] Aspects of the invention may be embodied in the form of a system, a computer program product, or a method. Similarly, aspects of the invention may be embodied in hardware, software, or a combination of both. Aspects of the invention may be embodied in a computer program product, which is stored on one or more computer-readable media in the form of computer-readable program code embodied on one or more computer-readable media.

[0167] A computer-readable medium can be a computer-readable storage medium. A computer-readable storage medium can be, for example, an electronic, optical, magnetic, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any combination thereof.

[0168] The computer program code in embodiments of the present invention can be written in any suitable programming language. The program code can be executed on a single computer or on multiple computers. The computer may include a processing unit that communicates with a computer-usable medium, wherein the computer-usable medium contains a set of instructions, and the processing unit is designed to execute that set of instructions.

[0169] The foregoing discussion is intended to illustrate the principles and various embodiments of the invention. Once the foregoing disclosure is fully understood, many variations and modifications will be apparent to those skilled in the art. The foregoing is intended to be construed as covering all such variations and modifications.

Claims

1. A computer-implemented method for autonomously and in real-time generating and modifying the dynamic hierarchical file structure of an electronic test master file (eTMF), specifically a computer-implemented method for autonomously and in real-time generating and modifying the dynamic hierarchical file structure of an eTMF, the method comprising: An event handler receives event messages from a messaging system that integrates multiple applications configured to automatically generate messages in real time in response to events, wherein: Each event corresponds to a user interaction at the first site based on activities associated with the clinical trial and via a user interface of the computer system and the application. Each event message corresponding to each user interaction contains metadata associated with the corresponding activity; The event processor determines one or more hierarchical file structure placeholders associated with the corresponding activity for each event message based on the metadata of each event message; The event handler automatically modifies the hierarchical file structure in real time to include the one or more hierarchical file structure placeholders; and One or more documents received from the computer system of the first site are stored in the one or more hierarchical file structure placeholders.

2. The method according to claim 1, wherein, The metadata associated with the corresponding activity includes one or more of the following: a site identifier associated with the site, a milestone identifier corresponding to the activity, and a role designation for the user associated with the activity.

3. The method according to claim 1, further comprising: Before storing one or more documents received from the computer system in the placeholders of the hierarchical file structure, the one or more documents are automatically validated according to a predefined format or compliance standard.

4. The method according to claim 1, wherein, Determining the one or more hierarchical file structure placeholders by the event processor includes: dynamically applying one or more predefined templates from a template library, which specify the expected document type for activities associated with the clinical trial.

5. The method according to claim 1, wherein, Automatically modifying the hierarchical file structure in real time includes dynamically adapting the structure to meet region-specific requirements associated with the first site.

6. The method of claim 1, further comprising: The completion of the hierarchical file structure placeholders is tracked across multiple sites, and quantitative metrics are generated in real time for completed or pending placeholders at each site.

7. The method of claim 1, further comprising: An alert is generated when one or more placeholders remain unfilled for an extended period of time, and the alert is sent to a specified user.

8. The method of claim 1, further comprising: In response to receiving an event message indicating a change in the corresponding activity, the existing placeholders in the hierarchical file structure are automatically replaced with updated placeholders.

9. The method of claim 1, further comprising: The one or more documents are associated with placeholder-specific metadata, wherein the metadata includes document type, author, version, and applicable event identifier.

10. The method according to claim 1, wherein, The hierarchical file structure is configured to logically divide placeholders into regions and sections corresponding to research phases, test sites, or geographical areas.

11. A system for autonomously and in real-time generating and modifying a dynamic hierarchical file structure of an electronic test master file, i.e., a system for autonomously and in real-time generating and modifying a dynamic hierarchical file structure of an eTMF, the system comprising: A computer having one or more processors in communication with a memory that stores instructions that can be executed by the one or more processors to perform the method according to any one of claims 1-10.

12. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a computer, cause the one or more processors to perform the method according to any one of claims 1-10.

Images