Automatic randomized verification for RTSM systems

An automated system using natural language processing and computer simulation addresses inefficiencies in randomization list validation, enhancing the reliability and efficiency of RTSM system configuration in clinical trials.

CN119072702APending Publication Date: 2024-12-034G CLINICAL LLC
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202380021398.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-01-18
Filing Date
2023-01-18
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Current methods for validating and configuring randomization lists in clinical trial management are inefficient and lack statistical reliability, requiring manual processing and intermediate verification steps.

Method used

An automated system for generating and validating randomization lists using natural language processing and computer simulation, enabling efficient configuration of RTSM systems with statistically sound randomization.

Benefits of technology

The system provides a more reliable and efficient method for configuring RTSM systems by automating the generation and validation of randomization lists, reducing manual intervention and enhancing the statistical reliability of clinical trials.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119072702A_ABST
    Figure CN119072702A_ABST
Patent Text Reader

Abstract

A method for managing a clinical trial for a medicament includes determining a list of randomized patients and a list of randomized kits for the clinical trial, automatically validating the randomized list, and configuring an RTSM system based on the validated list. Verifying the randomized patient list may include verifying that the patient list includes a balance or spread above a balance or spread threshold, and verifying that the list has randomness above a predetermined randomness threshold. Verifying the randomized list of kits may include verifying that the list of kits has randomness above a predetermined randomness threshold.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Related Applications

[0002] This application claims the benefit of U.S. Provisional Application Serial No. 63 / 300,589, filed on January 18, 2022, under 35 U.S.C. §119(e), the entire contents of which are incorporated herein by reference. Background Art

[0003] The present disclosure relates to automated validation of randomization lists in clinical trial management, and configuration of a randomization and trial supply management (RTSM) system based on such validated randomization lists. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 is a diagrammatic view of an example system for managing a drug trial.

[0005] Figure 2 is a flow chart illustrating an example method of configuring an RTSM system for clinical trials.

[0006] Figure 3 is a flow chart illustrating an example method of validating a randomized patient list.

[0007] Figure 4 is a flow chart illustrating an example method of validating a randomized kit list.

[0008] Figure 5 is a flow chart illustrating an example method for interpreting clinical trial specifications using natural language processing.

[0009] Figure 6 is a graphical view of a natural language processing based interpreter for trial specifications.

[0010] Figure 7 An example graphical display of a randomized patient list is shown that may be output to a user in conjunction with validating the randomized patient list.

[0011] Figure 8 and Fig. 9 An example graphical display of a randomization kit list is shown that may be output to a user in conjunction with a validation randomization patient list.

[0012] Fig.10 is a diagrammatic view of an example implementation of a user computing environment. DETAILED DESCRIPTION

[0013] Randomization and trial supply management (RTSM) systems typically implement trial supply plans based on randomized patient lists and randomized kit lists, both of which can be generated based on trial parameters provided by the sponsor of the trial. Validation of those randomized lists by automated processing and computer simulation enables efficient configuration of RTSM systems with statistically sound randomization that is more reliable than that confirmed under known manual processing.

[0014] Referring now to the drawings, in which like reference numerals designate like or similar features throughout the various views, for example, Figure 1 is a diagrammatic view of an example system 10 for managing a clinical trial, such as a randomized drug trial. The system 100 may include a randomization list generation and verification system 102 and an RTSM system 104, either or both of which may be connected to a plurality of user computing devices 106 (in Figure 1 One of which is shown in ) electronic communication.

[0015] In general, the randomization list generation and verification system 102 and the RTSM system 104 may be involved in different aspects of planning and management for a drug trial. As will be described in more detail below, the RTSM system 104 may generally direct and manage the trial, while the randomization list generation and verification system 102 may generate randomization lists, including randomization patient lists and randomization kit lists, and verify the content and randomness of those lists.

[0016] In an embodiment, the RTSM system 104 may include one or more computing devices and may perform a number of operations for organizing and choreographing a drug trial, such as blinding, drug dispensing, and automatic drug resupply. The RTSM system 104 may communicate electronically with one or more user computing devices 106 to receive input from a user (e.g., parameters of a trial) and provide output to the user (e.g., reports on a list of verified patients and test kits, orders for one or more drugs, status of patients participating in the trial, information about the trial site, etc.).

[0017] The RTSM system 104 can be configured to communicate and exchange information with various user types (i.e., one or more user computing devices related to such user types communicate). For example, in a given trial, the RTSM system 104 can provide an input / output interface for a clinical trial sponsor 108 (e.g., a pharmaceutical company). In an embodiment, the trial sponsor 108 can input complete parameters for a clinical trial or a phase of a clinical trial for the RTSM system 104. Such a complete parameter set may include, for example, patient visits or other actions on patients, batches and operations on batches, inventory at the site or warehouse level and operations on inventory, shipping and operations on shipping, temperature excursions and their management, returns of distributed or undistributed drugs and related approvals and parameters, destruction status of inventory, actions on alarms or notifications, addition and parameterization of sites and warehouses, countries, settings of prediction parameters such as confidence dials and long window dials, settings of site group registration rates, settings of users and their access rights, settings of groups, loading of test kits and randomization lists, and unblinding of patients or test kits. The RTSM system 104 can then determine that resupply actions may have to be taken and output them to the trial sponsor. As will be described below, the RTSM system 104 can also track and record the status of the trial based on data from one or more warehouses, sites, doctors, and patients, and can make such information available to the trial sponsor.

[0018] For example, the RTSM system 104 can be in electronic communication with one or more trial sites 110, such as hospitals or clinics. For example, the RTSM system 104 can output instructions, such as dispensing instructions for a particular patient, to the trial site 110. The RTSM system 104 can also receive data, such as patient information, dispensing records, etc., from the trial site 110, so that the RTSM system 104 has data indicating which patients involved in the trial took which drugs and at which sites, and at what time.

[0019] The RTSM system 104 can also communicate with one or more distribution facilities 112 such as warehouses, warehouses or other facilities involving the distribution of active drugs, placebos and / or control substances involved in the test to the test site 110. For example, the RTSM system 104 can issue a delivery instruction to such a facility 112. In an embodiment, such a delivery instruction can be automatically generated and / or transmitted by the RTSM system 104 based on a supply plan. The RTSM system can also receive data such as a delivery record and inventory information from such a facility 112, so that the RTSM system 104 has data indicating the distribution inventory and the remaining inventory of the drugs involved in the test and the location of these inventories. Drugs (e.g., active drugs, control substances and placebo test kits) can be distributed and distributed to patients according to the instructions of the RTSM system, and in particular according to the verified, randomized patient and test kit lists generated according to the present disclosure.

[0020] One or more researchers (e.g., physicians) 114 may also be in electronic communication with the RTSM system 104. The RTSM system 104 may output information about the drugs involved in the trial and information about the trial to such researchers 114 so that the researchers can notify patients about the trial as appropriate. The RTSM system 104 may receive enrollment information (e.g., the number of patients recommended by the researchers for the trial, characteristics of those patients, specific patients who have been enrolled, etc.) from such researchers so that the RTSM system 104 has data indicating the enrollment volume and characteristics of enrolled patients.

[0021] The randomization list generation and verification system 102 may include a processor 118 and a non-transitory computer-readable memory 120 including instructions that, when executed by the processor 118, cause the processor 118 to perform one or more methods, algorithms, processes, etc. of the present disclosure. The memory 120 may store one or more modules in the form of executable instructions (e.g., software) for execution by the processor 118. Example modules are described below.

[0022] In an embodiment, memory 120 may store specification interpretation module 122. A user (e.g., via a user computing device, which may be associated with a trial initiator, as described above) may submit information indicating trial parameters, and supply forecasting system 12 may receive the information indicating trial parameters. Specification interpretation module 122 may process and parse the received information to extract and determine the trial parameters.

[0023] The memory 120 may also store a randomization list validation module 124 configured to automatically generate and validate a randomization list, such as a randomization patient list and a randomization kit list, based on the experimental parameters determined by the specification interpreter 122. Figure 3 and Figure 4 An example method for automatically validating a randomization list is described.

[0024] The randomization list generation and verification system 102 may provide an electronic user interface for use by the trial sponsor. For example, the verification system may output a verification report associated with one or more patient lists and / or kit lists. The verification report may include, for example, a graphical display of the patient list and / or kit list (an example of which is shown in FIG. Figure 7 is shown in and will refer to Figure 7 description) and other information.

[0025] In an embodiment, one or both of the RTSM system 104 and the randomization list generation and verification system 102 may be implemented on a software as a service (SaaS) basis. Thus, the verification system and / or the RTSM system 104 may be implemented in one or more servers, databases, etc. that are electronically accessible to users via the Internet. Additionally or alternatively, one or both of the randomization list generation and verification system 102 and the RTSM system 104, or one or more aspects or functions of the randomization list generation and verification system 102 and the RTSM system 104 may be implemented, for example, on a local server of a trial sponsor, or in a local copy stored on a user computing device associated with a trial sponsor, trial site, warehouse, etc.

[0026] In embodiments, although the RTSM system 104 and the randomization list generation and validation system 102 are shown and described as separate systems, the RTSM system 104 and the randomization list generation and validation system 102 or some of their functionality may be implemented in a single system or in common computing resources.

[0027] Figure 2 2 is a flow chart illustrating an example method 200 for configuring an RTSM system with an automatically verified randomization list. The method 200 or one or more aspects of the method 200 may be implemented, for example, by a verification system and an RTSM system, for example Figure 1 The randomization list generation and verification system 102 and the RTSM system 104 are performed by a combination of the randomization list generation and verification system 102 and the RTSM system 104.

[0028] Method 200 may include receiving one or more specification files from a user at block 202. The user may be, for example, a trial sponsor (e.g., a pharmaceutical company). The specification file may include one or more file types. For example, in an embodiment, the specification file may include one or more text files and one or more spreadsheet files. In an embodiment, the text may be entered in a free or natural form rather than in response to a structured prompt. The specification file may include information about the proposed drug trial that may be used by the sponsor, such as block size, treatment groups, ratios, treatment kits, controls and placebos, kits and serial number ranges, groups, layers, geographic or demographic information, etc.

[0029] The method 200 may also include converting and merging the specification file at block 204 to extract or otherwise determine the parameters of the trial, including the parameters of the desired patient randomization list and / or the parameters of the desired kit randomization list included in the specification file. Figure 5 and Figure 6 An example method for performing block 204 is described.

[0030] Method 200 can also include generating a randomized patient list and a randomized kit list based on the test parameters determined at block 206 at block 206. For example, the randomized patient list can have a randomization sequence of patients assigned to the active drug group or placebo group of the test, or an active group or control group, or one of a plurality of different active drug strength groups. In some embodiments, the randomized patient list can be based on individual randomization (e.g., wherein each patient has the same possibility of being assigned to any given group such as an active drug group, a placebo group, a control group, an active drug strength group, etc. as each other patient). In other embodiments, the randomized patient list can be based on a randomization block, and the block can have a variable or random size, or other appropriate randomization techniques can be adopted. Similarly, the kit list can be randomized separately, or can be based on a randomized block, and the block can have a variable or random size, or other appropriate randomization techniques can be adopted.

[0031] Method 200 may also include, at block 208, validating the patient list and kit list generated at block 206. Figure 3 and Figure 4 Example operations are described for block 208. In some implementations, block 208 can be performed automatically when the patient list and the kit list are generated.

[0032] Validating the patient list and the kit list at block 208 may include generating a report including the validation results. Figures 7 to 9As described, the report may include one or more graphical displays of patient distribution and / or kit distribution. The report may additionally or alternatively include information or tables related to the patient list or kit list, and / or information related to one or more simulations or mathematical analyses about the patient list or kit list, such as the number and sequence of patient registrations in one or more simulations.

[0033] If the patient list or the kit list is not valid, the method 200 may output an error at block 212 and return to block 206 to generate a new randomized patient list and / or randomized kit list to replace the invalid list. The error output at block 212 may identify the failed validation step or portion (e.g., Figure 3 and Figure 4 The error output at block 212 may include results of one or more simulations, such as one or more patient enrollment simulations, and an indication of whether a simulation meets or does not meet one or more thresholds for a randomization list.

[0034] If the patient list and the kit list are valid, the method 200 may include generating a kit label according to the validated list at block 210. Block 210 may also include printing the kit label according to the randomization list and applying the kit label so that distribution and delivery of the kit can be achieved based on the user's randomization and other parameters.

[0035] If the patient list and the test kit list are valid, the method 200 may include configuring the RTSM system according to the verified randomized patient list and the verified randomized test kit list at block 212 and according to the remaining test parameters determined at block 206. The RTSM system may be configured to execute and monitor the test plan including the randomized patient list and the randomized test kit list. In addition, active drugs, placebos and / or control test kits may be distributed and dispensed to patients according to the verified randomized patient list and the verified randomized test kit list.

[0036] Due to the inclusion of automatic document conversion and interpretation for determining trial parameters, automatic generation of randomization lists, and automatic verification of randomization lists, system 100 and method 200 can enable automatic configuration of an RTSM system for a particular clinical trial, requiring only free-form text and spreadsheet files from the trial sponsor, without requiring intermediary manual verification of the randomization list. In addition, the automatic verification of the randomization list as described in method 200, method 300, method 400 provides a more statistically reliable verification than current methods, thereby achieving a more reliable trial.

[0037] Figure 3is a flow chart illustrating an example method 300 for automatically verifying a randomized patient list. The method 300 or one or more portions thereof may be performed by Figure 1 The randomization list generation and verification system 102 is performed.

[0038] Method 300 may include verifying the validity of a patient list file at block 302. Verifying the validity of a patient list file may generally include confirming that a list, such as a patient list including valid content such as a randomized list, matches or satisfies a user's description of a trial. For example, block 302 may include verifying whether the group ratios, block sizes, number of blocks per layer, and other ratios and numbers in the patient list satisfy the trial description. Block 302 may include one or more of: verifying the total number of rows (i.e., patients) and the total number of blocks of the patient list, verifying the number of rows and the number of blocks for one or more treatment groups, verifying the number of rows and the number of blocks for one or more group and layer combinations, or verifying the size and number of each of a plurality of block types.

[0039] The method 300 may also include verifying the quality of the patient allocation in the randomized patient list at block 304. Verifying the quality of the patient allocation may include, for example, verifying that the patient list includes an appropriate balance of active drug patients and placebo patients within the entire list and / or each defined layer or group, or an appropriate balance of active drug patients and control drug patients, or an appropriate balance of patients receiving a first strength active drug and patients receiving a second strength active drug, etc. In some embodiments, the balance within each grouping (e.g., the percentage of active drug patients to all patients) may be compared to a predetermined threshold, and if the balance is above the threshold or within a certain range of the predetermined threshold, the balance may be considered effective. For example, if the trial is intended to include 2 / 3 of the patients in the active drug portion of the trial, a simulation of about 60% to about 72% of the patients in the active drug treatment group may be considered effective. The exact effective boundary depends on several aspects of the clinical trial, such as the number of patients, stratification, treatment effect, etc. In some embodiments, the balance of each layer or group may be calculated, and if the balance of any layer or group fails to meet the threshold, the entire patient list may be considered invalid. That is, if the balance of each layer or group meets the balance threshold, the patient list may have an effective balance.

[0040] In some embodiments, block 304 may include calculating one or more simulations of the patient enrollment, calculating one or more balances for each simulation, and comparing the balance of each simulation to a predetermined threshold. In some simulation embodiments, each patient may have an equal chance of each of the two or more blocking factors for the trial. In other simulation embodiments, different probabilities may be applied to different strata.

[0041] In embodiments where a simulation is performed, the balance of the patient list can be calculated multiple times during the course of the simulation (e.g., after enrolling 100 patients, then after enrolling 200 patients, etc.), and each time the balance compared to a threshold. Thus, the evolution or progress of the patient balance can be assessed during the course of the simulation. In some embodiments, if the balance fails to meet the threshold at any calculation time in the simulation - i.e., if the number of patients in the treatment group of the trial is too far above or too far below a desired ratio or percentage - the patient list can be considered invalid. In some embodiments, the time points of the simulation at which balance is assessed and / or implemented, including the end of the simulated trial and / or one or more mid-points in the trial, can be user-selected.

[0042] In addition to or instead of calculating one or more balances for the patient list, in some embodiments, a spread of the patient list can be calculated and the spread can be compared to a threshold. For example, a spread can be calculated for a patient list having multiple block types. In such embodiments, the number of treatment group patients in each block type can be calculated, and the spread (e.g., standard deviation) of those numbers can be calculated and compared to a threshold. If the spread is below the threshold, then the patient list can be valid at block 304.

[0043] In some embodiments, multiple separate patient lists can be generated from the trial specification, and block 304 can be performed with respect to each of those patient lists, and the results of block 304 on the multiple patient lists can indicate the quality of the trial specification parameters provided by the trial sponsor. That is, the results of block 304 with respect to the multiple patient lists can indicate whether the trial parameters are compatible with the desired patient randomization technique (e.g., by virtue of the number or proportion of valid or invalid patient lists based on those parameters).

[0044] The method 300 may also include verifying the randomness of the patient assignments in the randomized patient list at block 306. For example, block 306 may include determining that the block pattern is evenly distributed throughout the patient list. In some embodiments, determining that the block pattern is evenly distributed may include determining the distribution of treatment patterns across the blocks and calculating a p-value for the distribution, which indicates the probability of obtaining a distribution that is at least as extreme as the subject distribution. In some embodiments, a chi-squared test may be used to calculate the p-value. In other embodiments, another appropriate mathematical method for calculating the p-value. In some embodiments, the p-value may be compared to a predetermined threshold (e.g., 0.05 or 5%), and if the p-value exceeds the threshold, the list may be considered to be effectively random.

[0045] In an example of determining a distribution of treatment patterns, consider an example trial with a block size of three patients, where each block is intended to include two active drug patients and one placebo patient. In this example, there are three treatment patterns: AAP (i.e., the first and second patients in the block receive active drug, and the third patient receives placebo); APA, and PAA. Thus, in this example, block 306 can include determining the number of blocks with each of these three patterns and calculating a p-value for the distribution of those block / pattern numbers.

[0046] In method 300, if the patient list is determined to be invalid at any one of blocks 302, 304, and 306, the patient list may be considered invalid. Conversely, if the patient list is determined to be valid at all of blocks 302, 304, and 306, the patient list may be considered valid.

[0047] Figure 4 is a flow chart illustrating an example method for automatically verifying a randomized kit list. Method 400 or one or more portions thereof may be performed by Figure 1 The randomization list generation and verification system 102 is performed.

[0048] Method 400 may include verifying the validity of a test kit list file at block 402. Verifying the validity of a test kit list file may generally include confirming that the list, for example, includes valid content such as a randomized test kit list that matches or satisfies the user's test description. For example, block 402 may include one or more of the following: verifying the total number of rows and the total number of blocks in the test kit list, comparing the use (or non-use) of a test kit block with a user's request for the use (or non-use) of a test kit block, or verifying the number of rows and the number of blocks of one or more test kit types. In some embodiments, each portion of the test kit list may contain different test kit types and use different block sizes (e.g., as if each portion were its own test kit list). Therefore, in some embodiments, block 402 may include evaluating the test kit type and block size of each of the multiple portions to ensure that the test kit type, block type, block count, and sequence range found for each portion meet expectations.

[0049] Method 400 may also include verifying the randomness of the distribution of reagent kits in the randomized reagent kit list at block 404. For example, block 404 may include calculating the average sequence of each reagent kit type and confirming that the reagent kit types in the same section have an average sequence close to the middle of the section. Block 304 may also include, for a non-blocked list, calculating the distribution of batch lengths (i.e., the number of continuous reagent kits) and comparing the distribution with the expected number of batches of each length. For example, block 406 may include determining that the batch lengths are uniformly distributed. In some embodiments, determining that the batch lengths are uniformly distributed may include calculating a p-value for the batch length distribution. In some embodiments, a chi-square test may be used to calculate the p-value. In some embodiments, the p-value may be compared to a predetermined threshold (e.g., 0.05 or 5%), and if the p-value exceeds the threshold, the list may be considered to be effectively random.

[0050] In some embodiments, block 404 can include, for the chunked list of kits, determining that the chunking pattern is evenly distributed throughout the list of kits. In some embodiments, determining that the chunk pattern is evenly distributed can include determining the distribution of the kit pattern across the chunks and calculating a p-value for the distribution, the p-value indicating the probability of obtaining a distribution that is at least as extreme as the subject distribution. In some embodiments, a chi-squared test can be used to calculate the p-value. In other embodiments, another suitable mathematical method for calculating the p-value. In some embodiments, the p-value can be compared to a predetermined threshold (e.g., 0.05 or 5%), and if the p-value exceeds the threshold, the list can be considered to be effectively random.

[0051] Figure 5 is a flow chart illustrating an example method 500 of determining parameters of a proposed drug trial, including patient parameters, kit parameters, and randomization parameters. In embodiments, the method 500, or one or more portions of the method 500, may be performed by the RTSM system 104, by the randomization list generation and validation system 102, and / or by another system in communication with the RTSM system, such as a prediction system.

[0052] In some embodiments, only the portion of the method 500 related to the randomization parameters can be performed by the randomization list generation and verification system 102 independently of other trial parameters in a separate process from the processes performed for determining other parameters of the trial. For example, the randomization list generation and verification system 102 can be implemented as a separate system from the RTSM system 104 to construct the randomized patient and / or kit list separately from the remaining aspects of constructing the trial. Therefore, although the following discussion will discuss randomization parameters in conjunction with other trial parameters, it should be understood that aspects related to randomization can be performed separately.

[0053] The method 500 may also include receiving a spreadsheet file and one or more text files from a user at block 502. The files may include information about the studies to be conducted that are available to the user at the time. The text file may be, for example, an HTML file. For example, the spreadsheet and text file may be received via file upload through a website provided by the supply forecasting system. In an embodiment, the supply forecasting system may provide an interface in which a user may enter free-form text to describe the proposed study (the entered free-form text may be or may become one or more text files of block 502), and further upload one or more spreadsheets through the interface.

[0054] The method 500 may also include converting the spreadsheet into one or more text-based tables at block 504. In an embodiment, the spreadsheet may be converted into, for example, one or more HTML-based tables or into any format that provides a text-based file.

[0055] The method 500 may also include inserting a text-based table into one or more text files at block 506 and performing natural language processing on the one or more text files to determine test parameters at block 508. Figure 6 508. An example of natural language processing that may be performed at block 508 is described below. Further details regarding natural language processing are provided in U.S. Patent 2017 / 0154168, entitled “Generating RTSM Systems for Clinical Trials,” assigned to 4G Clinical, LLC by inventor Edward A. Tourtellotte, which is hereby incorporated by reference in its entirety. By using natural language processing (NLP), users can describe their trials in plain English, independent of the level of detail of available information, and the system can determine the available details of the trial for further processing.

[0056] Figure 6 is a diagrammatic view of an example electronic specification interpreter 600 that applies natural language processing. In an embodiment, the specification interpreter 600 may perform Figure 5 The canonical interpreter 600 may include one or more aspects of the method 500 of , and in particular may perform block 508. The canonical interpreter 600 may include a main interpreter 602, one or more specialized interpreters 604, one or more information extractors 606, one or more specialized natural language processing (NLP) tools 608, and one or more core NLP libraries 610. In embodiments, the canonical interpreter 600 may also include additional components.

[0057] The core NLP library such as core NLP library 610 can include information of natural language (e.g., English), such as vocabulary and grammar, corpus, tagger, tokenizer, stem analyzer, lemmatizer, learning algorithm, etc. The core NLP library 610 can be used by the specification interpreter 600 to check the specification input by the specification writer. For example, the specification interpreter 600 can rely on the core NLP library 610 to identify words and stems, mark incorrect spellings, analyze grammar and meaning, detect logical operations and data definitions and / or suggest alternative terms. In some cases, the core NLP library 610 can be provided by a third party, such as from some programming languages ​​(e.g., Java or Python) development toolkits (e.g., natural language toolkits or NLTK) provide.

[0058] Special NLP tools, such as special NLP tools 608, can provide system-specific extensions to the core NLP library 610. For example, special NLP tools 608 can include additional information such as words, phrases, terms, grammars, reserved words / phrases, keywords, and operators that can be generally used in the context of certain fields (e.g., clinical trials). Special NLP tools 608 can also include advanced language analysis functions, such as tokenization, stemming, grammar, collocation, language corpora, sentence structure and meaning analysis, and other natural language processing features. Special NLP tools 608 can also include artificial neural networks.

[0059] Special NLP tool 608 can also be used by specification interpreter 600 to check the specification input by specification writer.For example, specification interpreter 600 can rely on special NLP tool 608 to identify system-specific words, mark incorrect spelling and / or suggest alternative terms.Special NLP tool 608 can be provided by a third party or developed internally.Information extractor such as information extractor 606 can extract certain information from specification.One type of information extracted can include matrix (for example, information constructed in a tabular manner).When organizing data in matrix form is more convenient than organizing data in natural language form, matrix can be used in specification development.For example, matrix can be used to define patient visit schedule and associated dosage regimen or other visit events; Matrix can also be used to define patient test kit type, supply chain relationship and delivery time or any other data naturally defined in matrix.

[0060] Another type of information extracted may include certain patterns of sentences, such as definitions. Definitions may be structured, for example, as [subject of definition][verb proposal][item or list of items], but the structure may be different or made more flexible to enable the use of a free English style in writing and extraction of relevant information. Other patterns of sentences may be defined as needed.

[0061] The main interpreter 602 can receive a test specification. The main interpreter 602 can perform a top-level analysis of the specification and can organize the interpretation in a logical order of the subject matter. The subject matter can include one or more defined test topics, and the one or more defined test topics can run through one or more sections of the specification document. A specific topic of the interpretation can correspond to a dedicated interpreter 604, which is configured to analyze the information related to the specific topic throughout the specification document (in one or more sections). The main interpreter 602 can call different dedicated interpreters through the specification to handle different topics of the test.

[0062] Special interpreter 604 can include research level interpreter, randomization interpreter, action interpreter, drug supply interpreter, manufacturing interpreter, registration interpreter or other required interpreter. Research level interpreter can interpret the high-level information in the specification. For example, high-level information can include some research level information, such as the total number of patients expected to be screened and randomized, randomization scheme and method and other required parameters. Randomization interpreter can interpret the randomization information in the specification. Randomization interpreter can identify the size of the random list, the treatment group used, the treatment group ratio, stratification and group, the stratification allocation of each part of the list (for example, the random number [101,200] for women and the random number [201,300] for men), randomization method (using minimization or using random list with or without dynamic block), the type of block with count of each block type (the size and kit ratio used in each block), the layout of the list (column header and the content of each column), the file naming convention and the separator to be used in the generated file.

[0063] An action interpreter can interpret event action information in a specification. For example, an event action can include information such as what medication distribution may occur in the context of each patient visit. A medication supply interpreter can interpret medication supply information in a specification. For example, medication supply information can include supply networks, shipping rules, distribution kit information, etc. A manufacturing interpreter can interpret the various production steps and intermediate manufacturing lead times to convert an investigational substance into a kit that can be distributed to a patient. A registration interpreter can explain the rate at which patients start studies at various levels (e.g., study level, regional level, etc.). The list of dedicated interpreters is not exhaustive. Other dedicated interpreters can be added as needed.

[0064] In some cases, the main interpreter 602 can be an entry point into the specification interpreter 600. For example, the main interpreter 602 can receive the specification and divide the interpretation into different parts or topics. The main interpreter 602 can then call the appropriate dedicated interpreter 604 to analyze each part or topic. Each dedicated interpreter 604 can call one or more information extractors 606 as needed to extract information such as matrices and definitions from each part or topic of the specification. Each information extractor 606 can call one or more dedicated NLP tools 608 as needed. Each dedicated NLP tool 608 can then call one or more core NLP libraries 610 to help process each part or topic of the specification.

[0065] The output result of the core NLP library 610 may be returned to the dedicated NLP tool 608 that calls the core NLP library 610. The output result of the dedicated NLP tool 608 may be returned to the information extractor 606 that calls the dedicated NLP library 610. The output result of the information extractor 606 may be returned to the dedicated interpreter 604 that calls the information extractor 606. The output result of the dedicated interpreter 604 may be returned to the main interpreter 602 that calls the dedicated interpreter 604.

[0066] In some embodiments, the output of the specification interpreter 600 may include basic data structures and driving logic. The basic data structures and driving logic may include logical containers and other necessary or related structures. For example, the data structure may hold: planned patient visits and dosage information; data driver information (e.g., BMI for dosage); data related to supply chain organization, supply shipments, and / or patient kits; or any other data that is expected to be collected or acted upon during the trial. The logical container may hold specific processing instructions for the prediction system to comply with the specification, as necessary.

[0067] Figure 7 An example graphical display 700 of a randomized patient list is shown that may be displayed by a user after validating (or attempting to validate) the randomized patient list. Figure 1 The verification system generates and outputs. The display may include an ordered plurality of coded adjacent squares or other shapes, each of which corresponds to an individual patient or test kit. The shapes may be ordered in the same manner as the patient list. The coding of the squares may be color-coded, pattern-coded, or other coding that can visually distinguish one square type from another square type. In the illustrated embodiment, the squares are color-coded, with a first color assigned to patients in the active drug group and a second color assigned to patients in the placebo group. Thus, by manual inspection, a user can quickly assess the distribution of patients in a trial or in a portion of a trial to better understand the numerical and statistical verification information generated by the verification system.

[0068] Figure 8 and Fig. 9 Example graphical displays 800, 900 of randomized kit lists are shown, which may be displayed after verifying (or attempting to verify) the randomized kit list. Figure 1 The display 800 and the display 900 are related to the same list of reagent kits, but are ordered differently. The first display 800 is ordered by reagent kit sequence (i.e., all reagent kits of the same type are grouped), and the second display is ordered by reagent kit number. Similar to Figure 7 , the test kit list display 800, 900 can be color-coded, pattern-coded, or coded in other ways to achieve visual distinction of one square type from another square type. In the illustrated embodiment, the squares are color-coded, each of the four test kit types having its own corresponding color. Therefore, in display 800, the user can quickly visually confirm that the relative quantities of different test kit types are appropriate. In display 900, the user can quickly evaluate the distribution of the test kit in the test or in a portion of the test, including the length of the batch, the sequence of the batch, etc., to better understand the numerical and statistical verification information generated by the verification system.

[0069] Fig.10 1 is a diagrammatic view of an example implementation of a user computing environment that includes a general computing system environment 1000 such as a desktop computer, laptop computer, smart phone, tablet computer, or any other such device having the ability to execute instructions, such as instructions stored in a non-transitory computer-readable medium. In addition, although described and illustrated in the context of a single computing system 1000, those skilled in the art will also understand that the various tasks described below may be practiced in a distributed environment having multiple computing systems 1000 linked via a local area network or a wide area network, in which the executable instructions may be associated with and / or executed by one or more of the multiple computing systems 1000.

[0070] In its most basic configuration, the computing system environment 1000 typically includes at least one processing unit 1002 and at least one memory 1004, which can be linked via a bus 1006. Depending on the exact configuration and type of the computing system environment, the memory 1004 can be volatile (e.g., RAM 1010), non-volatile (e.g., ROM 1008, flash memory, etc.), or some combination of the two. The computing system environment 1000 can have additional features and / or functions. For example, the computing system environment 1000 can also include additional storage devices (removable and / or non-removable), including but not limited to magnetic or optical disks, tape drives, and / or flash drives. Such additional storage devices can be accessed by the computing system environment 1000 through, for example, a hard disk drive interface 1012, a magnetic disk drive interface 1014, and / or an optical disk drive interface 1016. As will be understood, these devices, which will be linked to the system bus 1006 respectively, allow: reading from and writing to the hard disk 1018, reading from or writing to the removable disk 1020, and / or reading from or writing to the removable optical disk 1022, such as a CD / DVD ROM or other optical media. The drive interface and its associated computer-readable media allow non-volatile storage of computer-readable instructions, data structures, program modules, and other data for the computing system environment 1000. Those skilled in the art will also understand that other types of computer-readable media that can store data can be used for this same purpose. Examples of such media devices include, but are not limited to, magnetic tape cassettes, flash memory cards, digital video disks, Bernoulli cassettes, random access memories, nano drives, memory sticks, other read / write and / or read-only memories, and / or any other methods or technologies for storing information such as computer-readable instructions, data structures, program modules, or other data. Any such computer storage media can be part of the computing system environment 1000.

[0071] Many program modules may be stored in one or more of the memory / media devices. For example, a basic input / output system (BIOS) 1024 containing basic routines such as those that help transfer information between elements within the computing system environment 1000 during startup may be stored in ROM 1008. Similarly, RAM 1010, hard disk drive 1018, and / or peripheral memory devices may be used to store computer-executable instructions including an operating system 1026, one or more application programs 1028 (which may include, for example, an operating system 1026, and one or more application programs 1028). Figure 1The randomization list generation and verification system 102 and / or the RTSM system 104 functions), other program modules 1030 and / or program data 1022. In addition, the computer executable instructions can also be downloaded to the computing environment 1000 as needed, for example, via a network connection.

[0072] An end user may enter commands and information into the computing system environment 1000 through input devices such as a keyboard 1034 and / or a pointing device 1036. Although not shown, other input devices may include a microphone, a joystick, a game controller, a scanner, and the like. These and other input devices will typically be connected to the processing unit 1002 through a peripheral interface 1038, which in turn will be coupled to the bus 1006. The input devices may be connected to the processor 1002 directly or indirectly via an interface such as a parallel port, a game port, FireWire, or a universal serial bus (USB). To view information from the computing system environment 1000, a monitor 1040 or other type of display device may also be connected to the bus 1006 via an interface, such as via a video adapter 1032. In addition to the monitor 1040, the computing system environment 1000 may also include other peripheral output devices not shown, such as speakers and printers.

[0073] The computing system environment 1000 may also utilize logical connections to one or more computing system environments. Communications between the computing system environment 1000 and a remote computing system environment may be exchanged via another processing device responsible for network routing, such as a network router 1052. Communications with the network router 1052 may be performed via a network interface component 1054. Thus, in such a networked environment, such as the Internet, the World Wide Web, a LAN, or other similar types of wired or wireless networks, it will be understood that program modules or portions thereof described with respect to the computing system environment 1000 may be stored in (one or more) memory storage devices of the computing system environment 1000.

[0074] The computing system environment 1000 may also include positioning hardware 1056 for determining the location of the computing system environment 1000. In implementations, the positioning hardware 1056 may simply include, for example, a GPS antenna, an RFID chip or reader, a WiFi antenna, or other computing hardware that may be used to capture or send signals that may be used to determine the location of the computing system environment 1000.

[0075] Although the present disclosure has described certain embodiments, it should be understood that, except as explicitly stated in the claims, the claims are not intended to be limited to these embodiments. On the contrary, the present disclosure is intended to cover substitutions, modifications, and equivalents that may be included in the spirit and scope of the present disclosure. In addition, in the detailed description of the present disclosure, many specific details are set forth in order to provide a thorough understanding of the disclosed embodiments. However, it is obvious to those of ordinary skill in the art that systems and methods consistent with the present disclosure can be practiced without these specific details. In other cases, well-known methods, processes, components, and circuits are not described in detail to avoid unnecessarily obscuring various aspects of the present disclosure.

[0076] Some parts of the detailed description of the present disclosure have been presented in terms of processes, logic blocks, processing and other symbolic representations of operations on data bits in a computer or digital system memory. These descriptions and representations are means by which a person skilled in the art of data processing uses the essence of his or her work to most effectively convey to other persons skilled in the art. In this article, processes, logic blocks, processing, etc. are generally conceived as self-consistent sequences of steps or instructions that produce a desired result. These steps are steps required for physical manipulation of physical quantities. Typically, although not necessarily, these physical manipulations take the form of electrical or magnetic data that can be stored, transmitted, combined, compared and otherwise manipulated in a computer system or similar electronic computing device. For convenience and with reference to common usage, with reference to various currently disclosed embodiments, such data are referred to as bits, values, elements, symbols, characters, items, numbers, etc. However, it should be remembered that these terms should be interpreted as referring to physical manipulations and quantities, and are merely convenient labels that should be further explained according to terms commonly used in the art. Unless otherwise specifically noted, as is apparent from the discussion herein, it should be understood that throughout the discussion of the present embodiments, discussions utilizing terms such as "determine" or "output" or "send" or "record" or "locate" or "store" or "display" or "receive" or "identify" or "utilize" or "generate" or "provide" or "access" or "inspect" or "notify" or "deliver" etc. refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data. Data is represented as physical (electronic) quantities within the registers and memories of a computer system, and data is transformed into other data similarly represented as physical quantities within computer system memories or registers or other such information storage devices, transmission devices, or display devices as described herein or otherwise understood by those of ordinary skill in the art.

Claims

1. A method for managing a clinical trial for a drug, the method comprising: receiving, by the server from the user, one or more electronic files containing parameters of the clinical trial; performing, by the server, natural language processing on the one or more electronic documents to determine the parameters; generating a randomized patient list based on the determined parameters; The randomized patient list is verified by the server by: verifying that the patient list includes balance or spread above a balance or spread threshold; as well as verifying that the list has a randomness above a predetermined randomness threshold; and The RTSM system is configured by the server according to the validated randomized patient list.

2. The method according to claim 1, further comprising: A kit for distributing and administering the drug to patients according to the configured RTSM system.

3. The method according to claim 1, further comprising: Verify that the patient list contains valid content by one or more of the following operations: verifying the total number of rows and the total number of blocks in the patient list; Verify the number of rows and blocks for one or more treatment groups; Verifying the number of rows and the number of blocks for one or more group and layer combinations; or Verify the size and number of multiple block types.

4. The method according to claim 1, wherein: Verifying that the patient list includes a balance or spread above a predetermined threshold includes: computing multiple simulations of a patient registry; and The equilibrium or spread is calculated based on the simulation.

5. The method according to claim 4, wherein: Computing multiple simulations of patient enrollment includes assuming equal likelihood of patient enrollment in each of multiple strata or groups of the trial.

6. The method according to claim 1, wherein: Verifying that the list has a randomness above a predetermined randomness threshold comprises: determining a block pattern distribution within the list; calculating the probability of obtaining a distribution of block patterns that is at least as extreme as the observed block pattern distribution; and The probability is compared to the predetermined randomness threshold.

7. A method for managing a clinical trial for a drug, the method comprising: receiving, by the server from the user, one or more electronic files containing parameters of the clinical trial; performing, by the server, natural language processing on the one or more electronic documents to determine the parameters; generating a randomized kit list based on the determined parameters; The server verifies the randomization kit list by performing the following operations: Verifying that the kit list has a randomness above a predetermined randomness threshold; The server configures the RTSM system according to the verified randomization kit list.

8. The method according to claim 7, further comprising: A kit for distributing and administering the drug to patients according to the configured RTSM system.

9. The method according to claim 7, wherein: Verifying the randomized kit list includes verifying that the kit list contains valid content by one or more of the following operations: Verify the total number of rows and the total number of blocks in the kit list; or Verify the number of rows and the number of blocks for one or more reagent kit types.

10. The method according to claim 7, wherein: Verifying that the list has a randomness above a predetermined randomness threshold comprises: For a non-blocked list, determining that the probability of the distribution of batch lengths in the reagent kit list exceeds the threshold; or For a block list: determining a block pattern distribution within the list; calculating the probability of obtaining a distribution of block patterns that is at least as extreme as the observed block pattern distribution; and The probability is compared to the predetermined randomness threshold.

11. A method for managing a clinical trial for a drug, the method comprising: receiving, by the server from the user, one or more electronic files containing parameters of the clinical trial; performing, by the server, natural language processing on the one or more electronic documents to determine the parameters; generating a randomized patient list and a randomized kit list based on the determined parameters; The randomized patient list is verified by the server by: verifying that the patient list includes balance or spread above a balance or spread threshold; as well as verifying that the list has a randomness above a predetermined patient list randomness threshold; The server verifies the randomization kit list by performing the following operations: Verifying that the kit list has a randomness above a predetermined kit list randomness threshold; as well as The RTSM system is configured by the server according to the validated randomized patient list and the validated randomized kit list.

12. The method according to claim 11, further comprising: A kit for distributing and administering the drug to patients according to the configured RTSM system.

13. The method according to claim 11, wherein: Validating the randomized patient list includes verifying that the patient list contains valid content by one or more of the following operations: verifying the total number of rows and the total number of blocks in the patient list; Verify the number of rows and the number of blocks for one or more treatment groups; Verifying the number of rows and the number of blocks for one or more group and layer combinations; or Verify the size and number of multiple block types.

14. The method according to claim 11, wherein: Verifying that the patient list includes a balance or spread above a predetermined threshold includes: computing multiple simulations of the patient registry; and The equilibrium or spread is calculated based on the simulation.

15. The method according to claim 14, wherein: Computing multiple simulations of patient enrollment includes assuming equal likelihood of patient enrollment in each of multiple strata or groups of the trial.

16. The method according to claim 11, wherein: Verifying that the patient list has a randomness above a predetermined patient list randomness threshold comprises: determining a distribution of block patterns within the patient list; calculating the probability of obtaining a distribution of block patterns that is at least as extreme as the observed block pattern distribution; and The probability is compared to the predetermined randomness threshold.

17. The method according to claim 11, wherein: Validating the kit list includes verifying that the kit list includes valid content by one or more of the following operations: verifying the total number of rows and the total number of blocks based on the one or more electronic files; or Verify the number of rows and the number of blocks for one or more reagent kit types.

18. The method according to claim 11, wherein: Verifying that the kit list has a randomness above a predetermined randomness threshold comprises: For a non-blocked reagent kit list, determining that the probability of the distribution of batch lengths in the reagent kit list exceeds the threshold; or List of block kits: determining a block pattern distribution within the kit list; calculating the probability of obtaining a distribution of block patterns that is at least as extreme as the observed block pattern distribution; and The probability is compared to the predetermined randomness threshold.

19. The method according to claim 11, wherein: The randomized patient list is a second randomized patient list, and the randomized kit list is a second randomized kit list, the method further comprising: generating a first randomized patient list and a first randomized kit list based on the determined parameters; attempting to validate, by the server, the first randomized patient list; attempting to verify the randomized kit list by the server; determining, by the server, that the randomized patient list or the randomized kit list is not valid, and in response: outputting an error by the server; and The second randomized patient list and the second randomized kit list are determined by the server.

20. The method of claim 11, further comprising generating and outputting a report of validation of the randomized patient list and the randomized kit list, the report comprising a graphical representation of one or more of the randomized patient list or the randomized kit list.

Citation Information

Patent Citations

  • Content development and distribution using cognitive sciences database

    CN101379496A

  • Error recognition and correction method for clinical research subject randomized grouping and treatment method allocation

    CN108242268A

  • Systems and methods for demand and supply forecasting for clinical trials

    CN111357005A

  • Method and system for automated pharmaceutical, biomedical and medical device research and reporting

    CN1914615A