Improved clinician-centered clinical trial verification system and method

Abstract

To provide a method for electronic verification of a source for a source capture in a clinical trial workflow system.SOLUTION: The method includes the steps of: opening an EMR system; logging into the EMR system on behalf of a user by using securely stored credentials; capturing screenshots of EMR pages containing desired information for answering questions on a study subject; and verifying that the captured screenshots are of the correct study subject by using data points. When the defined number of data points is clearly identified in the captured screenshots, the captured screenshots are labeled as having been verified.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 122,677, entitled "IMPROVED CLINICAL TRIAL VERIFICATION SYSTEM AND METHOD," filed December 8, 2020, and PCT application PCT / US21 / 33900, entitled "CLINICIAN-CENTERED CLINICAL TRIAL VERIFICATION SYSTEM AND METHOD," filed May 24, 2021, which are incorporated by reference in their entireties herein.

[0002] The present invention provides an improvement to a clinical trial validation system that allows for a more efficient and streamlined workflow for answering clinical trial questionnaires and entering information. The improvement includes additional electronic validation that the correct patient record is being used to create the source capture. Furthermore, in addition to providing greater certainty about the origin of the source capture, the present invention can also provide validation that the EMR record used to create the source capture is for the correct patient and visit date. [Background technology]

[0003] This idea relates to the field of clinical research, where experiments are performed on humans to evaluate the safety and effectiveness of new medical procedures. The process of data collection for clinical trials begins with a dialogue between the doctor (researcher) and the patient (subject), which is documented in the patient's medical record and considered a legal document. The medical record consists mainly of alphanumeric characters written by the doctor to describe the dialogue with the patient, and is considered the first point at which a record is created, which is called the source document (SD). The doctor records information in the SD, for example, about the patient's complaints, previous conditions, medications, allergies, physical examination findings, laboratory tests, imaging tests, plans. However, for the purpose of clinical research, only a subset of the information in the SD is needed. For example, a clinical trial on hypertension may only require information about blood pressure, and only blood pressure needs to be extracted as relevant data from the SD, even though the SD could also contain information written by the doctor about the patient's prostate exam.

[0004] Manufacturers of medical procedures frequently enlist the help of researchers to conduct clinical studies. These manufacturers are known as sponsors. Sponsors create forms (also known as case report forms or CRFs) to be filled out by researchers and / or their hospital staff. CRFs contain questions about the subjects relevant to the clinical trial. Answers to these questions must be submitted by the researchers in order for the sponsors to collect data about the effectiveness and safety of their medical procedures. Upon receiving the completed CRF, the sponsor then transcribes the information from the CRF into a database. Because this transcription process is a problematic point for potential human error, sponsors often enter the data twice to later correct transcription errors. This process is known as double data entry (DDE).

[0005] It has been recognized that another potential step that can introduce human error is the process in which information is transcribed from the SD to the CRF by the investigator or their staff. To minimize this possibility of error, sponsors implement a process in which the information written on the CRF is checked against the information in the SD. This process is known as source document verification (SDV). In practice, this requires personnel representing the sponsor to take the CRF to the investigator's office to view the SD and perform a comparison. This process is very cumbersome and time consuming, requiring a lot of travel and cost.

[0006] Over the past two decades, the industry has adopted new technologies that allow CRFs to be presented to researchers in electronic form. An electronic CRF (or eCRF) is generally a web-based form viewed via a computer with an area of ​​text representing a question, followed by one of a variety of mechanisms, such as open text fields, radio buttons, drop-down menus, etc., that allow the researcher to submit a response. However, the process of completing the online form representing the eCRF is still performed by the researcher with reference to the patient's medical record (SD). Thus, it remains a transcription process and prone to human error. Thus, the use of eCRFs still requires sponsors to perform quality control checks through SD verification.

[0007] However, there are advantages to using eCRFs because there is no longer a need for paper forms to be faxed between the researcher and sponsor, and electronic forms are easier to read than handwritten forms. Also, eCRFs can be programmed so that potential errors can be flagged by the eCRF at the time of data entry. For example, if the researcher enters a heart rate value of 800, the eCRF can be programmed with computer code to detect when the entered value is out of range. Such computer code is called an edit check. In this example, an edit check could be used to alert the researcher that a heart rate of 800 is out of range. The researcher would be able to react immediately to the problematic data and make a correction. This same example would result in many days being lost in the data cleaning process for a clinical trial if a paper CRF was used, as the doctor may not notice the error, the paper CRF would be faxed to the sponsor, the erroneous data would be entered into the database, and days to weeks later, an edit check would be programmed and applied at the database level to flag the data as being out of range.

[0008] These flags raised against problematic data are known as queries. A report of the queries must then be printed and faxed back to the investigator, who attempts to resolve the queries by referencing back to the SD. This process of cleaning the data is time consuming for any clinical trial and typically adds weeks to months or even years to the drug development cycle. While the use of eCRFs can save some time in the data cleaning process, significant time and expense is still incurred in the process of programming the edit checks. In summary, whether the sponsor uses paper or eCRFs, weeks to months are spent by data management staff to program edit checks for each of the CRF questions. Also, the process of transcribing information from the SD to the CRF (or eCRF) remains a point of potential human error and therefore still requires the sponsor to perform SDV.

[0009] One prior art process for validating clinical trial source data is described in U.S. Patent No. 7,669,114 to Wood, which teaches software and hardware for a clinical research system that includes creating new paper CRF data collection documents with unique identifiers and that are organized and tracked by the unique identifiers in the local system. The CRF documents are then uploaded to a central system and reviewed to accept them as clean or to request clarifications. The original paper CRFs filled out by the researcher can be represented in a digital manner and sent over the network from clinics on the same network with the same software installed. Using Wood's invention, there is no longer a need to fax or mail forms to pharmaceutical companies or to create "three-part" carbon copy forms. The physician prints out a blank CRF directly from the Wood system and then gets a paper copy to fill out. Importantly, the physician still fills out the paper CRFs using his own handwriting. The completed forms are then scanned back into digital form and fed into Wood's software. The completed forms are then sent over a network to a "central system" that becomes a database for tracking the documents. Wood's patent also describes scanning separate source documents (documenting doctor-patient interactions) into the system and sending them to the central system in the same manner as CRFs. Similarly, any data queries made by the pharmaceutical data administrator can be scanned into Wood's system and sent in digital form to the doctor. However, source document validation must still be performed in Wood's system because the doctor is still transcribing information from the source documents into a paper CRF. Because the source documents are scanned into the same system as the CRFs, there is no longer a need to travel to the doctor's office to perform source document validation.Pharmacy personnel can verify the information on the CRF from anywhere there is a network connection and Wood's software system. Although the CRF is described in Wood as "digital," it lacks the ability to directly accept data onto the CRF, requiring that the CRF be printed on paper to be completed by a doctor or nurse and then scanned back into the system. In other words, the form is electronic, but the data capture is not.

[0010] Nos. 10,706,958, 10,811,122, 16 / 919,396, and 17 / 021,456, which are incorporated herein by reference in their entirety, teach a clinical trial research system and method that allows researchers to create electronic data documents (EDDs) in response to clinical trial research case report forms (CRFs). As part of the EDDs, snippets are created by medical professionals to answer questions on the CRFs. Instead of doctors transcribing information from a patient's chart into an electronic CRF (eCRF), doctors use digital images of the source documents. For example, they are asked to log into their electronic health record (EHR) (or electronic medical record (EMR)) system, select the portions of the screen that correspond to the information that answers each of the questions on the eCRF, and create an image (such as a JPEG file) of the portions of the screen. The clinical trial data collection workflow has a medical professional at the clinical site capture a screen image and then create a separate snippet for each CRF question. The medical professional then reviews, signs, and sends the completed form to the sponsor. Once received by the pharmaceutical company, pharmaceutical data management personnel use the snippet to transcribe the information into the corresponding fields in the clinical database by accessing the unmasked portions of the image contained in the EDD. The pharmaceutical company receives image information directly from the doctor's official electronic medical record. With this approach, there is usually no reason to double-check or verify the answers by later reference to the source document (SD). However, many of the transcribed data points may be edited later to correct errors. In these cases, when the data is changed, the medical professional must re-review and re-sign the document. This may happen multiple times, which creates inefficiencies.

[0011] Recently, the industry has created several systems that "integrate" with some popular EMR back-end data systems through APIs and SDKs published by EMR manufacturers. Although these systems may result in rapid acquisition of data from the EMR software, they have been slow to be adopted. This is for several reasons that are addressed by the present invention.

[0012] One reason for the lack of adoption of existing systems is the sheer number of EMR manufacturers, each with their own integration interface. This creates difficulties of scale when deploying a back-end integration system in a clinical trial. A single clinic or hospital using a less popular EMR system causes the entire study to require a hybrid process.

[0013] Another reason for the lack of adoption of existing systems is that EMR integration requires interaction with the EMR vendor and hospital systems to set up the integration. This often requires the deployment of an on-site server and the creation of a VPN tunnel to securely extract the data. Even if an integration is already developed with a given EMR system that does not exist, it still requires a deployment process with vendor and site agreements and slight customization for each individual site. This is a daunting task and can be hampered by bureaucratic processes or technical obstacles at any individual site. The present invention overcomes this problem by requiring only user credentials comparable to on-site onboarding for a typical clinical user and no vendor interaction or agreements.

[0014] A third reason for the lack of adoption of existing systems is that EMR manufacturers and hospitals have not placed much emphasis on clinical trials as a primary source of revenue. This results in a lack of major stakeholder alignment and little incentive for EMR manufacturers and hospitals to collaborate on integration beyond the stock interfaces offered through their APIs or SDKs. This has resulted in very little progress in this clinical research methodology over the past decade. The present invention overcomes this issue by interacting with the EMR in the same way that a user does. Whatever a user can do, the present invention can do. This removes the limitations imposed by EMR manufacturers on the types of integrations that are possible. It also removes the incentive structure regarding the types of integrations that are possible.

[0015] Pharmaceutical companies tend to keep their data transfer and informatics systems proprietary because they do not want research data to easily leave their firewalls. Thus, pharmaceutical companies' computer and informatics systems are not easily "plugged in" to hospital systems. Thus, there is a need to provide a convenient, reliable method that complies with patient privacy restrictions to securely and confidentially transmit and verify original patient data recorded in hospital systems to provide the necessary clinical trial data to pharmaceutical companies' informatics systems that have engaged such hospital systems to participate in clinical trials for medical procedures developed by the companies. There is also a need to efficiently create EDDs and allocate the work required to create and verify them among the parties (clinical researchers and sponsors) who are in the best position to do what is needed. [Prior art documents] [Patent documents]

[0016] [Patent Document 1] U.S. Pat. No. 7,669,114 [Patent Document 2] U.S. Pat. No. 10,706,958 [Patent Document 3] U.S. Pat. No. 10,811,122 [Patent Document 4] U.S. Patent Application Serial No. 16 / 919,396 [Patent Document 5] U.S. Patent Application Serial No. 17 / 021,456 Summary of the Invention

[0017] The present invention provides a computerized system and method for enabling a medical field person, such as at least one medical professional (researcher), to answer questions on a clinical trial questionnaire (such as an electronic case report form (eCRF)) associated with a clinical trial of a medical procedure by performing a source capture and defining specific areas of the source capture image that contain the answers. The system enables a workflow in which images of the answers, e.g., screenshot capture snippets, captured through a mouse click are used instead of transcription of alphanumeric values.

[0018] In a further embodiment, the system allows a medical professional, such as a medical professional user, to restructure how questions are answered in an eCRF by allowing the medical professional to answer and organize questions in an order that best suits a particular EMR.

[0019] In a further embodiment of the present invention, after at least one source capture (SC) is performed at the site of care and a snippet is created to initially answer an eCRF question, the system of the present invention remembers where within the SC of the EMR page the particular question of the eCRF was answered. The system of the present invention provides a recommendation for a question-answer pair or creates an EDD for a subsequent visit selected by the site of care where the medical professional user accepts / confirms the suggestion of the system of the present invention.

[0020] Another embodiment of the invention uses Robotic Process Automation (RPA) to automate elements of the existing source capture and source document verification (SDV) workflow. This includes automating the process of accessing the EMR, finding the correct EMR document or data, extracting EMR data in the form of both screenshots of electronic source documents, and extraction of individual data through automated copy and paste and through direct reading of data from either Windows® format (for native Windows® EMR applications) or HTML Document Object Model (for web EMR applications), redacting personally identifiable information (PII) through PII pattern matching, OCR, and machine learning, and sending the redacted SC and SDV data to a centralized server for storage, collation, analysis, and utilization. In a further embodiment of the invention, a medical professional creates a source capture, matches CRF questions with portions of the source capture, and forwards this information to a non-medical site user, who creates snippets to perform data entry and offload the work required from the medical site. The medical professional at the clinical site will not have to review and sign the final CRF package for accuracy and will not have to review each and every change that may be made during the process.

[0021] The present invention provides a clinical trial research workflow method including the steps of a field professional selecting a clinical trial study; the field professional selecting a patient in the clinical trial study, where a list of patient visits is provided; the field professional selecting a particular patient visit; the field professional restructuring the clinical trial questions on the clinical study form to match an order of information viewable on the active operating system; the field professional creating a source capture of relevant information based on the particular patient visit viewable on the active operating system; the field professional selecting questions from the clinical study form and associating the relevant information on the source capture with questions from the clinical study form, where the questions from the clinical study form are restructured to be answered in the order of the information on the active operating system; the field professional providing the associated source capture to a clinical trial management professional; the clinical trial management professional creating a snippet; the clinical trial management professional entering the information from the snippet into a clinical trial database;

[0022] Another embodiment of the invention provides a clinical trial research workflow method for answering clinical trial questions, including the steps of: the field professional restructuring the clinical trial questions to match the order of viewable information on the active operating system; the field professional creating a source capture of a source document for the first patient visit; the field professional selecting a question from a clinical study form and associating relevant information on the source capture with the question from the clinical study form, where the question from the clinical study form is answered in the order of viewable information on the active operating system; the field professional creating additional source captures for the remaining patient visits; the system associating information from the additional source captures with the relevant clinical trial questions to create question-answer pairs; and the field professional approving or rejecting the system question-answer pairs with an electronic signature.

[0023] Another embodiment of the present invention provides a clinical trial workflow method including the steps of: a medical field professional selecting a clinical trial study; the medical field professional selecting a patient in the clinical trial study, where a list of patient visits is provided; the medical field professional selecting a particular patient visit; the medical field professional creating one or more source captures of the particular patient visit information viewable on an active operating system; the medical field professional providing the one or more source captures to a clinical trial management professional; the clinical trial management professional associating information on the one or more source captures with questions from a clinical study form, where the clinical trial management professional continues to associate the information on the one or more source captures until each of the questions on the clinical study form is answered; the clinical trial management professional creating a snippet using the associated information and question pairs; the clinical trial management professional entering information from the snippet into a clinical trial database; and an investigator reviewing the entered information and approving the entered information with an electronic signature.

[0024] Another embodiment of the present invention provides a clinical trial workflow method using robotic process automation, the method comprising the steps of initiating an EMR system; logging into the EMR system on behalf of a user using securely stored credentials; automatically selecting the correct navigation link in the EMR system and completing an appropriate navigation form to reach an EMR page with desired information for the correct study subject record and visit date; capturing one or more screenshots of one or more relevant pages containing answers to one or more questions on a Case Report Form (CRF); automatically marking the one or more captured screenshots corresponding to areas containing answers to respective questions on the CRF to create snippets; and permanently redacting areas for personally identifiable information before the one or more screenshots leave the robotic process automation system, which uses optical character recognition and automatic marking of the EMR page with the appropriate information. identifying portions of the one or more captured screenshots containing personally identifiable information using a mapping of the individual text data taken from the EMR page by the OCR and RPA system to determine alphanumeric data represented by the one or more captured screenshots; identifying sections of the one or more captured screenshots containing answers to one or more CRF questions using the OCR and mapping of the individual text data read from the EMR page by the RPA system to determine alphanumeric data represented by the one or more captured screenshots; providing suggestions for data entry using the identified sections; assembling an electronic data document (EDD) by combining and associating the snippets with the one or more captured screenshots, the one or more questions from the CRF, the alphanumeric data represented in the snippets, and a complete audit trail of actions; transmitting the EDD to a server over a secure network; automatically selecting a logout button on the EMR system for the user; andand providing a report to a user summarizing the actions taken by the RPA system and the EDD.

[0025] In further embodiments, devices and methods for adding additional security using the system of the present invention further reduce the risk of inadvertent error. This verification can be driven by OCR and programming logic, secure, encrypted communications ("handshakes") between the software of the present invention and the EMR system, checks for active EMR application windows, inter-process communication (IPC), and web-based APIs.

[0026] Another embodiment of the present invention provides an improvement to a clinical trial research workflow system and method that provides a method for electronic verification of information source for source capture in a clinical trial workflow system, the method includes the steps of initiating an EMR system, logging into the EMR system on behalf of a user using securely stored authentication information, automatically selecting the correct navigation link in the EMR system and filling in the appropriate navigation form to reach the EMR page related to desired information for the correct study subject's record and visit date, capturing one or more screenshots of one or more relevant pages containing the desired information related to answering one or more questions on a case report form (CRF), and verifying that the one or more captured screenshots are of the correct study subject by using one or more data points for verification. Verification may occur with real-time scanning by the clinical trial system of the one or more screenshot captures to detect the presence of keywords of one or more data points, evaluating the one or more captured screenshots as verified or unverified, and labeling the one or more captured screenshots as verified if a defined number of the one or more data points are clearly identified in the one or more captured screenshots by the clinical trial system, and labeling the one or more captured screenshots as unverified if a defined number of the one or more data points are not clearly identified by the clinical trial system. Verification may also occur, with electronic verification including secure, encrypted, two-way communication including a handshake when a connection is established to verify that the EMR system provides the source of information for the one or more captured screenshots, and, once the source is verified, requesting the EMR system to confirm that it is displaying information about the correct study subject. [Brief description of the drawings]

[0027] Further objects, features and details of the invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings illustrating exemplary embodiments of the invention.

[0028] [Figure 1] A typical screenshot of a user study selection page. [Diagram 2] 1 is an exemplary screenshot of a landing page for a user performing a source capture. [Diagram 3] 1 is an exemplary screenshot for a medical professional user showing a patient visit status. [Figure 4] A typical screenshot showing the CRF questions on a SC implementer landing page. [Diagram 5] A typical screenshot of the CRF Questions tab showing the SC implementer. [Figure 6] 1 is a typical screenshot of the SC Executor showing the application selection window. [Figure 7] A typical screenshot showing a patient record open in the EMR. [Figure 8] A typical screenshot showing information edited on the SC. [Figure 9] 1 is an exemplary screenshot showing the system of the present invention overlaid on an existing application. [Figure 10] A typical screenshot of an SC performer showing the SC HUD. [Figure 11] A typical screenshot showing a snippet generated using the Markup Palette. [Figure 12] A typical screenshot showing a snippet about SC in form view. [Figure 13] A typical screenshot showing the CRF progress screen for a SC performer. [Figure 14] 1 is a representative screenshot showing a landing page for a data entry user. [Figure 15] 1 is an exemplary screenshot showing a data entry user landing page for backup data entry with validator. [Figure 16] 1 is an exemplary screenshot showing a landing page for a user to review a completed CRF package. [Figure 17] 1 is a flowchart of an exemplary workflow 1. [Figure 18] 1 is a flow chart of an exemplary workflow 2 with data entry performed by a supervisor. [Figure 19] 1 is a flow chart of an exemplary workflow 3 with data entry performed by a data administrator. [Figure 20] 1 is a flowchart of an exemplary workflow 3. [Figure 21] 1 is a flow chart illustrating an example embodiment of an SC validation process for an EMR in accordance with the present invention. [Figure 22] 1 is a diagram illustrating another exemplary embodiment of an SC verification process for an EMR system according to the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] The present invention provides a computerized system and method that allows the researcher (doctor, nurse, or other designated medical professional) to provide the sponsor with SC corresponding to the answers to the questions posed by the CRF or eCRF. SD image sections (snippets) of the SC are then prepared with question-answer pairs to address each of the questions on the CRF or eCRF. Snippets can be digital photographs, or the output of an imaging device, which can be a digital camera, computer screen capture software, a mobile phone, a medical scanner or imaging machine, and a portable computer. Snippets also include direct electronic or visual recordings that extract audio or visual segments or both (but not metadata). This includes, for example, direct electronic recordings from telemedicine interactions. Because the eCRF questions are answered by images that come directly from the SD, it is no longer a transcription process, thus eliminating the need to perform SDV. Also, because the SD is the ultimate resource of the original data, there is virtually no need to program edit checks to assess the quality of the data or to perform other time-consuming data cleaning. The SD is considered a legal document and the information therein is considered final. Also, in this modern day, most clinics and hospitals have adopted Electronic Medical Records (EMRs) and these systems already contain complex error checking programming. The present invention allows sponsors to leverage the error checking already present within the EMR system.

[0030] The present idea presents a process whereby the sponsor can perform one or two data entry directly based on the relevant parts of the SD, thus skipping the step of the researcher entering the CRF with alphanumeric data. The present invention includes a method whereby the researcher enters the CRF with the SC of all or selected parts of the SD where the relevant alphanumeric information is visible. The SD image can be captured by a digital camera for paper source documents, or by a screen capture function in the software for electronic SD. To protect patient privacy and direct the sponsor's attention only to the relevant information, as required by the Health Insurance Portability and Accountability Act (HIPAA), FDA Title 21 CFR Part 11, and other laws and regulations, the researcher can mark the image with drawing and labeling tools. Also, to protect patient privacy, the areas not marked by the researcher are made invisible to the sponsor by editing (automatically or manually) so that the sponsor can see only the alphanumeric information on the SC that the researcher has determined to be relevant and corresponding to the clinical trial.

[0031] Since the investigator is no longer filling out forms but rather retrieving the SC from the original SD, there is no longer a risk of transcription errors. Since the sponsor is viewing the exact digital images of the SD compartments required for the clinical study, one or two data entry steps are performed directly from the SD images, thus eliminating or greatly reducing the need to perform SD validation directly.

[0032] The present invention may be implemented by a medical professional in a medical setting, such as a doctor or nurse, who takes the SC of a page (e.g., a screenshot or snapshot of a particular section on the page), labels the SC to identify the relevant data it contains (e.g., temperature, blood pressure, etc.), and adds to the SC any comments regarding the data or its application to clinical research. Thus, without having to enter patient information into a new form, the medical professional can identify information generally relevant to studies that may be conducted by various pharmaceutical companies. Similarly, the medical professional may take a photo (e.g., with a smartphone app) of a paper lab record that is not available on the computer, label the information with or without comments, and include such lab records electronically using the same program. Patient information that is not associated with a CRF question by the medical professional is electronically redacted, thus allowing only material relevant to the study to be shared electronically with the pharmaceutical company. The company receives this information in such a manner that only the portion of the record selected by the medical office is shown, surrounded by the redacted portion of the SC, or alternatively, showing only the relevant data in each of the source captures, one by one. This in turn shows the pharmaceutical company all the data they need to see and not information that is not relevant to the study and / or that raises privacy concerns. Because sections of the medical record are selected and labeled by the medical office, the pharmaceutical company can use the Clinical Research Form (CRF) to decide what information they want to see for each study, and the present invention provides that information. This saves the pharmaceutical company time in having to review the entire medical file to verify the information. And, because only information relevant to the study can be viewed by the sponsor's reviewers, the information can be shared electronically without raising privacy concerns.

[0033] Furthermore, if the redacted portions of the medical file need to be reviewed for regulatory or tamper-evident purposes (e.g., by an FDA auditor, or senior management at the sponsoring company), users with appropriate authorization can view the redacted information in a limited but sufficient manner (e.g., viewport). Thus, the present invention hides sensitive information from general viewing, but retains that information as part of the redacted SC in the event that an authorized individual needs to view it, for example, to verify that the data being shared is in fact related to a particular patient.

[0034] The present invention provides a system with multiple user interfaces, including an interface for performing SC, an interface for performing data entry, and an interface for reviewing completed CRF packages for approval. The use of different interfaces provides personalization for specific roles performed by users. For example, a user preparing an SC, such as a medical professional, can convert eCRF questions into an EMR-compliant format that allows a medical professional user to answer eCRF questions in an easier and faster manner. The present invention is agnostic to the order in which questions appear on the CRF. This allows field users to organize the order in which they answer CRF questions to match the order in which the data appears in the electronic health record (EHR). The lack of synchronization in the prior art between the location of data on a page of the EHR and the order in which questions appear on the CRF results in a large amount of "page flipping" when field users respond to CRF questions using traditional methods. Using an OCR matching process, the system of the present invention searches the full text of the SC for every possible search term configured for every CRF question, unless the keyword is already assigned elsewhere (i.e., there is only one location in the EHR per search term for the patient encounter record). This allows the user to capture the SC of the SD and match the relevant locations of the answers to the CRF questions, without any constraints on the order in which the questions are presented on the CRF.

[0035] The system of the present invention also provides the ability to deviate from traditional clinical research workflows and roles. While the system of the present invention supports traditional roles, the system of the present invention optimizes workflow by minimizing the workload of the medical site. For example, a medical professional at the medical site attaches a source capture ("SC") from the EMR that answers every question on the eCRF list. This SC can be an individual part of the EMR or an entire page of the EMR record that includes the question-answer association. Personal patient information on the SC of the EMR is redacted, so that the patient's information on the EMR is anonymous on the SC. Once the medical professional creates the SC and the question-answer association, the medical professional user can close the eCRF questionnaire, and a non-medical site personnel, such as a sponsor or a third party, retrieves the associated questions and SC to prepare a snippet that includes the question-answer pair for the eCRF question. To further maintain confidentiality, the sponsor can use a third party to create the question-answer snippet and distribute the snippet to the parties performing the data entry, such as the sponsor or monitor. The prepared snippet is then used to perform the data entry. This optimized workflow shifts most of the work, including clerical work, away from the clinical site (which, depending on the workflow selected, may also prepare snippets on behalf of non-clinical site personnel). This is a significant departure from current industry practice, since the medical professional has always been the user tasked with submitting answers to CRF questions. However, since the source capture is the ultimate reference of truth, medical interpretation of the responses is no longer required, there is only a reference to a location on the source document that can be performed by a machine, verified by a medical user, and procedurally performed by a non-site user.

[0036] In a further embodiment, the eCRF forms and questions can be programmed in the system software of the present invention with keywords associated with each of the questions, allowing the system of the present invention to scan the SC provided by the medical professional to select and group the patient records and relevant information required. The system of the present invention allows the user to configure a database incorporating key search terms related to any given question on the CRF. When the user retrieves the SC of the SD, the system of the present invention applies optical character recognition (OCR) techniques to identify a match between the content of the SC and the search terms configured for the many questions on the CRF. If a match is identified, the area of ​​the SC is presented to the user as corresponding to the search term for the particular question. A skilled medical professional user familiar with the contents of the medical chart can determine whether the area identified by the OCR process in the system of the present invention accurately or inaccurately correlates to the associated question that is also presented to the user. If the correlation is accurate, the user accepts the match. If the correlation is not accurate, the user can reject the area matched via OCR. In a preferred embodiment, this process of "CRF question to source capture matching" allows the user to identify the location where the snippet should be created, but not have to create the snippet itself. The SC can then be sent to a non-site user (such as a CRO user or sponsor user in data management or clinical operations) to view the SC with the specific areas indicated as a match and run the snippet process (CROs are contract research organizations utilized by sponsors to perform the operational aspects of clinical trials, such as data management, site management, monitoring, etc.). The completed snippet is then used by the non-site user for alphanumeric data entry into a database.

[0037] In a further embodiment, the system of the present invention creates EDDs for all subsequent visits once the medical professional provides the initial SC with question-answer associations for the initial patient visit. This is done through a combination of OCR (to identify keywords from the CRF on the EMR page), pattern recognition, and artificial intelligence (AI). Because EMR pages are structured similarly for each visit, the system of the present invention recognizes patterns in the SC to provide suggestions for answers to CRF questions based on the location of answers in previous submissions.

[0038] Source Capture Interface

[0039] Source capture is typically performed by someone at the medical site, e.g., a doctor, nurse, or other medical professional at the medical site. In the following, a medical professional user is used to describe the interface, however, the interface can be used by any user delegated to perform source capture. With reference to FIG. 1, in an exemplary embodiment, the present invention provides a study selection screen where a medical professional user selects a clinical trial study from a menu 110 that provides a list of all clinical trial studies available to the signed-in user based on the user's access details. The medical professional user can also select a configuration environment from a menu 112 based on the user's project access details.

[0040] Referring to FIG. 2, in an exemplary embodiment, the present invention provides a user landing page 200 in a grid view. Other views, such as list views, are also available. The landing page 200 is specific to and tailored to the user's role. The medical professional user's information 204 is provided at the top of the landing page 200. The landing page 200 provides a list 208 of all clinical trial patients for the clinical site (in the selected clinical trial) and a list 210 of all patient visits. The patient list 208 provides the patient's unique study ID (or the patient's initials) and a progress status indicator (e.g., red / green / yellow color indication) made regarding the completion of the CRF questionnaire. The patient list 208 is sortable by patient name or color status. The visit list 210 provides a list of the selected patient's visits by name and can be filtered by visit status, e.g., completed visits, canceled visits, overdue visits, and visits with unresolved issues. A user can select a specific patient and then a specific visit to collect information from that visit. The necessary information is then selected to create the SC and associate the information with the question and forwarded to the relevant party, such as the monitor, to create a snippet of question-answer pairs. This is an optimized workflow where the medical site only performs the SC and the monitor or third party performs snippet creation, data entry, etc. An additional workflow may also be followed where the medical professional does both source capture and snippet creation. For added convenience of the medical professional, a search bar 212 is provided on the landing page 200 that allows alphanumeric input to search for patients, visits, forms, and questions. New patients in the study may also be added.

[0041] Referring to FIG. 3, in an exemplary embodiment, the present invention provides a patient visit status 302. The visit status 302 may include, for example, canceled visits or missed visits along with the reason for such actions. The user may review the visits that have taken place and provide the associated date 304. A calendar widget may be used to remove ambiguity in data entry. This date becomes a date check against all SCs associated with the visit. Ad hoc, unscheduled visits may be added to the system by pulling from forms or collections of data points already set up for a particular clinical trial.

[0042] Referring to FIG. 4, an exemplary embodiment is shown having CRF questions that are displayed on the landing page of the SC performer once the user indicates that a visit, e.g., a screening visit, has occurred. The CRF form provides an automatically populated clinical trial question on the landing page 400. The user can select the type of clinical trial question they want to answer via tab 402 - "No Source" questions or "Source Capture" questions. Based on the tab selected, the relevant questions appear on the screen. "No Source" questions are simple questions that do not require source evidence for an answer. They can be listed and answered by the user using a response widget such as yes or no, or the comment feature provided. "Source Capture" questions require source evidence, which requires the medical professional user to associate a SC with each of the listed CRF questions.

[0043] Referring to FIG. 5, in an exemplary embodiment, the present invention provides a Source Capture tab 502. These questions are organized into two groups - questions attached to an SC "Attached to Source Capture" 506 and questions not attached to an SC "In Progress" 504. The user must associate all or part of an SC with every question on the list. Once all questions have SCs attached, the user's job of answering clinical trial questions may be completed, depending on the workflow selected. FIG. 5 shows that blood pressure, temperature, pulse, respiratory rate, allergies, serum potassium, serum sodium, creatinine, ECG QT interval, and pain scale are questions that are still in progress and do not have SCs attached. (These questions are just examples of what may be asked). Using the Source Capture icon 508, the user may begin the process of adding SCs.

[0044] 6, in an exemplary embodiment, the present invention provides an application selection window 600. When a user selects the source capture icon 508, the application selection window 600 opens. All software programs open in the operating system appear as icons on the application selection window 600. The user can select and browse within the various selected applications to open relevant records for SC. Once the user selects the software program they wish to use, e.g., the EMR application 602, the user interface of the present invention is minimized to be an adaptable transparent floating widget that remains over the open active application.

[0045] 7, in an exemplary embodiment, an open patient record in an EMR is shown. Icon 700 is the system of the present invention, transparent and floating over the active EMR application.

[0046] Referring to FIG. 8, in an exemplary embodiment, the present invention provides automatic and manual redaction of information and redaction authentication using electronic signature of the SC. Patient's personal information 810 is automatically redacted from the SC 850. This automatic redaction of personal information is done on the medical professional's machine, for example in a Google chrome® add-on, or on a server such as Amazon Web Services. On the client side, the patient's personal information is not sent to the server / internet on the SC for redaction processing. (The patient's authentication associated with the SC is done by the machine and then redacted before sending the SC over the network). On the server side, the OCR and redaction is fast and uses the computing power of the server architecture to do something computationally intensive. A screenshot is taken and the image is securely transferred to the server in a completely unredacted but encrypted format. Once the screenshot reaches the server, all OCR and redaction processing is done using temporary memory such as RAM and no PII is recorded on permanent storage. The user can select between client side and server side OCR and redaction using, for example, a toggle selection. The SC 850 is placed in the system pane 800 of the present invention overlaid on an open active system application 860, e.g., an EMR system. The automatic edits may use optical character recognition (OCR) to find matches for edited fields, e.g., date of birth, social security number, address, phone number, email address, hospital ID, insurance company, insurance ID, and patient ID photo section. The edited information is blocked from unauthorized users with an overlay, which may be a dark block. The user may also use the editing tools 816 to manually edit information that was not automatically edited. For example, the patient photo 820 is manually edited with the editing tools 816 by selecting the information and surrounding it with a box. The manually edited area shows an "X" icon on the section when the cursor is placed over it. After the edits are made, the user approves both the automatic and manual edits with an electronic signature 840. Manual edits must be applied to each of the SCs individually.A medical professional typically performs manual editing; however, a supervisor, for example, may also perform manual editing of the SC.

[0047] Referring to FIG. 9, in an exemplary embodiment, the present invention provides a pane 900 that overlays an existing application 960 on the system. The pane 900 includes a SC 950 and an SC markup window 920. The markup window 920 lists the patient's study ID and the visit name and date. The markup window 920 also includes a questions list 930 that lists all of the questions for the specified visit that require an SC and are not paired with an SC response. The user can select a suggested list of questions that may have answers found in the SC 950 or a list of all study questions that require an SC. When a suggested list is selected, OCR can be used to search for keywords in the SC 950 that may answer the CRF questions. (These keywords for search purposes can be set up during study configuration). For example, the questions list 930 provides a list of questions that may have supporting evidence found in the SC 950, such as temperature, blood pressure, medications, pulse rate, respiratory rate, allergies, and lab findings. Hovering over / highlighting a particular question on the Question List 930 highlights the corresponding section on the SC 950 containing suggested keywords for that question, e.g., Blood Pressure 970. This helps the user quickly find answers to CRF questions. Clicking the arrow next to the suggested question adds a node to a pull-out drawer to place the question in the SC Question Heads-Up Display (HUD) 940. The HUD 940 is a semi-transparent drawer overlaid on the SC 950. The HUD 940 is permanently linked to the SC 950. Once a question is added to the SC HUD 940, it is removed from the Question List 930. If there is no supporting evidence on the SC 950, the user may also remove the question from the Question List 930 by clicking the "x". The user may also drag the question from the SC HUD 940 back to the SC Question List 930.

[0048] 10, in an exemplary embodiment, the present invention provides an SC HUD 1040 and computer software tools used to annotate SCs. When a question 1035 is associated with an SC 1050 and added to the SC HUD 1040, the question 1035 is removed from the question list 1030 and an SC markup palette 1080 appears, since the question 1035 can only occur once. The markup palette 1080 includes tools related to the functionality available to the user's role. (For example, the data entry interface may also include a markup tool for snippet creation. User roles are established in the administration module of the software). The markup palette 1080 includes a snippet tool 1082, an editing tool 1084, a multi-selection tool 1086, and a zoom tool 1088. The markup palette 1080 may be a floating rectangle graphic that can be moved and reconfigured. The editing tool 1084 allows the user to manually select the information to be edited, as discussed above. The multi-select tool 1086 allows the user to select multiple pieces of information on the SC that are not vertically or horizontally aligned by drawing a markup box around the desired information and then drawing another box from the edge of the previously drawn markup. Each subsequent region created with the multi-select tool 1086 is attached to the previous selection of the subsequent region with a connecting line. The zoom tool 1088 allows the user to zoom in or out on only the SC image 1050. The snippet tool 1082 allows the user to create a snippet by drawing a markup rectangle 1083 around the desired information for the snippet. It allows the user to define which section of the screen capture represents evidence of an answer to a particular question on the clinical trial questionnaire. The result is one or more subsections of the larger screen capture, each representing a particular answer to the clinical trial question. Once the markup rectangle 1083 is completed, the labeling tool 1081 provides a drop-down list of all the questions in the question list 1030.The user may select a related question with which a markup rectangle 1083 is associated. This causes the markup rectangle to park in place with a green border and change the status indicator node to green, informing the user (or other viewer) that this question has been answered / completed and a snippet has been created. These smaller screen capture sections with associated answers are snippets. (Depending on the workflow selected, snippet creation may be done by a non-clinic user, such as a supervisor, instead of a clinician user as discussed above). Snippets include color-coded boxes with editing options and a comment box to inform the sponsor of any additional information. This snippet creation process can be repeated until all of the information on the SC that is relevant supporting evidence for answering the CRF question has been paired with the related question. The created snippet may be edited later by the user. However, once a snippet is approved by the sponsor, it is locked and can no longer be edited. A lock symbol may be used to inform the medical professional user (or other snippet creator) that the snippet can no longer be edited.

[0049] Referring to FIG. 11, in an exemplary embodiment, the present invention provides snippets 1155 created using a markup palette 1180. The process is repeated for each new SC page. A status symbol is provided to notify the medical professional user when questions have been answered for each patient visit. The status symbol is context sensitive depending on the user's role. If their role is to obtain source captures and assign CRF questions to their SCs, the status marker will display green when their SC task is completed. If a user has been assigned a role that requires marking up source captures to create snippets, then their status symbol will display red for any questions until they create a snippet-question pair for the question. For users configured for both source capture and snippet creation, the status symbol will remain red until the SCs are created and associated with CRF questions, and then each of the CRF questions has a snippet-question pair.

[0050] 12, in an example embodiment, snippets 1255 on a SC 1250 are shown in a form view. The form view only shows snippets 1255 that were created from the SC 1250 and paired with a question. Other information that was on the SC but was not paired with a question is not shown. A native view is the other viewing option.

[0051] 13, in an exemplary embodiment, a progress screen is provided once the user has a completed SC or the user chooses to exit the SC process. Questions listed under "In Progress" 1304 still require an SC associated with the question. (For users who also have snippet / markup capabilities assigned to their role, similar progress screens are provided for CRF questions that have snippet pairs or still require snippet associations). The user may provide notes for in-progress unanswered questions, such as data unavailable, source unavailable, set a reminder, or open a query. Questions under "Attached to Source Capture" 1306 have been answered using an SC. (Snippets are created from the SC attached to each of the questions). The system notifies the user if there are any unresolved queries associated with answers by placing an alert icon 1308 next to the questions. An asterisk 1310 next to the questions indicates the presence of an audit trail. The audit trail provides a history of all actions taken, including responses provided, data entered, data changes, and marked-up SC images, with detailed date / time stamps, reasons for changes, and users who performed the actions. Once a non-clinical site personnel, such as a supervisor or data custodian, accepts the snippet, the question is italicized, for example, "Pulse" in FIG. 13, to lock further editing to the medical professional user. At the bottom of the screen, a tray 1350 of all the SCs used to answer the listed CRF questions is available to the medical professional user. Hovering over an SC highlights related questions that were answered from the highlighted SC page.

[0052] Data Entry Interface

[0053] Data entry can be performed by any party. This includes clinical or non-clinical personnel, such as nurses, monitors, data entry administrators, third parties, or physicians. Sponsors may delegate a clinical trial management professional to perform the data entry. As with the source capture interface, users of the data entry interface log into the system of the present invention and select the study and configuration environment they want to work with.

[0054] Referring to FIG. 14, in an exemplary embodiment, the present invention provides a landing page 1400 to a data entry user. The screen is specific to the data entry user's role and the selected study and environment. The data entry user may receive SCs with question-answer pair snippets from the clinical site user. The snippets are used to inform the data entry. However, if the data entry user is creating snippets in addition to data entry, the data entry user receives SCs with questions associated with the SCs created by the SC user. The data entry user uses this information for snippet creation and data entry. If the user's role is snippet creation, a markup palette, as discussed above, including snippet creation tools, is available to the data entry user. If the same data entry user is creating both snippets and data entries, they can simultaneously create snippets and enter data. The present invention allows data entry users to organize snippets or EDDs based on the clinical sites for which they are responsible. Users can select which data points they want to view based on the data entry stage, or select all to view all of the currently unprocessed items. The site column 1420 provides a list of sites for the user and data entry specific team members. When a data entry user selects a site, the data points column 1430 opens a list of data points. These data points can be filtered using filter presets 1410 that filter data points based on the stage of data entry, such as initial entry, backup entry, verification, or flagging. When a data point is selected from the data points column 1430, the details of the data point are displayed in a data processing details panel 1440. The data processing details panel 1440 can include a data processing page header 1450, a CRF question 1455, a snippet box 1460, a data entry widget 1470, and automatic OCR suggestions 1465. The data processing page header 1450 provides information about the data point.This information may include subject ID, site name, site ID, CRF name, nature of the request, e.g., backup entry request, and time of last processing step. Auto OCR suggestions 1465 provide data entry suggestions based on the associated snippet. As the first data entry user, the data entry user may need to accept or reject the snippet if another observer (non-clinical site personnel) has not already done so. The data entry user accepts the SD snippet if the snippet is supporting evidence for the answer to the question and is representative of the answer to the question. The data entry user rejects the SD snippet if the SD snippet is not supporting evidence for the answer to the question and is not representative of the answer to the question. (If the snippet is rejected, the data point is processed for query placement). The data entry user enters information from the accepted snippet box 1460 into the data entry widget 1470. Once data entry is complete, the user submits the information and the system of the present invention checks to ensure the information has been entered, sends the information to the audit trail, and changes the data to read-only text. The data entry information can still be edited at this point. If data is missing, the system will provide an error unless the field has indicated that no data exists for this data point. This can be set up when the system is configured.

[0055] Referring to FIG. 15, in an exemplary embodiment, the present invention provides a data entry landing page for backup data entry using a corroborator. A second data entry user, the corroborator, may be added for quality control. Once the data entry user enters information and the status for the data entry user becomes complete, the status indicator for the second data entry user, e.g., a quality control ("QC") user, is set to incomplete. The second data entry user may provide backup entry or verification. A data point is sent to the data point column of the landing page for the second data entry user selected to perform this function. Upon entering data as a backup corroborator, the system of the present invention automatically evaluates whether the data entered by the first data entry user is the same as the data entered by the second data entry user, i.e., the QC user. If the entered data is not the same, the entry user is notified that a match was not achieved and an additional corroborator may be added. If the entered data is the same, a match is achieved 1575. The data entry user may send information to a database or request verification. Once an entry is sent to the database, the audit trail is updated, the status indicator on the data point is set to completed, and the data point is removed from the data point list 1530. The system stores a data entry history, including a list of all previous data entries with the user name and data entered. This also applies to other data entry stages, such as initial entry, validation, and flagged data points.

[0056] Authorization Interface

[0057] Similar to the source capture interface, researchers log into the system of the present invention and select the study and configuration environment they want to work with. This user interface is for a specific role with the primary function to review and electronically sign the fully finalized version of the completed CRF page and keep a record of the fully finalized version of the completed CRF page. This user has the ultimate responsibility for the conduct of the clinical trial in the hospital or clinic. This is typically done by a researcher or a physician, a specialist in the medical field.

[0058] In the current workflow used in the industry, the researcher reviews and signs the data on the CRF document multiple times when there are edits to the originally submitted answers. In the present invention, only the completed and finalized CRF pages are presented to the researcher for review and signature. Using the disclosed preferred inventive workflow, snippet creation is performed by the downstream user at the clinical site. Thus, as a next step after the clinical site user provides the SC, the data is fully cleaned and finally entered into the database. Once the data is entered into the final database, the researcher can review and approve the data after any steps of data cleaning have been resolved. This is a significant departure from current practice, making the inventive system and method capable of enabling such a workflow.

[0059] Referring to FIG. 16, in an exemplary embodiment, the present invention provides a researcher landing page 1600. The researcher landing page 1600 includes a patient column 1620, a form column 1630, and a form details pane 1650. The researcher landing page 1600 differs from the source capture interface landing page in that the researcher page lists completed packages for the researcher to review and confirm / approve. These packages exist at the end of the process, e.g., after data entry and other cleaning queries have occurred. The system of the present invention notifies the approving user when the data associated with a package has been reviewed by the researcher and is ready to be electronically signed with a ready to sign icon 1655 next to the package. The ready to sign icon 1655 may appear at any or all of the three package levels (e.g., at the form level with all associated questions, at the visit level with all associated forms, or at the patient level with all associated visits). The ready to sign icon 1655 indicates that the data is complete with associated SCs and has been sent to the database and has no outstanding queries. The patient column 1620 includes the site name which, when selected, provides a list of patients or an archive folder. All of the information may be sorted alphabetically. A list of visits for that patient may be shown or hidden to the researcher next to the patient's name. When a patient visit is selected, the form associated with the selected visit is displayed in the form column 1630. The selected form is then displayed in the form details pane 1650, which may include a header 1651, a review icon 1652, a snippet of question-answer pairs 1653, and an electronic signature section 1654. The header 1651 may include, for example, the patient study ID, the date of the visit, and the form name. The review icon 1652 allows the researcher to view the original SC to verify the source of the snippet's question-answer pairs. The SC may be viewed alone or in a split screen that allows the user to view both the SC and the form simultaneously.The researcher may view the SC in a native view or also in a form view. The question-answer pair snippet 1653 provides, for example, the full text of the question written in the CRF and the answer sent to the database by the data entry user. Clicking on the icon 1655 opens the e-signature section 1654 at the appropriate package level in the form details pane 1650. The e-signature section 1654 contains affidavit text indicating that the e-signature guarantees that the researcher has reviewed the data and SC and guarantees that the data is correct. When the e-sign button is clicked, the e-signature is applied to the package, the form details pane is closed, and the signed content is moved to the archive folder 1660 in the patient column 1620. It is a regulatory requirement that physicians keep clinical trial records for themselves for 15 years. At any time, physicians can download the complete records of all their patients (e.g., in pdf or uncomplicated text) by accessing the archive folder 1660.

[0060] Robotic Process Automation

[0061] An embodiment of the invention uses Robotic Process Automation (RPA) to automate elements of existing source capture and source document verification (SDV) workflows, including automating the process of accessing the EMR, finding the correct EMR document or data, extracting EMR data in the form of both screenshots of electronic source documents and extraction of individual data through automated copy and paste and through direct reading of data from either Windows® format (for native Windows® EMR applications) or HTML Document Object Model (for web EMR applications), redacting personally identifiable information (PII) through PII pattern matching, OCR, machine learning, and sending the redacted SC and SDV data to a centralized server for storage, collation, analysis, and utilization.

[0062] The RPA system of the present invention is developed for each electronic medical record system used at clinical sites participating in the clinical trial by mapping and automating the user interactions required to access patient records in these EMR systems, verify patient identity, and find relevant clinical trial data for extraction. The adapted RPA solution is then deployed to automatically extract the SC and SDV data required for a given clinical trial and is coordinated through interactions with a centralized web service (API) that assigns automated RPA data extraction tasks for given study subjects and data fields. Based on the secure web service task assignment, the automated RPA system accesses the EMR records to extract the data assigned through the task. (The RPA system operates remotely or at the client site, depending on network construction, data, and legal restrictions). The RPA web service communication is protected by end-to-end TLS encryption, which is securely established over insecure channels using industry standards such as TLS, Diffie-Hellman key exchange, and AES encryption. The RPA system caches the extracted data locally in a secure encrypted container using 256-bit+AES encryption for all stored data. The RPA system maintains appropriate authentication information for the EMR system to interact with that is provided by clinical sites participating in the clinical trial and securely stored using 256-bit+AES encryption. The redaction process can occur on the client side or the server side. The client side redaction process ensures that all PII extracted from the EMR is stripped from the SDV and SC data before transmission to the web service, ensuring that the web service never processes the PII. PII is never persisted in long-term storage on the RPA system, but only maintained in temporary memory until redaction is complete.

[0063] After submission to the web service, the RPA extracted SC and SDV data is verified by a human reviewer to ensure that no PII is present and that the data is accurate. Patient identity is verified through OCR checks against the pre-redacted SDV screenshot, checks against the individual EMR data extracted by the RPA, and verification of the steps taken by the RPA system (through automation) regarding the user navigation process to access the data. These verifications ensure that the redacted data without PII is assigned to the correct study subject. If the subject cannot be found by the RPA or these verifications fail, an error is raised to a human operator who can intervene to address the issue, find the correct data, and address any errors in the RPA process.

[0064] For example, a clinical site nurse or research coordinator may log into the system of the present invention and select a clinical trial, and a particular study subject within that trial. They may be able to further drill down to a particular clinical trial patient visit. When they do, they are presented with a series of forms that require completion for the selected visit, and a button to "automatically complete selected forms." Selecting the automate button initiates the RPA process, which recognizes the relevant EMR page based on the nurse's previous selection of study subject, desired visit date, and form. The RPA of the present invention may then automatically proceed through, for example, the following steps:

[0065] Open the EMR software if it is not already open.

[0066] Logs into the EMR system on behalf of the nurse / user (if they are not already logged in) using securely stored credentials.

[0067] It automatically "clicks" on the correct navigational link in the EMR system and fills in the appropriate navigational form (eg, patient search form) to reach the desired information about the correct study subject record and visit date.

[0068] Capture screenshots of the relevant pages that contain your answers to the questions on the CRF.

[0069] To create snippets (cropped portions of an EMR page screenshot that represent answers to specific CRF questions), markings are automatically "painted" on a jpeg or other image file of the screenshot that correspond to areas that contain answers to individual questions on the CRF.

[0070] OCR is used to permanently redact areas with PII. Either client-side or server-side OCR may be used, and a configuration toggle may be provided for the user to select the preferred method. Client-side redaction is performed before the screenshot capture leaves the RPA system, by using OCR and a mapping of the individual text data read from the EMR page by the RPA system to identify areas of the screenshot that contain PII. The advantage of client-side processing is that there is zero transfer of PII, since any sensitive data is redacted immediately at the point of source capture. On the server side, the screenshot is sent in an encrypted manner to the server, where OCR is applied and redaction is performed before the redacted screenshot is sent back to the client. Redaction can be performed, for example, in a Google Chrome® add-on to the user's computer. On the server, for example, Amazon Web Service (AWS®), OCR and redaction use only temporary memory (RAM), so there is no permanent storage for the PII. Server-side processing may be the faster of the two options, however, if a country's data privacy regulations prohibit the transfer of PII from local residents to a server outside its borders, client-side editing may be the only option to perform the edits. In an effort to improve efficiency and speed, a queuing system may be provided that allows a field user to collect multiple screenshots in a session and then continue to perform other activities within the system while the OCR and editing process proceeds in the background. Eventually, the screenshots complete processing and the user is notified when the screenshots are ready for further interaction.

[0071] To provide suggestions for data entry, OCR and mapping of the individual text data read from the EMR page by the RPA system is used to determine the alphanumeric data represented by the image and to identify areas of the image that contain answers to the CRF questions.

[0072] The snippets are combined and correlated with screenshots of the EMR page, the questions from the CRF, the underlying alphanumeric data represented within the snippet, and a full audit trail of actions (date / time / user) to assemble a detailed Electronic Data Document (EDD).

[0073] Transmit the EDD (with the redacted PII) to a server over a secure network.

[0074] Initiate other processes already described to allow for human review of the EDD information and final data entry and / or quality checks.

[0075] Automatically "click" a logout button on the EMR system for the nurse / user and provide the nurse / user with a report summarizing the actions taken and the resulting EDD for review, comments, and edits.

[0076] Exemplary Workflow 1 as shown in FIG.

[0077] Step 1 - The nurse logs into the EMR and goes to the patient visit information. The nurse then opens the system of the present invention which presents a list of CRF questions requiring responses. The nurse first executes the SC 170 of any page in the EMR that contains information related to clinical trial questions. The SC images can be organized into a hierarchy, for example, by study subject visit date. The nurse then begins the process of marking up and editing 172 as needed, such as the patient's name. Any questions that do not require source documentation are entered as direct input, while questions that require source document evidence only require mouse-driven source markup. During the process of marking up source document type questions, the system automatically regroups the questions according to the markup of the questions on the particular SC page. (The nurse may also manually regroup questions through a dedicated UI).

[0078] During source markup 172, the system intelligently reorganizes the CRF questions to fit the evidence present on any given source capture. If a given source capture contains only four of seven answers for a CRF page with seven questions, the system recreates the page in the CRF to reflect only the four answerable questions and moves the remaining three questions to a new page that the system auto-labels (i.e., page name "Vital Signs" becomes "Vital Signs 2," etc.). Since the screen area captured within the EMR system remains roughly the same from visit to visit, the system remembers the grouping for future instances. When all questions have been completed, the source markup process is finished and the nurse's work is complete.

[0079] Step 2 - Completion of the work by the nurse brings a notification for the researcher (doctor) to log into the system. The doctor can see the markups, edits and data entries performed by the nurse and can edit any. Finally, when the doctor is satisfied, the system allows the doctor to provide an electronic signature 174. The researcher's work is complete.

[0080] Step 3 - Sign-off by the researcher results in notification to the monitor that the package of visit data is ready for them. The monitor logs into the system and opens the SC, but can only see the submitted responses to non-source document questions and those questions that have corresponding snippets. The monitor can use the system's "periscope" view to see the context surrounding any snippet, but cannot see edited sections of the SC. The monitor then systematically accepts 176 or rejects each of the responses. If a response is rejected, the system opens a dialog to allow the monitor to place a query 178 on the response, in the form of free text and / or choosing from a selection of categories. Once all responses from the sites have been addressed, the monitor's work is complete. Data accepted by the monitor is considered "locked" and should appear visibly, designated as such in the user interface. The site cannot modify locked data unless the monitor manually unlocks it by rejecting the data point and placing a query.

[0081] Step 4 - Any rejected response from the monitor results in a notification for the appropriate nurse to view the data points of the question. The nurse then logs into the system to see a list of "open queries" with direct links to the relevant SC pages. The nurse can use dialog boxes within the system to adjust the markup and / or respond to the monitor with free text. The nurse's query response is then re-queued for the monitor to re-examine and either accept (resolved query) or reject again (re-query) and repeat step 3.

[0082] Step 5 - Any accepted response from the monitor results in a notification for the data custodian staff to perform data entry (DE) 179 of responses to all questions requiring source document evidence. The system provides an interface to systematically examine each of the responses and transcribe the responses. In some cases, a double data entry workflow can be deployed where data is entered twice (by the same or different users) and the system flags discrepancies. The system also allows the data custodian to code the data at this point by allowing the data custodian to select from a menu of pre-determined options. The coding functionality can be performed by the same user who does the data entry, or by another user specifically designated for this purpose, using a workbench and a queue of data points ready for coding.

[0083] Step 6 - The system can be used to output the data in the form of a CSV file, with some parameters that allow the user to map fields to client specifications for purposes of labelling, consolidation, etc.

[0084] Exemplary Workflow 2 as shown in FIGS. 18 and 19

[0085] It is the same as workflow 1 above, except that the monitor can perform the source markup and subsequent data entry. In particular, the site performs the SC and edits 180, 190. However, while the site may not be performing the source markup, they can still regroup the questions so that any given SC is associated with at least one question from the CRF. The SC is digitally associated with the designated grouping of questions, which is then sent to the monitor as an individual package. The monitor is provided with an interface that allows the monitor to perform source markup 182, 192 according to the questions that have been grouped and attached to the SC by the nurse. Note that this method largely removes any need for queries (although the functionality still exists). It is conceivable that the monitor may be able to query the grouping associated with the SC by the nurse, but it would be rare for the monitor to query themselves about the source markup they just performed. Finally, in this workflow, the supervisor may perform data entry 189 immediately or may hand off the process to a data manager 199 as described in example workflow 1 above. The main difference between workflow 1 and workflow 2 is that the physician signatures 184, 194 in workflow 2 occur after the supervisor has completed all their work for the visit, not before.

[0086] Example workflow 3

[0087] It is the same as Workflow 2, except that the supervisor can perform SC 203. In detail, the nurse only needs to respond to questions 201, which do not require source document evidence. This then results in a notification for the supervisor to log into the EMR to take screenshots and perform markup 202. This method assumes that the supervisor has their own access to the hospital EMR system. The supervisor then performs data entry 209 and the doctor provides an electronic signature 205.

[0088] Image snippet safety improvements

[0089] The present invention provides an improvement to the embodiments presented above that adds additional security by validating the source of the SC, further reducing the risk of inadvertent error. This validation may be driven by OCR and programming logic, secure, encrypted communication ("handshake") between the software of the present invention and the EMR system, checking for active EMR application windows, inter-process communication (IPC), and / or web-based APIs. The improvement to the system of the present invention validates that the SC is legitimate by identifying several key pieces of data present on the SC. This information may include the patient's name (first and last name), the patient's date of birth, and / or the date of the patient's visit to the hospital or clinic.

[0090] The use of screen OCR technology allows for real-time scanning of the user's screen to detect the presence of keywords related to the current set of CRF questions and snippets. The use of OCR determines or ensures that the name of the patient for whom the snippet is being created is on the user's screen at the time of the question. For example, a snippet session begins and a screen reader is activated to search for keywords on the user's screen. Monitoring continues until the information is found, and if not, the session is considered "unverified."

[0091] Referring to FIG. 21, an embodiment of the verification process using OCR is shown. Each of the above data points is fed to the system of the present invention in a different way. The patient's name 2108 is provided to the system at the time of the screenshot source capture process 2120. The patient's name is considered PII and therefore is only processed in temporary memory on the server side and is never stored in a persistent medium. On the client / browser side, the patient's name 2108 (client cache) is cleared the moment the user logs out of the session 2135. This can be intentional or due to an automatic logout on inactivity based on some time threshold. The client cache may also be cleared the moment the web browser software (such as Google Chrome®) is terminated. Given this architecture, the system never stores the patient's name in a persistent medium either on the server side or the client side. The patient's date of birth 2100 is provided by the caregiver staff upon enrollment in the clinical trial 2110 and is available to the system of the present invention. The patient's date of birth 2100 is obtained separately and is not PII. The patient's visit date 2105 to the hospital or clinic is recorded in the EMR and provided to the system of the present invention by the caregiver staff for each visit. The visit date 2105 is also not PII.

[0092] When the SC process 2120 starts / occurs, validation is driven by the OCR 2130. The validation evaluation 2140 is based on what the system of the present invention can find, such as the patient's date of birth 2100, the patient's visit date 2105, and / or the patient's name 2108. The SC is evaluated as "verified" 2142 or "not verified" 2144 and marked with a visually distinct icon so that a user, such as a supervisor user, can see it. The SC is designated as "verified" 2142 if all three data points 2100, 2105, 2108 are positively identified by the system. The SC is designated as "not verified" 2144 if one or more data points 2100, 2105, 2108 are not identified on the system. The degree of strictness in the validation evaluation 2140 can be changed as a configuration setting, such that in some cases, two of three data points, or a certain combination of data points (such as the patient's name 2108 and date of birth 2100), may be deemed sufficient to designate a source capture as "verified" 2142.

[0093] When an SC is designated as "unverified" 2144, it is marked in the system so that a user, such as a supervisor user, can view a list of all "unverified" SCs, organized by researcher site, export the list to a spreadsheet or database, and print the unverified SCs on paper. A user, such as a supervisor user, can manually review the SCs 2146 in the system of the present invention to see that the correct SCs were captured, and can access the original source media to manually verify 2150 that the "unverified" source capture is "acceptable" (verified) 2152 or "rejected" (unverified) 2154. This allows the user to adjust the "unverified" SC list for quality assurance measures. Any remaining "unverified" SCs 2154 that the monitor manually verified 2150 and double-checked by viewing the original EMR and marked as "rejected" 2154 will be flagged for recapture of the source image 2160, and any snippets resulting from the "rejected" SCs 2154 will be detached from the CRF questions and returned to the researcher site to start over.

[0094] Another validation embodiment involves a device and process where there is a secure, encrypted communication ("handshake") between the software of the present invention that creates the SC and a computer window containing the content based on the EMR system where the SC originated. This "handshake" between the two software components is preferably a two-way communication whereby an electronic validation occurs every time an SC is created. This electronic validation is recorded so that a human readable log can be generated and can be added as a component to the EDD described in the above embodiment.

[0095] The two-way communication described above verifies that the window that is actively being used ("in focus") at the time of SC creation is, in fact, the EMR system. Additionally, once the two-way communication is created in a secure, encrypted manner, the EMR may be requested to send data, preferably from an application programming interface ("API") for the EMR system, that verifies that the window that is "in focus" is the medical record for the correct patient and / or correct visit to the clinic.

[0096] Referring to FIG. 22, an embodiment of this improvement to the system and method described above is shown. Validating the source of the snippet at the time of capture is a two-step process. The first step 2200, a "handshake," occurs when a connection is established and the EMR application / window providing the source of the snippet or SC is verified. Once the source is verified, the second step 2210 provides a means for the EMR system to verify that it is displaying information about a particular patient and / or a particular visit for that patient. Screen optical recognition (OCR), checking the active EMR application window, inter-process communication (IPC), and web-based APIs may be used to verify that the current EMR record in which the snippet is being created matches the EMR record of the expected subject.

[0097] Another verification embodiment checks for an active EMR application window and analyzes open windows on the user's operating system to generate information about whether a known EMR software application currently has an active window on the display. If the window has a title or metadata that matches the keywords and provides further insight into the current contents of the window, the SC is verified. Information such as the patient's name or ID and date of birth may be used. If the information is not available or does not match the queried keywords, the snippet is not verified. For example, when an SC session begins, the software of the present invention detects the operating system window. Available metadata for the keywords are queried. The presence or absence of the expected information determines verification.

[0098] Another verification embodiment provides and uses inter-process communication (IPC) that allows different processes (applications) to share or exchange information simultaneously, provided that the processes involved are all utilizing a given set of protocols, i.e., they all expect to accept messages and respond as necessary. Following a secure exchange (or handshake) of encryption keys, in which both the software of the present invention and the computer window containing the content from the EMR system identify themselves and the software and computer window verify the provided key, encrypted information can be transferred through the use of either pipes (one-way data channels) or sockets (endpoints for sending or receiving data). Once authentication is established, messages can be transferred between the two software components to establish whether the EMR is displaying information related to a particular subject. IPC requires the cooperation of the EMR application, which implements software protocols defined by the system of the present invention. These protocols do not require patient information, but only that the provided information about a particular subject, e.g., name or ID, is confirmed to be the information the application window is displaying at the time of the query. For example, when an SC session is initiated, the system of the present invention requests a connection with the EMR software through the operating system. Once a connection is established, the EMR software is asked to verify whether information about a particular patient is currently being displayed. The response or lack of response determines the validation.

[0099] Another embodiment of validation uses a web-based API that allows users, such as researcher users, to request information on their behalf directly from the EMR's web service, allowing the system of the present invention to generate not only information about what is being displayed, but also information that validates the user accessing it. Using standard secure internet protocols, such as OAuth, the system of the present invention is provided with a token that is sent to the EMR API with a request for information while the SC is being created. The EMR system validates the currently displayed patient record. This validation may also include more detailed non-sensitive patient information, such as the type of record being viewed by the user, which gives the SC data a higher degree of accuracy. This type of authorization token persists until it is revoked. Thus, the authorization token can be reused with each new SC event without repeating the authorization steps. For example, the system of the present invention grants authorization to the EMR web service, and the token is stored for later use. When an SC session begins, an API query is made to the EMR software web service to request validation that information about a particular patient is currently being displayed. The response determines the validation of the data source.

[0100] The improvements can provide additional functionality to the systems and methods of the invention that add further value. The improvements can allow validation between the systems of the invention and the SC / EMR to ensure that the correct patient information is being used.

[0101] General

[0102] The invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment containing both hardware and software elements. In an exemplary embodiment, the invention is implemented in software, including but not limited to firmware, resident software, and microcode. Any conventional EMR system can be used as the source of SC information, such as Cerner, Epic Systems, and Allscripts. Preferably, the system and method of the present invention is implemented in a client / server network connected via the Internet.

[0103] The system and method of the present invention also preferably uses data security, encryption, and data capture and transfer protocols that enhance patient privacy and security and add desirable authentication and verification features. Preferably, all data is transferred over an SSL connection (also known as HTTPS). Preferably, the system and method of the present invention uses data encryption (public / private key pairs) to protect the patient medical records represented in the SD media, and data encryption protects both the data and media while they are stored and while they are being transferred to ensure that only the intended recipient can access / view the data and media. Preferably, every user must be authenticated on the system by logging in with their private credentials. Preferably, during each interaction with the server, the server verifies the authenticity of the request for interaction by an authentication token issued by the server. Preferably, the system requires users to change their passwords periodically. Preferably, users only have access rights for functionality assigned to them by the system administrator. Preferably, information such as patient or subject ID, data capture date, and other necessary identifying information is not only included within the metadata, but is also embedded in the SC and SD images themselves and is accessible to EDD-qualified users and viewers.

[0104] In another preferred embodiment, the media or data is not stored on any local machine or device by the machine or device when it is created, or by the observer / sponsor or researcher when it is viewed. Rather, the data is captured directly from the screen output by the software of the present invention, and is not processed by the native operating system, which may write the data to disk, even if only as a temporarily cached file. Any additional image processing that may be required, such as file compression for storage, is handled by a server off the local machine or device. Preferably, the computer system and programs used in the embodiments of the present invention do not save the EDD or other files created incidental to the operation of the present invention as files that can be recalled at a later time. The SC can be obtained from any EMR software running on the same machine as the software of the present invention, such that the EMR software displays the information of the EMR software on the same screen as that accessible by the software implementing the present invention. Additional digital media imported by the present invention from any external source, such as photographs of paper documents or medical scans, are treated in the same manner as the SC once loaded.

[0105] Furthermore, the present invention may take the form of one or more computer program products available from a computer usable or computer readable medium providing program code for use by or in association with a computer system or any instruction execution system. The computer program product includes instructions for implementing the methods of the present invention. The computer usable or computer readable medium may be any apparatus capable of containing, storing, communicating, propagating, or transferring a program for use by or in association with an instruction execution system, apparatus, or device. The medium may be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device), or a propagation medium. Examples of computer readable media include semiconductor or solid state memory, magnetic tape, removable floppy disk, random access memory (RAM), read only memory (ROM), rigid magnetic disk, and optical disk. Current examples of optical disks include compact disk-read only memory (CD-ROM), compact disk-read / write (CD-R / W), and DVD.

[0106] A computer system suitable for storing and / or executing program code includes at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements include local memory utilized during the actual execution of the program code, mass storage, and cache memory that provides temporary storage of at least some of the program code to reduce the number of times the code is retrieved from mass storage during execution. Input / output (I / O) devices (including, but not limited to, keyboards, displays, pointing devices, etc.) can be coupled to the computer system directly or through intervening I / O controllers. Network adapters can also be coupled to the computer system to allow the computer system to be coupled to other computer systems, or remote printers or storage devices, through intervening private or public networks. Modems, cable modems, and Ethernet cards are just a few of the currently available types of network adapters. The computer system can also include an operating system and a computer file system.

[0107] It should be understood that the above description and examples are intended to be illustrative and not limiting. Many embodiments will become apparent to those skilled in the art upon reading the above description and examples. Therefore, the scope of the invention should not be determined with reference to the above description and examples, but instead with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. The disclosures of all articles and references, including patent applications and publications, are incorporated herein by reference for all purposes.

Claims

1. 1. A method for electronic verification of information sources for source capture in a clinical trial workflow system, comprising: initiating an EMR system for use with said clinical trial workflow system; logging into the EMR system on behalf of a user using securely stored credentials; capturing one or more screenshots of one or more relevant EMR pages containing the desired information to answer one or more questions about the study subject related to the clinical trial that are included in a Case Report Form (CRF); verifying that the one or more captured screenshots are of the correct study subject by using one or more data points for verification contained in the one or more captured screenshots; Equipped with If a defined number of the one or more data points are clearly identified in the one or more captured screenshots by the clinical trial workflow system, the one or more captured screenshots are labeled as verified.

2. 2. The method of claim 1, wherein the verifying step comprises: scanning the one or more screenshot captures in real time by the clinical trial workflow system to detect the presence of keywords for the one or more data points; and evaluating the one or more captured screenshots as verified or unverified, wherein if the defined number of the one or more data points are not clearly identified by the clinical trial workflow system, the one or more captured screenshots are labeled as unverified.

3. 3. The method of claim 2, further comprising manual review of the one or more captured screenshots labeled as not verified, wherein the user manually reviews the source of the one or more captured screenshots to determine that if correct study subject data was used in the one or more captured screenshots, the one or more captured screenshots are approved and labeled as verified, and if incorrect study subject data was used in the one or more captured screenshots, the one or more captured screenshots are rejected and discarded.

4. 4. The method of claim 3, wherein the EMR system verifies that it is displaying information about the correct study subject using screen optical recognition, inter-process communication, or a web-based API.

5. extracting alphanumeric data from the one or more screenshots of the one or more relevant EMR pages containing the desired information to answer the one or more questions about the research subject related to the clinical trial; and providing one or more data entry suggestions for input to the one or more questions from the CRF according to the extracted alphanumeric data.

6. The method of claim 5 , wherein the one or more data inputs include automatic OCR (optical character recognition) suggestions.

7. 7. The method of claim 6, further comprising the step of confirming, by a data entry user, the data entered into the CRF pursuant to the data entry suggestions.

8. The method of claim 1 , further comprising redacting personally identifiable information (PII) from the one or more screenshots.

9. 10. The method of claim 8, further comprising using OCR (Optical Character Recognition) processing to verify that no personally identifiable information (PII) is retained in the one or more screenshots.

10. 10. The method of claim 9, further comprising the step of storing the edited one or more screenshots associated with the corresponding one or more questions in the CRF.

Images