Systems and methods for proactive signal detection

Abstract

Disclosed herein are systems and methods for processing structured and unstructured data and identifying correlations between interventions (e.g., use of regulated products) and events (adverse and beneficial).

Description

[0001] SYSTEMS AND METHODS FOR PROACTIVE SIGNAL DETECTION

[0002] RELATED APPLICATIONS

[0003] This application claims the benefit of U.S. Provisional Application No. 63 / 467,824, filed May 19, 2023, which is incorporated by reference herein in its entirety.

[0004] BACKGROUND

[0005] Various products exist that are subject to safety monitoring / regulation. In the United States, for example, the U.S. Food and Drug Administration (FDA) regulates a number of products including: food; drugs; medical devices; radiation-emitting products; vaccines, blood, and biologies; animal and veterinary; cosmetics; and tobacco products (fda.gov / ).

[0006] SUMMARY

[0007] Systems and methods according to embodiments of the present invention can automatically process structured and / or unstructured data and determine correlations between specified interventions (such as, but not limited to, use of one or more regulated products) and adverse events, positive side effects, or both. The correlations that are identified may be scored and / or ranked, for example to indicate the relative strength of the underlying evidence available to support each of them.

[0008] In some embodiments, a system is provided for identifying correlations between interventions and events, wherein the system includes one or more processors configured, in accordance with instructions on a non-transitory computer readable medium, to process inputs comprising a list of one or more interventions and data from a plurality of sources, wherein the data includes structured data and unstructured data, and wherein the processing includes clustering, anomaly detection, natural language processing, causal feature analysis, and sensitivity feature analysis; determine, responsive to the processing, a presence in the data of a correlation between an intervention from the list of one or more interventions and an event; and output a list of one or more intervention-event combinations, each combination comprising an intervention from the list of one or more interventions and a correlated event.

[0009] In some embodiments, the processors are configured to identify correlations to adverse and beneficial events.

[0010] In some embodiments, the interventions comprise at least one of health-related regulated products and medical treatments.

[0011] In some embodiments, the data includes at least one of claims data, disease registries, electronic health / medical records, individual case safety reports, labeled signals data sets, and scientific literature.

[0012] In some embodiments, the causal feature analysis includes one or more of time to onset, challenge / de-challenge / re-challenge, event frequency, disproportionality, frequency trend.

[0013] In some embodiments, the sensitivity feature analysis includes one or more of: important medical event, designated medical event, event seriousness factors.

[0014] In some embodiments, the processors are further configured to generate a score for each intervention-event combination based on relative strength of evidence for the correlation between the respective intervention and event.

[0015] In some embodiments, a method of identifying correlations between interventions and events is provided, the method comprising processing inputs comprising a list of one or more interventions and data from a plurality of sources, wherein the data includes structured data and unstructured data, and wherein the processing includes clustering, anomaly detection, natural language processing, causal feature analysis, and sensitivity feature analysis; determining, responsive to the processing, a presence in the data of a correlation between an intervention from the list of one or more interventions and an event; and outputting a list of one or more intervention-event combinations, each combination comprising an intervention from the list of one or more interventions and a correlated event.

[0016] In some embodiments, the identifying comprises identifying correlations to adverse and beneficial events. In some embodiments, the interventions comprise at least one of health-related regulated products and medical treatments.

[0017] In some embodiments, the data includes at least one of claims data, disease registries, electronic health / medical records, individual case safety reports, labeled signals data sets, and scientific literature.

[0018] In some embodiments, the causal feature analysis includes one or more of: time to onset, challenge / de-challenge / re-challenge, event frequency, disproportionality, frequency trend.

[0019] In some embodiments, the sensitivity feature analysis includes one or more of: important medical event, designated medical event, event seriousness factors.

[0020] In some embodiments, the method further comprises generating a score for each intervention-event combination based on relative strength of evidence for the correlation between the respective intervention and event.

[0021] Additional features and advantages of embodiments of the present invention are described further below. This summary section is meant merely to illustrate certain features of embodiments of the invention, and is not meant to limit the scope of the invention in any way. The failure to discuss a specific feature or embodiment of the invention, or the inclusion of one or more features in this summary section, should not be construed to limit the invention as claimed.

[0022] DETAILED DESCRIPTION

[0023] In various embodiments, the systems and methods disclosed herein can detect / identify correlations between regulated products (or other interventions) and adverse events and correlations between regulated products (or other interventions) and previously unknown positive side effects. These correlations, which may have been previously unknown, can be referred to as “signals.” The signal detection / identification described herein can be proactive rather than reactive (e.g., initiated in response to a problem). Embodiments of the invention can also assign each signal a “signal score” that indicates the relative strength of the causal evidence available for the respective correlation.

[0024] As used herein, the term “regulated product” refers to any product where the manufacturer is required to perform continuous safety monitoring, including, for example, health-related products such as those regulated by the FDA (z.c., food; drugs; medical devices; radiation-emitting products; vaccines, blood, and biologies; animal and veterinary; cosmetics; and tobacco products). A regulated product explicitly includes all human pharmaceuticals, human medical devices, and human combination products. In addition, a regulated product explicitly includes animal pharmaceuticals, animal medical devices and animal combination products. A regulated product also explicitly includes cosmetics and vaping devices.

[0025] As used herein, the term “signal” refers to information arising from one or multiple sources, including observations and experiments, which suggests a potentially causal association (including, for example, a new aspect of a known association) between an intervention and an event or set of related events, either adverse or beneficial, that is judged to be of sufficient likelihood to justify further investigation. An intervention may include, for example, the use of a regulated product. An intervention may also include, for example, a medical treatment, such as surgery or psychotherapy. In the present embodiments, a “signal” is not constrained to an association with an adverse event (AE; undesired effect of a drug or other type of treatment, such as surgery). This is important as embodiments of the invention can detect signals between a regulated product and positive side effects, which may lead, for example, to drug repurposing.

[0026] As used herein, the term “signal score” refers to a numeric value that embodiments of the invention assign to each correlation or “signal” that indicates the relative strength of the causal evidence available for the correlation.

[0027] The systems of the various embodiments described herein may take on any of a number of configurations. In certain embodiments, the system is a server computer system with one or more processors in communication with one or more databases housing the list of intervention(s) and other inputs and non-transitory tangible computer readable medium, such as memory (e.g., ROM, NAS, etc.) containing instructions for providing the processors to perform the functions described herein. In certain embodiments, the system is a server-client system, with users having access to the functionality provided by the server (and related databases) via a workstation or other computing device (e.g., in communication with the server via the Internet or private network). In certain embodiments, the system provides a SaaS offering to remote end users.

[0028] Inputs

[0029] Systems and methods according to embodiments of the present invention can provide proactive signal detection with structured or unstructured data, including real world data. As used herein, the term “real world data” (RWD) refers to data relating to patient health status and / or the delivery of health care, and can be collected from a variety of sources. The systems can accept multiple inputs (e.g., stored in one or more connected databases or received via one or more data feeds) , including but not limited to the example inputs listed in Table 1. In some embodiments, the systems can accept all of the inputs listed in Table 1 and may require at least one. While labeled data sets can be used by embodiments of the invention and are known to increase consistency; a labeled data set is not required.

[0030] TABLE 1

[0031] In addition to inputs such as those listed in Table 1, systems and methods according to embodiments of the present invention utilize a list (e.g., in a database) specifying one or more interventions (e.g., regulated products).

[0032] Processing

[0033] Systems and methods according to embodiments of the present invention perform the following processing on the inputs received: Clustering and Anomaly Detection; NLP on unstructured data elements; Causal Feature Analysis; and Sensitivity Feature Analysis. This processing may be performed in any order, and may include one more methods e.g., various different anomaly detection methods, as in the Example below).

[0034] NLP

[0035] NLP (natural language processing) may include, but is not limited to, a machine learning model trained to mine adverse events from clinical notes.

[0036] Causal Feature Analysis

[0037] Causal Features are those features that impact the likelihood of causality between the product and the event.

[0038] Causal Feature Analysis may include, but is not limited to, analysis of one or more of the following: Time to Onset (of the event); Challenge / De-Challenge / Re-Challenge (i.e., the existence and prevalence of positive de-challenge); Event Frequency (i.e., the absolute or relative frequency with which an event occurs); Disproportionality (i.e., the contrast between observed and expected numbers of reports, for any given combination of intervention and event); Frequency Trend (i.e., the trend, if any, of the frequency with which an event occurs); Confounders (i.e., the presence of one or more variables that may cause a distortion in a measure of association). As used herein, the term “De-Challenge” refers to the clinical decision to withdraw or discontinue an intervention (e.g., administration of a drug) to monitor the effect on an adverse event, and the term “Re-Challenge” refers to the point at which a drug is given again to a patient after its previous withdrawal.

[0039] Causal Feature Analysis can increase or decrease the final Signal Score. For example, a shorter, rapid Time to Onset may result in an increased Signal Score. A reduction in an AE as part of a De-Challenge (and / or reintroduction or an increase of the AE as part of a Re-Challenge) may also increase the Signal Score. The Causal Feature Analysis may result in adjustment of the Signal Score, for example, by increasing / decreasing the score by a predetermined amount or by applying a multiplier / divider to the Signal Score.

[0040] Sensitivity Feature Analysis

[0041] As used herein, the term “product-event-combination” refers to a regulated product or other intervention combined with an event. The event may be an AE, or a positive side effect.

[0042] Sensitivity Features are those features that increase the sensitivity of a product-event- combination, without necessarily qualifying as causal evidence. For example, the product-event- combination of Product X and Patient Death does not, in and of itself, imply causality. However, as the event is extremely serious, interested parties will have a heightened sensitivity to the occurrence of this event, even in the absence of little causal evidence.

[0043] Sensitivity Feature Analysis may include, but is not limited to, analysis of one or more of the following: IME (important medical event); DME (designated medical event); Event Seriousness Factors (including, but not limited to: Fatality; Hospitalization; Disability; Congenital Defect). DMEs are serious and rare medical events that are often causally associated with drugs across multiple pharmacological or therapeutic classes.

[0044] Sensitivity Feature Analysis can increase the final Signal Score. For example, a signal with a hypothetical score of 30 based on Causal Feature Analysis may increase to 45 with the addition of Sensitivity Feature Analysis. This increase represents the relative greater protentional impact of the signal on public health, due to the seriousness of the adverse event. Output

[0045] Systems and methods according to embodiments of the present invention output a list of product-event-combinations. In some embodiments, each product-event-combination includes a Signal Score (see, e.g., the final score in the Example below). In some embodiments, statistical measures and records level information that influenced the Signal Score are included in the output.

[0046] Example

[0047] In this example, as outlined in Table 2, a correlation between Product A and Heart Attack was detected using one or more of the following inputs: ICSRs, EMRs, Claims Data, Disease Registries, and Scientific Literature.

[0048] During the processing performed by an embodiment of the invention, the clustering for Product A and a heart attack was loose, resulting in a clustering score of 5. However, several anomaly (e g., anomalous patient readings, patient health conditions or other anomalous data) detection methods identified the product and event combination, adding a total of 15 to the score.

[0049] During the Causal Feature Analysis, the embodiment detected a pattern of the DEC (drug-event combination) occurring (i.e., frequency), a pattern of rapid time to onset, along with a high disproportionality, adding to the score. However, the presence of confounders ( / .<?., other medications or conditions known to cause heart attacks) lowered the signal score by 10.

[0050] Because the event itself, heart attack, results in hospitalization and is listed as DME, the Sensitivity Feature Analysis added another 30 points to the score, for a total of 80.

[0051] TABLE 2

[0052] It is to be understood that the specific scoring in the foregoing example is for illustrative purposes only. Other scoring methodologies (e.g., including fewer or additional component scores and / or the amount by which the overall score is increased and / or decreased by each component) is within the scope of the embodiments.

[0053] While there have been shown and described fundamental novel features of the invention as applied to the preferred and illustrative embodiments thereof, it will be understood that omissions and substitutions and changes in the form and details of the disclosed invention may be made by those skilled in the art without departing from the spirit of the invention. Moreover, as is readily apparent, numerous modifications and changes may readily occur to those skilled in the art. For example, various features and structures of the different embodiments discussed herein may be combined and interchanged. Hence, it is not desired to limit the invention to the exact construction and operation shown and described and, accordingly, all suitable modification equivalents may be resorted to falling within the scope of the invention as claimed. It is the intention, therefore, to be limited only as indicated by the scope of the claims appended hereto.

Claims

CLAIMS1. A system for identifying correlations between interventions and events, wherein the system includes one or more processors configured, in accordance with instructions on a non-transitory computer readable medium, to: process inputs comprising a list of one or more interventions and data from a plurality of sources, wherein the data includes structured data and unstructured data, and wherein the processing includes clustering, anomaly detection, natural language processing, causal feature analysis, and sensitivity feature analysis; determine, responsive to the processing, a presence in the data of a correlation between an intervention from the list of one or more interventions and an event; and output a list of one or more intervention-event combinations, each combination comprising an intervention from the list of one or more interventions and a correlated event.

2. The system of claim 1, wherein the processors are configured to identify correlations to adverse and beneficial events.

3. The system of claim 1, wherein the interventions comprise at least one of health-related regulated products and medical treatments.

4. The system of claim 1, wherein the data includes at least one of claims data, disease registries, electronic health / medical records, individual case safety reports, labeled signals data sets, and scientific literature.

5. The system of claim 1, wherein the causal feature analysis includes one or more of: time to onset, challenge / de-challenge / re-challenge, event frequency, disproportionality, frequency trend.

6. The system of claim 1, wherein the sensitivity feature analysis includes one or more of: important medical event, designated medical event, event seriousness factors.

7. The system of claim 1, wherein the processors are further configured to: generate a score for each intervention-event combination based on relative strength of evidence for the correlation between the respective intervention and event.

8. A method of identifying correlations between interventions and events, comprising: processing inputs comprising a list of one or more interventions and data from a plurality of sources, wherein the data includes structured data and unstructured data, and wherein the processing includes clustering, anomaly detection, natural language processing, causal feature analysis, and sensitivity feature analysis; determining, responsive to the processing, a presence in the data of a correlation between an intervention from the list of one or more interventions and an event; and outputting a list of one or more intervention-event combinations, each combination comprising an intervention from the list of one or more interventions and a correlated event.

9. The method of claim 8, wherein the identifying comprises identifying correlations to adverse and beneficial events.

10. The method of claim 8, wherein the interventions comprise at least one of health-related regulated products and medical treatments.

11. The method of claim 8, wherein the data includes at least one of claims data, disease registries, electronic health / medical records, individual case safety reports, labeled signals data sets, and scientific literature.

12. The method of claim 8, wherein the causal feature analysis includes one or more of: time to onset, challenge / de-challenge / re-challenge, event frequency, disproportionality, frequency trend.

13. The method of claim 8, wherein the sensitivity feature analysis includes one or more of: important medical event, designated medical event, event seriousness factors.

14. The method of claim 8, wherein the method further comprises: generating a score for each intervention-event combination based on relative strength of evidence for the correlation between the respective intervention and event.CLAIMS1. A system for identifying correlations between interventions and events, wherein the system includes one or more processors configured, in accordance with instructions on a non-transitory computer readable medium, to: process inputs comprising a list of one or more interventions and data from a plurality of sources, wherein the data includes structured data and unstructured data, and wherein the processing includes clustering, anomaly detection, natural language processing, causal feature analysis, and sensitivity feature analysis; determine, responsive to the processing, a presence in the data of a correlation between an intervention from the list of one or more interventions and an event; and output a list of one or more intervention-event combinations, each combination comprising an intervention from the list of one or more interventions and a correlated event.

2. The system of claim 1, wherein the processors are configured to identify correlations to adverse and beneficial events.

3. The system of claim 1, wherein the interventions comprise at least one of health-related regulated products and medical treatments.

4. The system of claim 1, wherein the data includes at least one of claims data, disease registries, electronic health / medical records, individual case safety reports, labeled signals data sets, and scientific literature.

5. The system of claim 1, wherein the causal feature analysis includes one or more of: time to onset, challenge / de-challenge / re-challenge, event frequency, disproportionality, frequency trend.

106. The system of claim 1, wherein the sensitivity feature analysis includes one or more of: important medical event, designated medical event, event seriousness factors.

7. The system of claim 1, wherein the processors are further configured to: generate a score for each intervention-event combination based on relative strength of evidence for the correlation between the respective intervention and event.

8. A method of identifying correlations between interventions and events, comprising: processing inputs comprising a list of one or more interventions and data from a plurality of sources, wherein the data includes structured data and unstructured data, and wherein the processing includes clustering, anomaly detection, natural language processing, causal feature analysis, and sensitivity feature analysis; determining, responsive to the processing, a presence in the data of a correlation between an intervention from the list of one or more interventions and an event; and outputting a list of one or more intervention-event combinations, each combination comprising an intervention from the list of one or more interventions and a correlated event.

9. The method of claim 8, wherein the identifying comprises identifying correlations to adverse and beneficial events.

10. The method of claim 8, wherein the interventions comprise at least one of health-related regulated products and medical treatments.

11. The method of claim 8, wherein the data includes at least one of claims data, disease registries, electronic health / medical records, individual case safety reports, labeled signals data sets, and scientific literature.

12. The method of claim 8, wherein the causal feature analysis includes one or more of: time to onset, challenge / de-challenge / re-challenge, event frequency, disproportionality, frequency trend.

13. The method of claim 8, wherein the sensitivity feature analysis includes one or more of: important medical event, designated medical event, event seriousness factors.

14. The method of claim 8, wherein the method further comprises: generating a score for each intervention-event combination based on relative strength of evidence for the correlation between the respective intervention and event.12