Drug combination strategy management decision support method and system based on artificial intelligence

Abstract

The invention relates to a risk management and decision support method, system and platform for the pharmaceutical industry. The method comprises strategic investment portfolio management, supervision affair, clinical drug development, pharmaceutical economics, investment strategy optimization, risk management, merger and purchase full-duty investigation and stock markets. The system uses artificial intelligence and diverse data from public and private sources, including clinical trials, regulatory decisions, economic data, pharmacological data, and company data. The system integrates a plurality of decision modules for clinical development, such as clinical risk, supervision risk, pharmacological risk and economic risk. This risk factor network generates predictive and normative information for strategic decisions.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This patent application claims priority to U.S. Provisional Patent Application No. 63 / 366,875, filed with the U.S. Patent and Trademark Office on June 23, 2022, entitled “Artificial Intelligence-Based Drug Combination Strategic Management Decision Support Method and System,” the contents of which are incorporated herein by reference. Field of the Invention

[0003] The present invention generally relates to a drug combination strategic management decision support method and system based on artificial intelligence. More specifically, the present invention relates to a platform and portal using the drug combination strategic management decision support method and system based on artificial intelligence. Background of the Invention

[0005] It is well known that the drug development process is a long, expensive, complex and extremely risky endeavor. Upon successful drug development, pharmaceutical companies and life science venture capital funds can expect to reap significant revenues. However, this process faces some significant challenges. Indeed, it takes pharmaceutical companies more than a decade and typically more than $2 billion to bring a new therapeutic product from the laboratory to the market. Furthermore, the cost of developing new drugs, as well as total R&D expenditures, have increased, while the rate at which new products are brought to market has remained roughly constant, leading to what some authors call a pharmaceutical R&D productivity crisis (Pammolli et al., 2011). Given the ongoing productivity crisis in the pharmaceutical industry, we have to question the sustainability of the biopharmaceutical industry’s business.

[0006] Therefore, the global innovative pharmaceutical industry, with drug innovation as its core business, is facing major challenges and is adopting multiple strategies to reduce development risks and improve the productivity of its product pipeline. One of the major milestones in a drug development project, if not the most important, is obtaining regulatory approval for marketing authorization from regulatory agencies (e.g., U.S. Food and Drug Administration, Health Canada).

[0007] As we all know, successful advancement of the clinical trial phase is necessary to ultimately obtain regulatory approval, so success in the clinical trial phase becomes a crucial step. Phase I studies are designed to determine the safety and tolerability of a candidate drug in humans. It is traditionally conducted in healthy participants, but can be conducted in patients when it is ethically not permitted to conduct it in healthy volunteers. Phase II studies plan to test the efficacy and safety of the candidate drug in a larger group of patients (with the disease or condition it is intended to treat). Phase III studies, also known as pivotal studies (the most expensive clinical phase), must confirm the efficacy and safety of the candidate drug in a large group of patients with the disease. It is estimated that approximately one in ten potential drugs that begin Phase I clinical trials will enter the market (Hay et al., 2014). In fact, 60% to 70% of Phase II studies will not enter Phase III, and 30% to 40% of Phase III studies will not enter the market. In addition, only about 20% of new molecular entities (NMEs) can cover their average capitalized R&D expenses, and not all marketed drugs can generate revenues that are equivalent to or exceed R&D costs (Vernon et al., 2010). This means that obtaining regulatory approval does not guarantee a successful drug launch. These statistics highlight the significant challenges facing pharmaceutical companies, contract research organizations (CROs), and life science venture capital (LSVCs).

[0008] Risk is inherent in this industry, whose fundamental mission is pharmaceutical innovation. However, the nature of risk in this industry is multifaceted. One important aspect concerns clinical development, as failures in late development are the most costly. For each new drug launch, clinical development (Phase I-III) accounts for approximately 63% of the costs, while preclinical drug discovery accounts for only 32%. In fact, if pharmacological efficacy and safety remain well-known failure factors, the literature is full of numerous important failure risk factors of a strategic, operational and commercial nature. In addition to being difficult for humans to control, decisions related to these risk factors are also the main cause of failure in clinical development (Phase I, II, III).

[0009] Addressing this issue requires the development and implementation of effective portfolio management strategies in the pharmaceutical industry. While portfolio risk management strategies are highly developed in fields such as finance, the pharmaceutical industry has been slow to adopt effective methods to effectively manage its product pipeline. Without intelligent a priori and real-time risk monitoring throughout the drug development process, it is impossible to deploy effective and efficient portfolio management strategies in the industry. Artificial Intelligence (AI) offers us the opportunity to revolutionize these critical steps in drug development by intelligently controlling a large number of avoidable strategic and operational risk factors.

[0010] Pharmaceutical Pipeline Productivity

[0011] Pharmaceutical pipeline productivity can be defined as the ratio of the value (therapeutic and commercial) created by a new drug to the investment required to produce it (Paul et al., 2010a). Studies point to the existence of a productivity crisis in the pharmaceutical pipeline. In fact, data show that the number of new molecules approved has decreased significantly over the past decades, while total R&D expenditures have increased significantly (Pammolli et al., 2011). The repeated observation of this phenomenon in the pharmaceutical industry has even led many authors to express concerns about the sustainability of the business model of the biopharmaceutical industry (Paul et al., 2010b; Scannell et al., 2012; Schlander et al., 2021). Therefore, over the years, the pharmaceutical industry has had to find new ways to optimize strategic decisions in drug portfolio / pipeline management. In fact, like any other organization, the pharmaceutical industry is constantly seeking efficiency in its decision-making processes to optimize the use of resources and increase the return on investment. The productivity crisis defined here, then, is a major challenge for pharmaceutical portfolio managers.

[0012] Drug development process

[0013] In essence, the drug development process can be divided into two major periods in terms of technology and time, namely drug discovery (preclinical research) and drug development (human clinical research).

[0014] Drug discovery is the initial selection of the most promising molecules as candidate drugs that will enter Phase I (first-in-human studies). Drug development corresponds to the clinical development period, which mainly includes interventional clinical studies on humans. The clinical development period includes Phase I, II, and III studies.

[0015] Basically, Phase I trials are often called first-in-human studies and specifically involve evaluating the safety of a candidate drug in a dose-dependent manner. Testing is usually done on a small group of healthy volunteers. However, when it is not ethical to do Phase I studies on healthy volunteers, Phase I studies can also be done on patients.

[0016] Basically, Phase II studies are used to evaluate the short-term efficacy and toxicity of a candidate drug. These studies are usually used to establish proof of concept (proof-of-concept studies). These studies are conducted on hundreds of patients with the relevant treatment condition. The scientific robustness (elimination of bias) of these studies usually requires that the design of these studies is usually single-blind or double-blind with randomized controls. One group receives the experimental drug while the second group (called the control group) receives the control drug or a placebo (if ethically justified).

[0017] A blinded study means that the patients do not know what treatment they are receiving. In a double-blind study, neither the patients nor the clinical researchers know who is receiving the experimental treatment. There are even triple-blind studies, where in addition to the patients and clinical researchers, the statisticians who analyze the data are blinded to the randomization.

[0018] Phase III or pivotal studies evaluate the efficacy and safety of a candidate drug in patients with the disease of interest. They are typically randomized controlled trials to ensure the robustness of the data collected. This is a critical step prior to submitting a marketing application to a marketing regulatory agency, such as Health Canada.

[0019] While it is estimated that it costs approximately $2 billion to develop a drug, i.e., to carry out the discovery and development phases defined above, it is important to remember that the late clinical development phase is the most expensive for the pharmaceutical industry. Despite many technological advances that have increased the success rate of the discovery phase, the attrition rate in clinical development remains high. However, compared to early discovery, the cost of late-stage failures is the most expensive. In fact, for every new drug launched, clinical development (Phase I-III) accounts for approximately 63% of the costs, while preclinical drug discovery accounts for only 32%.

[0020] Phase transitions in clinical development are defined as the progression from one phase to the next (e.g., from Phase II to Phase III). By collecting historical data, we can statistically observe that the probability of progression from one phase to the next is variable and depends on many factors, such as therapeutic area, type of molecule, etc.

[0021] According to the U.S. Food and Drug Administration, 30% of compounds fail in Phase I, 67% fail in Phase II, and 67% to 75% of compounds never make it to the market despite reaching the end of clinical development. Other published studies report a success rate of 51% in the discovery phase, 69% in preclinical development, and 12.8% in clinical development (Phase I, II, III), which translates to an overall probability of technical and regulatory success of 4.1% (Paul et al., 2010). According to the Pharmaceutical Research and Manufacturers of America (PhRMA), the industry-wide success rate would be as low as 12%. In other words, this means that only one in ten molecules that enter clinical development is ultimately approved.

[0022] Pharmaceutical portfolio managers and life science investment fund analysts use these historical statistics to make forecast estimates for development programs. These methods have many limitations. On the one hand, they do not consider numerous risk factors, and on the other hand, they tend to generalize, while empirical observations show that many factors, such as therapeutic area, study location, company size (pharmaceutical vs. biotech), company clinical development experience in the relevant therapeutic area, etc., affect the likelihood of transformation and regulatory success.

[0023] Reasons for clinical development failure

[0024] The reasons for clinical trial failure are often automatically categorized as the inability to demonstrate efficacy or ensure the safety of the drug candidate. Efficacy and safety issues are most often cited to quickly justify the reasons for failure. Efficacy here is understood as the demonstration of the expected pharmacological effect (mechanism of action) of the studied molecule. As for safety, it is understood as an adverse pharmacological reaction. Therefore, failures in clinical development are often categorized in a binary way as reasons for efficacy or safety.

[0025] While categorizing the reasons for clinical trial failure in this way is not inaccurate, it may be simplistic. In fact, when it comes to efficacy, it is often a phenotypic conclusion due to many other potential reasons that are unrelated to the mechanism of action. Many other factors related to operational, strategic, and even economic decisions during the clinical development process can lead to development failure (Fogel, 2018; Harrison, 2016). Factors such as participant recruitment and enrollment rates, high dropout rates, study site selection, protocol compliance, insufficient funding, and study protocol complexity are all potentially important factors that lead to clinical study failure.

[0026] Patient recruitment

[0027] There is ample evidence that low rates of participant retention (dropouts) or suboptimal rates of participant recruitment can significantly affect the statistical power of a clinical trial, hampering any likelihood of demonstrating efficacy and leading to clinical trial failure. Specifically, studies estimate that when a trial experiences multiple dropouts, it may lose statistical power and fail to demonstrate efficacy. (Fogel, 2018; Hwang et al., 2016; Schroen et al., 2010). For example, a complex Phase II clinical development study protocol conducted at a clinical research site where the principal investigator recruitment rate is high but there are few available resources and the site does not have an adequate pool of eligible patients to ensure high recruitment rates to maintain the necessary statistical power will result in a significantly increased risk of study failure. This can have a significant financial impact on the organization sponsoring the trial.

[0028] Currently, it is estimated that every day a clinical trial is delayed costs between $600,000 and $8 million in lost revenue (CenterWatch, n.d.). To highlight the complexity of clinical development risk, it has even been shown that enthusiasm among the principal investigators at the study sites was the most important factor in successful enrollment of the 60 sites in a trial evaluating a local postoperative pain treatment (Fogel, 2018; Fouad et al., 2013). Furthermore, data suggest that slow recruitment can stem from inadequate staffing and poor prioritization of clinical trials over day-to-day operations (Thoma et al., 2010).

[0029] Adherence to treatment

[0030] Patient nonadherence to treatment is one of the most complex issues facing clinical development. Minor nonadherence to treatment can have a significant impact on the sample size required to detect differences between groups. For example, the literature shows that if compliance drops by 20% to 30%, the sample size may need to be increased by more than 50% to maintain equivalent power. (Serebruany et al., 2005; Smith, 2012). A nonadherence rate of 40% requires a threefold increase in sample size (Serebruany et al., 2005). The direct consequence of this situation is an increase in the time and cost of the clinical development project for the product under investigation.

[0031] Complexity of study design and choice of study site

[0032] The choice of inclusion and exclusion criteria affects the duration and cost of a clinical trial (Babbs, 2014). The complexity of a study protocol affects the likelihood that a clinical trial will achieve the desired level of recruitment and retention of participants, thereby meeting the statistical target for efficacy endpoints (power β). For example, one study noted that over 40% of 3,400 clinical trials underwent protocol amendments prior to the first visit of the first participant, resulting in a 4-month delay in the trial (Getz et al., 2011). Although some protocol amendments are unavoidable, the risk of protocol amendments prior to trial initiation can be reduced by better planning and anticipating the consequences of clinical trial design choices. Furthermore, scientifically and operationally complex protocols have been associated with lower recruitment and retention of clinical trial participants (Lamberti et al., 2012).

[0033] One study reported that 16% of protocol modifications were due to changes in inclusion and / or exclusion criteria, which could result in differences in the patient populations before and after the modification. In addition, the same authors reported a correlation between experience gained from positive results in previous clinical trials and the subsequent clinical trial results obtained. In this case, the data showed that sites that had conducted 6 to 10 clinical trials were more likely to meet enrollment criteria within the specified time frame than sites with fewer previous trials. Even considering the most well-known problem in clinical trials, namely, low enrollment rates in clinical trials, there is evidence that research sites with a history of successful performance are historically more likely to meet enrollment goals (Getz et al., 2011). Developing an approach that focuses solely on recruitment performance while ignoring that the enthusiasm of the investigators (especially the principal investigator) can be an important risk factor is not a successful strategy.

[0034] Regulatory agencies and drug evaluation

[0035] In addition to increasing development costs and high attrition rates in clinical trials, the pharmaceutical industry environment is characterized by increasing requirements for drug marketing from regulatory agencies (Canada: Health Canada; United States: U.S. Food and Drug Administration (FDA); Europe: European Medicines Agency (EMEA), etc.). In addition, it is increasingly challenging to obtain reimbursement for products from public reimbursement agencies, which is an asset for marketing drugs in Canada, Europe and certain Asian countries, where there are increasing requirements for demonstrating therapeutic and economic value. It is important to remember that the productivity of a pharmaceutical pipeline is defined as the ratio between the value created by a new drug (therapeutic value and commercial value) and the investment required to produce this drug.

[0036] Indeed, some regions require more investment to bring a drug to market, both in terms of time and money. In Canada in particular, the drug evaluation and commercialization process is considered arduous: it is “fragmented and sequential.” (Hoskyn, n.d.) Since the process of approving a drug for commercialization is separate from the reimbursement process, jurisdictions that provide marketing authorization evaluate drugs from a different perspective than reimbursement review agencies. Generally, in order to receive a compliance notice and be able to market the drug, the drug is evaluated individually based on the clinical trials conducted (efficacy, safety, and quality). However, during reimbursement evaluation, in addition to determining its therapeutic value, the drug is compared to all available options in the jurisdiction to determine its economic and social value. The drug comparators retained by agencies in the comparative analysis may vary depending on clinical practice, and the final price recommendations of health technology assessment agencies may also vary, which may disappoint manufacturers because the results of the evaluation may be unexpected. Therefore, if the public market potential of a particular molecule is important, the manufacturer must decide whether it is willing to negotiate a lower public price to meet the expectations of the drug regulator or to retain the private market only at a higher price. For some drugs, the private market does not pose a threat to commercial success where the return on investment exceeds the development costs. However, this does not apply to all drugs. For example, for a drug intended to treat a disease in patients aged 60 and over, the private market would be a very large commercial failure because patients aged 65 and over are typically enrolled in public programs, which indicates that the target population of the drug will not be able to access the drug, and therefore the sales originally expected by the pharmaceutical company will not be achieved. In Canada, for a new patented drug, the review process is divided into 5 general steps, starting with regulatory approval by Health Canada based on clinical trials for the drug's efficacy, safety, and quality. The patented drug then undergoes a price regulatory review by the Patented Medicines Prices Review Board (PMPRB) to ensure that the patented drug is not sold in Canada at an excessively high price. After the drug is evaluated, INESSS (National Institute for Excellence in Health and Social Services) in Quebec or CADTH (Canadian Agency for Drugs and Technologies in Health) in other parts of Canada will analyze the drug's therapeutic value and value to the healthcare system / social value. These evaluation bodies will make three possible recommendations: unconditional reimbursement, conditional reimbursement, or no reimbursement. After a positive recommendation from the Health Technology Assessment Agency, if the provinces are interested, they will negotiate with the pCPA (Pan-Canadian Pharmaceutical Alliance) on price and clinical conditions. Canadian provinces that wish to participate in the negotiations will participate in the discussions. The goal is to achieve fair use of drugs and uniform prices across the country, among other things. Successful negotiations will lead to the drug being introduced into the provincial public drug list. However, if the negotiations are not completed, the drug will not be reimbursed in the public plan and can only be purchased in the private market. After discussing the clinical conditions and prices of the pCPA process, the last step is to sign a confidentiality agreement with the participating provinces.The provinces will continue to implement public reimbursement at their own pace. The entire approval and reimbursement process will likely take more than three years. Each Canadian province is independent in its budget and in the peculiarities of its own health care system. Further complicating matters, when the pCPA negotiations are completed, these clinical conditions may become more stringent during the period of confidential individual agreements with the provinces.

[0037] Canada is by no means the only country with a public health system that assesses public reimbursement, but it is the country with the longest process and the most uneven decision-making. According to a publication by the Innovative Medicines Association of Canada that compares Canada to other international jurisdictions, Canada has a new molecule introduction rate of 65%, compared to 96% for 20 Organisation for Economic Co-operation and Development (OECD) countries that also have public reimbursement systems (Hoskyn, n.d.). Given that Canada is made up of 10 provinces and three territories, each with a different budget and healthcare system, the national reimbursement rate for the same product drops to 53%, highlighting the huge differences in provincial reimbursement decisions. Even if the drug has been approved elsewhere in the world, Canada ranks second last among the 20 OECD countries in terms of the length of the reimbursement process, with a review time of 559 days in Canada, compared to 138 days in Italy, 13 days in Germany, and 32 days in the UK (Hoskyn, n.d.).

[0038] The more innovative and unique a drug is, the more challenging it is to meet CADTH, INESSS, PMPRB and pCPA requirements. It is not enough to successfully complete clinical trials and obtain marketing authorization. In order to meet the recommendations of CADTH and INESSS, these strategic study design decisions need to be considered in advance. In order to successfully achieve commercialization in Canada, it is urgent to consider clinical trial design in Phase II.

[0039] These pharmacoeconomic evaluations are subject to submission requirements and evaluation criteria specific to each jurisdiction. Product submission guidelines and requirements continue to evolve in the face of continued innovation in drug administration methods (e.g., subcutaneous, intravenous, oral, etc.), patient types / subtypes (e.g., gene-specific), disease progression rates, companion testing, etc. Since 2020, CADTH, the Canadian (excluding Quebec) drug value assessment agency, has issued more than three notices of procedural changes, submission requirements, or review process fee increases. At the same time, since 2019, procedural reforms of the PMPRB (the agency that assesses the non-excessive prices of patented drugs in Canada) have not been finalized and implemented, which has greatly affected product launch decisions in Canada. This shows that in countries with public health systems where drugs are reimbursed, obtaining a drug compliance notice is far from enough to define commercial success, as there are many external factors that can change the situation.

[0040] Financial factors

[0041] In addition to the efficacy of the drug, management factors can also have a large impact, such as poor financial assessment of the capital requirements required to ensure that clinical development projects are properly carried out. Financial factors are critical and must be assessed throughout the development process taking into account all contributing factors. In fact, studies have shown that 22% of failed Phase III studies fail due to lack of funding (Hwang et al., 2016). The costs required to complete the entire development process of a drug from discovery to market vary, and estimates of these costs also vary. In fact, considering Phase III trials alone, the Pharmaceutical Research and Manufacturers of America (PhMRA) estimated that the cost per patient in 2013 was $42,000, with 1,680 Phase III clinical trials costing $10 billion and involving more than 600,000 patients.

[0042] Increase public spending on medicines

[0043] The fact that pharmaceuticals are increasingly important in public expenditures in healthcare systems around the world reinforces the growing interest of public authorities in controlling expenditures. In fact, pharmaceutical costs are now the single most important input in healthcare systems in almost all industrialized countries. The growth of public expenditures on pharmaceuticals in OECD countries clearly illustrates this importance. In Canada, in 2017, public expenditures on pharmaceuticals accounted for 16.4% of total healthcare expenditures. The same trend can be seen in the most industrialized countries (e.g. the United States, Austria, Switzerland, Greece, etc.). As drug prices continue to rise, controlling pharmaceutical expenditures is becoming a top priority for managers and political leaders of health systems in industrialized countries. In this context, the willingness of the Canadian government to impose stricter controls through its semi-judicial body, the PMPRB (Patented Medicine Prices Review Board), is a strong indication of the changing dynamics of the environment. The goal of the PMPRB is to set a maximum price for patented drugs in Canada. The Pan-Canadian Pharmaceutical Alliance (pCPA) will further reduce the PMPRB preset prices in provincial negotiations. The growing interest of government agencies in the issue of controlling pharmaceutical expenditures complicates the business environment for the pharmaceutical industry in markets such as Canada, which is sensitive to reimbursement issues as sales are heavily dependent on them.

[0044] Not surprisingly, the competitive landscape of the pharmaceutical industry is constantly changing. Fundamentally, the business model of the innovative pharmaceutical industry is based on innovation, which is the result of a lengthy research and development (R&D) process that leads to the development of new drugs. This business model has evolved into one that favors mergers and acquisitions over huge investments in R&D. However, it remains difficult for companies to develop new drugs in the face of low pipeline productivity and financial challenges posed by patent cliffs. Therefore, innovation remains the cornerstone of the industry's business model.

[0045] The consequences of this situation can be listed at the economic, social and even environmental levels. At the economic level, huge R&D investments combined with extremely high failure rates of molecules in development (sometimes at late stages) are driving up the prices of drugs entering the market. The price increase of patented drugs has been the subject of a continuous debate in recent years. It must be noted that the business model of innovative pharmaceutical companies will not survive if the huge losses caused by the failure of molecules in R&D are not compensated by the profits brought by the approval of molecules after marketing. From a social point of view, three consequences can be directly foreseen in this situation. First, patients in need will not be able to access high-quality drugs because of the huge costs that public payers cannot afford. Secondly, the “unnecessary” exposure of participants in failed clinical studies cannot be ignored and is fundamentally ethically problematic (Williams et al., 2015). Finally, this may lead to job losses for industry professionals. From an environmental point of view, clinical research requires a huge expenditure of resources and materials.

[0046] As the pipeline productivity crisis has been caused by the increasing cost of new drug development and high failure rates, the pharmaceutical industry has been told that the business model it relies on cannot be sustained in the long term given such statistics. Indeed, the pharmaceutical industry’s main mission is innovation, so risk is inherent. But as mentioned above, pharmaceutical innovation has specific characteristics that make it an extremely risky business. One might have thought that with such characteristics, the pharmaceutical industry would be equipped with strong strategic drug portfolio management tools to overcome these problems that have plagued the industry for decades. The pharmaceutical industry has been very slow in developing and implementing such tools (Kwak & Dixon, 2008). Therefore, it is crucial for the industry to continuously develop new strategies to optimize risks. An analysis by consulting firm McKinsey pointed out that the pharmaceutical industry urgently needs to draw inspiration from other business fields such as banking and finance and develop corresponding methods. Moreover, the analysis predicts that if new risk control methods are not implemented, the industry will face more difficulties in the coming years (Dhankhar et al., 2018). The problem of R&D efficiency in the pharmaceutical industry, characterized by its meager output, has been widely discussed. In fact, some authors define R&D efficiency as the ratio of successful approval and marketing of new drugs to the financial investment required for R&D (Schuhmacher et al., 2016). Over the past few decades, this efficiency has declined sharply. Considering the almost explosive growth of R&D investments in recent years, it is reasonable to ask what are the effective ways to improve R&D efficiency in this industry.

[0047] Conceptually, risk is a complex concept because the adaptability varies depending on the field of activity. Therefore, risk is understood differently whether it is in health, finance, technology, environment, cybersecurity, defense, etc. In medicine, we talk about medical risks; in finance, we talk about financial risks; in the environment, we talk about environmental risks in an oil pipeline project, for example, in manufacturing, we focus on the risk of eliminating defects in a production line, and so on.

[0048] According to Cornelius Keating, risk is an undesirable subset of a set of uncertain outcomes. In fact, research shows that there is no universal definition of risk and it depends on the industry. However, the concept of uncertainty seems to be inherent to the concept of risk. In fact, the International Organization for Standardization standard ISO / ISO31000 (2009) / ISOGuide73:2002 defines risk as "the effect of uncertainty on objectives." Uncertainty is inherently linked to the concept of probability. Fundamentally, risk management is a systematic process of identifying, analyzing, and responding to project risks. This involves maximizing the probability and consequences of positive events related to project objectives and minimizing the probability and consequences of undesirable events related to project objectives ( Guide, n.d.

[0049] Risk management is an integral part of management activities in almost all important industrial sectors, be it aerospace, energy, finance, insurance, etc. However, as the analysts of McKinsey & Company point out, risk management is increasingly becoming an issue for all industries, and according to them, even more so for the pharmaceutical industry, which will be exposed to risks, especially in the design and execution of clinical trials, drug approvals, product quality, and global business practices, which are growing in frequency and magnitude (Dhankhar et al., 2018). Therefore, the pharmaceutical industry must address this issue more deeply and develop better risk management methods that are suitable for the industry, as the industry has many peculiarities that require a risk management approach and effective tools that adapt to these peculiarities, which can be disadvantages of adopting standard approaches.

[0050] Therefore, over the years, the pharmaceutical industry has been in urgent need of finding new ways to optimize strategic decision-making in pharmaceutical portfolio management. In fact, like any other organization, the pharmaceutical industry is constantly seeking efficiency and effectiveness in its decision-making processes to optimize the use of its resources. The productivity crisis defined here represents a major challenge facing pharmaceutical portfolio managers.

[0051] The risky nature of the pharmaceutical industry can be understood as a result of scientific, regulatory, and economic uncertainty (Dickson & Gagnon, 2004). Further analysis leads us to understand that many factors can explain the high failure rate of drug development.

[0052] Traditional risk control and portfolio management approaches to drug development

[0053] Pharmaceutical portfolio managers, clinical development project managers, life science investment fund managers, contract research and clinical development organizations (CROs) use statistical methods to perform risk estimation. In particular, existing historical clinical trial data allow for the statistical calculation of phase transition probabilities. These statistics are combined with the opinions of industry experts who, with their years of accumulated knowledge, can make recommendations on the conduct of individual clinical development projects.

[0054] The pharmaceutical industry environment is characterized by its dynamic, rapidly changing regulatory and commercial environment, long research and development (R&D) lead times and the complexity of the development process. Statistics are therefore limited in their ability to account for these many factors that change over time. Statistics provide historical estimates but are not guaranteed to be generalizable over the projected timeframe.

[0055] In light of these challenges that threaten pharmaceutical innovation, pharmaceutical companies have developed several qualitative and quantitative approaches to risk mitigation strategies during drug development.

[0056] According to Cooper et al., pharmaceutical portfolio management can be defined as a dynamic decision-making process that involves the continuous updating and revision of a company's list of active new products and ongoing R&D projects. This dynamic decision-making process involves the evaluation, selection, and prioritization of new projects. Ongoing projects may be accelerated, terminated, or deprioritized; therefore, resources will be allocated or reallocated to active projects (Cooper et al., 1997).

[0057] The main goals of implementing a portfolio management strategy in the pharmaceutical industry are to improve R&D efficiency and reduce risk. R&D efficiency can be defined as the ratio of the value (therapeutic value and commercial value) created by a new drug to the investment required to produce the drug. To achieve these goals, the pharmaceutical industry has developed qualitative and quantitative drug development risk management methods.

[0058] Traditionally, pharmaceutical portfolio management has relied on the use of historical estimates of regulatory approval rates and human judgment to support drug development decisions (Betz, 2011; Krishnan & Ulrich, 2001). However, reliance on human judgment and historical estimates leads to significant limitations and inaccurate results. These estimates are inherently dependent on many factors, including the therapeutic class and development period, as well as the experiential knowledge of key opinion leaders (KOLs). Among these qualitative and quantitative traditional decision-making tools, we can cite:

[0059] Financial model: (e.g., net present value, internal rate of return, etc.)

[0060] Certain criteria and tools are taken from financial mathematics that allow pharmaceutical portfolio managers to assess the feasibility of investment development programs. These include: An estimate of unmet medical need. This is usually obtained through market research. It can estimate the size of the market and predict potential market share. Generally, the annual revenue of the product is estimated using current sales of drugs used to treat similar indications.

[0061] a. Commercial value.

[0062] b. Net present value (NPV). The value of future cash flows discounted to today’s cash. NPV = x / (1+k)n;

[0063] c. The net cash flow (x) is discounted annually at a discount rate (k) and paid over n years.

[0064] d. Probability of successful listing (P);

[0065] e. Development cost (C);

[0066] f. Risk-adjusted cost rC = C x P;

[0067] g. Productivity index PI = NPV / C;

[0068] h. Risk-adjusted productivity index = rNPV / rC;

[0069] i. Risk-adjusted net present value. It can be roughly expressed as: rNPV = NPV x P.

[0070] The net present value of a project when revenue, risk, cost and time are all taken into account; allows comparison of heterogeneous candidate portfolios. It is more accurate than NPV in the pharmaceutical industry where the risk component is particularly dependent on many factors. Therefore, NPV is adjusted (multiplied) by the probability of launch success to take this into account.

[0071] Specifically, it is the difference between the NPV of the projected benefits adjusted for the current probability of success P and the sum of the NPV of the costs adjusted for the relative risk after i periods.

[0072]

[0073] Probabilistic financial models: (e.g., Monte Carlo simulations and decision trees).

[0074] Strategic approach: The selection of projects is primarily driven by business strategy.

[0075] Scoring Models: Rating and scoring items based on various qualitative criteria.

[0076] Mapping method: (e.g., Boston Consulting Group portfolio model).

[0077] However, despite the implementation of these approaches in drug discovery and development over the years, the problem of high attrition rates persists. Some authors have identified the main factors to consider in building an optimal pipeline portfolio, including development costs, likelihood of survival, expected profitability, competitive landscape, market size, and novelty of the drug (Ding & Eliashberg, 2002; Jekunen, 2014).

[0078] Indeed, while it is well known that the drug development process is time-consuming, expensive, complex, and risky, the nature of risk in the pharmaceutical industry is multifaceted. Although pharmacological efficacy and safety are well-known failure factors in drug development, the literature is replete with many other important risk factors for failure that are strategic, operational, and commercial in nature. Decisions related to these risk factors are the main cause of late-stage failures and are difficult to control by humans (Sorintellis, 2022). An important limitation of these approaches is the accuracy of the probability estimates of successfully launching a development program, regardless of its stage of progression, by considering all potential risk factors and providing the possibility of continuous reassessment (given the time component that is closely related to clinical development).

[0079] With the advent of AI, several approaches have emerged in drug development, such as facilitating the discovery and design of new molecules (Spencer et al., 2015; Stokes et al., 2020), improving molecular structural features (Dahl et al., 2014), repositioning old molecules (Ashburn & Thor, 2004; Klaeger et al., 2017), and improving pharmacovigilance (Kompa et al., 2022).

[0080] Artificial Intelligence (AI) and Clinical Transition Prediction

[0081] The emergence of artificial intelligence, especially machine learning techniques, can overcome the limitations associated with statistical estimation methods. Machine learning can be defined as the ability to learn without being explicitly programmed, and it requires a combination of three essential factors: data, computing power, and algorithms / models. The vast amount of data sources accumulated over the years on clinical research projects is an indispensable asset for the development of machine learning algorithms.

[0082] Some publications show the possibility of predicting failures, but applied to a subspace of potential risks. It is uncertain whether the approach proposed in this literature is applicable to the business scenario considered, especially for the development of Canadian clinical trials requiring additional approvals.

[0083] In the literature, the definition of clinical trial success varies from article to article and does not take into account all operational constraints such as trust, decision making, etc. Current definitions do not take into account other aspects such as simplified approaches taken for the sake of competition / publication, with a single target variable (mono-objective), whereas modeling all response variables has added value.

[0084] In this context, commercial success in healthcare systems such as the US will, by definition, be associated with marketing approval (MA), i.e., obtaining regulatory approval for marketing from the US Food and Drug Administration (US FDA). In Canada, where the healthcare system has a significant public reimbursement component (INESSS, CADTH), commercial success is defined not only as obtaining marketing approval (Health Canada) but also as having the drug added to the public insurance reimbursement list. Therefore, in this context, if, in addition to predicting the transition from clinical phase to regulatory approval, we can also predict the success rate of inclusion in the reimbursement list, this will create more value for the industry. Currently, publications that are able to predict failure focus on specific scenarios and are uncertain whether they are applicable to Canadian or European settings. In addition, publications are limited to the US region (FDA), and here we consider extending the prediction of regulatory approval to other legal regions such as the EU (EMA: European Medicines Agency), Canada (Health Canada). In addition, the reasons for failure are diverse and not necessarily identified and annotated to clinical trials in public data sources. Therefore, predicting failure may require combining subtle interactions between multiple variables, which is not realistic for humans. Functional tools must be able to be used for multiple types of clinical trials, and acquiring specific knowledge for each type (e.g., identifying risk patterns) is not feasible at scale. Therefore, a reproducible machine learning strategy is needed to incorporate multiple clinical trial types.

[0085] However, these methods only consider clinical study data and therefore focus specifically on clinical risk. On the other hand, if they allow to provide a score / probability of success for a clinical study, they do not provide the ability to dynamically follow up during the clinical study in a more practical way. This fact is crucial in the daily practice of the pharmaceutical industry, since clinical studies are conducted prospectively over a long period of time, and the tracking of events that affect the probability of success score is essential for better risk control.

[0086] Furthermore, there is currently no method that can predict reimbursement approval of a drug in a publicly funded healthcare system. All current prediction methods are limited to the clinical transition and regulatory (FDA) approval phases. SUMMARY OF THE INVENTION

[0087] The above and other purposes of the present invention are generally achieved through systems, methods and platforms applicable to specific areas such as strategic drug portfolio management, regulatory affairs, strategic management of business processes in the clinical drug development stage, pharmacoeconomics, investment strategy optimization of life science venture capital funds, risk management, due diligence of drug mergers and acquisitions, and stock markets. The platform operates in a heterogeneous data environment from multiple open and private sources. The collection, organization, enrichment, processing and analysis of millions of data points through artificial intelligence algorithms (machine learning) can generate predictive and prescriptive actionable insights to better control risk factors a priori and continuously, thereby effectively optimizing decisions in strategic drug portfolio management and the implementation of clinical trials and all decision-making recommendations in the above other areas.

[0088] In one aspect of the present invention, a system for predicting the degree of success of a clinical trial of a drug product is provided. The system includes a data source and a server, the data source containing data related to the clinical trial. The server includes a data acquisition module that communicates data with multiple external data sources, the multiple external data sources containing data related to the clinical trial, regulatory approval, economic and reimbursement information, pharmacological information, and commercial and company information. The server includes a data processing engine that is configured to transform and normalize data from the data acquisition module using a natural language processor. The server includes a machine learning engine that includes a model trained using data processed by the data processing engine, and the trained machine learning engine is configured to execute an algorithm to analyze data from a data source related to the clinical trial and calculate a prediction of the success of the clinical trial based on the analyzed data.

[0089] The model trained by the machine learning engine may also be a multi-layer model. The multi-layer model may also include a first layer model configured to calculate multiple intermediate predictions of success, and / or multiple first layer models, each of which is configured to calculate intermediate predictions of success. The multiple first layer models may also include at least one of the following models: a model for calculating a clinical trial target recruitment prediction, a model for calculating a clinical trial protocol deviation prediction, and a model for calculating other factors related to the clinical trial. The successful intermediate predictions of each first layer model may also be input into a second layer model to calculate a prediction of clinical trial success.

[0090] The system may also include a module for interpreting and explaining the calculated prediction of clinical trial success. The module for interpreting and explaining the calculated prediction of clinical trial success may also include generating logical rules for calculating the prediction, and any of a set of contributing attributes of the clinical trial, studies for comparison with the clinical trial, comparative interpretations, and scenarios that affect the degree of expected success of the clinical trial.

[0091] The system may also include an application module configured to execute a machine learning engine using data related to the clinical trial, wherein the acquired data sources may be categorized into a plurality of repositories. The repositories may also include any of the following types of data: clinical data, regulatory data, economic data, molecular data, and MPP.

[0092] In another aspect of the invention, a computer-implemented method for predicting the degree of success of a clinical trial of a pharmaceutical product is provided. The method includes acquiring data from a plurality of external data sources, the external data sources including data related to clinical trials, regulatory approvals, economic and reimbursement information, pharmacological information, and commercial and corporate information. The method also includes processing, converting, and normalizing the acquired data using a natural language processor. The method also includes executing a machine learning model trained using the processed, converted, and normalized data to analyze data related to the clinical trial and calculate a prediction of clinical trial success based on the analyzed data.

[0093] The training model of the method can be a multi-layer model, including a second-layer model and multiple first-layer models, and the method also includes each first-layer model calculating an intermediate prediction of the success of a specific aspect of the clinical study.

[0094] The method can also allow multiple first-tier models to calculate a prediction of target recruitment for a clinical trial, a prediction of protocol deviation for a clinical trial, or one of other factors related to the clinical trial. The second-tier model can use each intermediate prediction calculated by the first-tier model to calculate a prediction of clinical trial success.

[0095] The method may also include real-time monitoring of clinical study progress characteristics, using these characteristics to calculate predictions of clinical trial success. These characteristics may also include predictions of recruitment needs and identification of influencing events.

[0096] The method may also include executing the machine learning model to further calculate any of the following: clinical risk of the clinical trial, regulatory risk of the clinical trial, commercial risk of the clinical trial, and pharmacological risk of the clinical trial. The execution of the machine learning model may also generate normative data for optimizing research behavior.

[0097] The method may also include developing multiple machine learning models for the clinical trial, training the developed models using the acquired data, and selecting one or more of the developed models based on a performance metric.

[0098] In another aspect of the present invention, a computer readable medium storing instructions for executing the above method is provided.

[0099] Application examples:

[0100] Enhanced criteria for estimating a priori the likelihood of successful development (regulatory approval and / or public reimbursement) of an investigational molecule while it is still early in clinical development.

[0101] Our inventions regarding platforms, systems or methods are generally intended to solve the main problems related to the efficiency of drug pipeline production. The platform, system or method is generally intended to optimize strategic decisions in drug portfolio management. In this case, the platform, system or method can be specifically designed to collect, integrate and dynamically and continuously analyze exhaustive risk factors that affect the clinical development and commercial success assessment of drug research. On the other hand, the platform, system or method allows the estimation and classification of decisive risk factors, allowing the optimal management of the drug pipeline of molecules in the clinical development stage (Phase I, II, III). The knowledge thus obtained allows the evaluation and optimization of the commercial success of the studied molecules at an early stage.

[0102] The drug development process can be divided into two major periods in terms of technology and time, namely the discovery phase and the clinical development phase. In view of this, despite many technological advances to improve the probability of success in the discovery phase, the current attrition rate in the clinical development phase remains high; in view of the recognition that, in addition to efficacy and safety issues (pharmacological factors), strategic, operational and even commercial decisions are also important causes of failure in the late stages of clinical development (Phase II, III); in view of the huge unsustainable costs involved in late stage failures compared to the early discovery phase; in view of the current artificial intelligence-based methods that aim to optimize the structure-activity-molecule relationship in the early discovery phase to correct the causes of efficacy and safety failure (pharmacological factors); a system that takes into account equally important risk factors, namely strategic, operational and commercial decisions in late clinical development, is needed to bring useful, necessary and relevant creative contributions.

[0103] The additional creative contribution of a platform, system or method based on artificial intelligence (machine learning, deep learning, NLP, etc.) allows for the dynamic collection, integration and real-time analysis of large amounts of various data, both structured and unstructured, thus going beyond the unchanging and non-evolving statistical estimates of current methods. The usefulness of dynamic and continuous analysis is based on the fact that the pharmaceutical industry environment is constantly changing and it takes many years to develop a drug. Therefore, methods that take time into account and continuously integrate data can increase analytical capabilities tenfold and optimize decision-making compared to statistics that are fixed over time. In addition, the platform, system or method is designed to provide predictive data and prescriptive data. Prescriptive data allows actions to be taken to improve the chances of success, but also allows for the continuous re-evaluation of decisions that have already been made.

[0104] The platform, system or method will enable users to receive predictions based on the features available for each clinical stage, and, moreover, our method will be equipped with counterfactual analysis that generates recommendations on how to parameterize the clinical study to obtain acceptable predictions of success. All these models and counterfactual explanations should be available to users through a user interface. In this way, users will have access to a set of different trained models for different clinical trial stages, as well as explanations of the model decisions. Thus, users can evaluate the influencing factors, adjust them as much as possible, and evaluate the success predictions. Finally, the method and system will generate these insights and deploy them in an easily understandable way to guide strategic decisions in drug portfolio management.

[0105] The system is structured around a module structure of major risks, including clinical development risks, which include operational risks, regulatory risks, economic or financial risks, and pharmacological risks (risks inherent to the molecule under investigation). The output allows the generation of predictive and prescriptive insights from drug clinical development (including clinical study initiation to market access and commercial success) based on a priori estimates of the multidimensional risks associated with the development of drug candidates. For different stakeholders, such as clinical research project managers, drug pipeline managers, clinical research organizations (CROs), life science venture capital analysts, dynamic prescriptive information about ongoing clinical development projects will facilitate strategic, operational or commercial or business decisions that may lead to failure if not properly controlled.

[0106] Ability to use structured or unstructured historical data from existing public and private databases, resulting in millions of data points from multivariate data sets enriched by manual labeling by domain experts. Their collection and integration allows for deep understanding and mapping of drug development risk networks,

[0107] Therefore, the process defined here considers the main pillars as the main basis for capturing the entire process, including pharmacological risk, clinical development risk (including operational risk), regulatory risk, economic or financial risk and strategic risk.

[0108] Overall clinical risk

[0109] The overall risk associated with conducting a clinical development process is defined in the present invention. Clinical risk is estimated by considering all micro-factors of clinical risk that contribute to it over time and intensity. These micro-risk factors that contribute to the estimation of the overall clinical risk are interconnected in a dynamic and evolving sub-network of interdependent dependencies. More precisely, the present invention allows the assessment of clinical risk regardless of the development stage, therapeutic area, study site, study protocol, etc., and by integrating all factors together instead of treating them separately as traditionally practiced. In addition, a process based on machine learning algorithms will allow the efficient prediction of this global clinical risk factor while fully understanding the normative information and taking corrective actions to improve the prediction score, thereby improving the dynamic nature of the process. This includes clinical trial decision elements that are administrative and operational in nature, such as the selection of principal investigators, the geographical location of the trial, site selection, contractual agreements, etc.

[0110] Overall pharmacological risk

[0111] The overall pharmacological risk is defined here as the risk inherent to the molecule. This includes, for example, the known pharmacological properties associated with the molecule under study. Knowledge about the molecule under study is very important in drug development. Paul Ehrlich stressed the importance of molecular activity in understanding pharmacological activity in his famous quote "Corpora non agunt nisi fixata" (Drugs only work when bound). In drug development, quantitative structure-activity relationships (QSAR) are the process of correlating chemical structures with defined effects, such as biological activity or chemical reactivity. It is therefore more useful because it allows explaining and predicting pharmacological properties, such as the activity of a compound on a target receptor, including the efficacy and even safety of a potential drug. It is important to take this risk factor into account because we know, for example, that the probability of success for a chemical molecule is different from that for a biological molecule.

[0112] Global regulatory risks

[0113] Global regulatory risk is defined here as the risk of success (marketing approval) at the regulatory level based on clinical stage data (e.g., approval by Health Canada (Canada), Food and Drug Administration FDA (USA), European Medicines Agency EMA (Europe), etc.). Under the existing healthcare system, regulatory approval is the initial step in determining commercial success, regardless of whether patients obtain drugs through public and / or private programs. The introduction of innovative drugs is slower than it was decades ago. Drugs that enter the market need to be significantly differentiated from competitors, such as through more complex mechanisms of action. Clinical trial requirements must adapt to evolving mechanisms of action and may result in unconventional clinical trials or measures. In the face of changes, the requirements for submitting drugs for approval may also change. The method and system can help discover risk factors for approval failure by compiling historical results, considering changes in requirements in real time, and determining regulatory success rates.

[0114] Global Economic Risks

[0115] This module is applicable to geographic jurisdictions where reimbursement bodies exist and play a key role in the marketing of pharmaceutical products (e.g. Canada, Europe, Asia, Australia, etc.). Typically, in these countries, drugs cannot be distributed under public drug programs without evaluation of the product and negotiation of clinical conditions and / or price. In this context, for countries with public health systems, commercial success depends on obtaining reimbursement at a specific price and for specific clinical conditions, depending on the characteristics of the product's target population.

[0116] Barriers to entry into Canada’s reimbursement process could deter manufacturers from pursuing commercialization of their medicines, limiting access to medicines in Canada versus the rest of the world.

[0117] Stakeholders in Canada's health care system are aware of the large inconsistencies between the drug review process and the various serial stakeholders. Efforts are underway to harmonize the process and reduce duplication of effort. The election brought new promises, such as a national drug plan, rather than one fragmented to the provinces. This means further reforms in regulation, access to medicines, and pricing.

[0118] The system, due to its dynamic nature and capturing changes in real time, will help monitor and alert changes in study design strategies or clinical practices by informing regulatory agencies (Health Canada, FDA, EMA, etc.) about new molecules under review, changes in policies and requirements of reimbursement regulatory agencies, changes in drug prices (including international drug prices), reasons for drug reimbursement / non-reimbursement, drug evaluation timelines, submission criteria, etc. This information helps in making strategic decisions that are critical in determining the commercial success of a drug and improving pharmaceutical productivity.

[0119] Reference now Figure 1 , illustrates a deep understanding and mapping of the drug development risk network. It illustrates a view based on the business needs of three important stakeholders. The stakeholders include contract research organizations (CROs), pharmaceutical companies and biotechs, and life science venture capital.

[0120] From a CRO perspective, the success of a clinical trial at any stage comes down to the effective execution of the clinical trial from the start of the trial to the “last patient last visit” (LPLV) milestone.

[0121] From the perspective of innovative pharmaceutical companies, fundamentally, as the first step to success in any phase of clinical trials, in addition to efficient execution of clinical trials (i.e., from trial initiation to LPLV), positive clinical or statistical results are obtained. The development program is successful when LPLV status and positive results are achieved, while obtaining marketing authorization (regulatory approval) and showing a positive return on investment (commercial success).

[0122] From a life science venture capital (LSVC) perspective, success comes down to achieving a positive ROI, which typically means obtaining regulatory approval and therefore demonstrating clinical trial success as defined from a pharmaceutical perspective.

[0123] See also Figure 2 ,Based on these different perspectives, the present approach includes different states of interest transitions of the ,prediction model. The model includes three (3) stages (I, II and III), approval steps and ,commercial success (CS).

[0124] "Ended" state: The state in which the clinical trial was successfully executed. We define the intra-phase probability as the probability that the clinical trial is operationally completed. It applies to all phases f(Ph1); f(Ph2); f(Ph3). For example, f(Ph2) is the probability that the Phase II study will be conducted and completed regardless of the statistical / clinical results. This is the operational success of the clinical trial.

[0125] “End + Positive Outcome”: The state of successful execution of a clinical trial from an operational and statistical / clinical perspective. This is a necessary condition for advancing from one phase to the next. It applies to all phases I and II, i.e., f(Ph1-Ph2); f(Ph2-Ph3). For example, f(Ph1-Ph2) corresponds to the probability that a clinical trial will progress from phase I to phase II.

[0126] "Regulatory Approval" status: Status of regulatory approval. This status depends on the successful execution of clinical trials from both an operational and statistical / clinical perspective. It applies to all phases f(Ph1-App); f(Ph2-App); f(Ph3-App). For example, f(Ph3-App) would correspond to the probability of obtaining regulatory approval for a given drug in a given indication before the initiation of a pivotal Phase III trial.

[0127] “Commercial Success” Status: In healthcare systems such as the United States, commercial success of a pharmaceutical product is primarily associated with the Marketing Authorization Holder (MAH), i.e., obtaining regulatory approval from the U.S. Food and Drug Administration (FDA) to market the product. In Canada and some European countries, where healthcare systems have a significant public reimbursement component, such as INESSS [French National Institute of Excellence in Health and Social Services] in Quebec and CADTH [Canadian Agency for Drugs and Technologies in Health] in the rest of Canada, commercial success depends not only on obtaining marketing approval, but also on having the drug of interest added to the list of publicly reimbursed drugs (RAMQ [Quebec Health Insurance Agency] in Quebec). Therefore, in this context, if, in addition to predicting the “Ended” status, “Ended + Positive Outcome” and “Regulatory Approval” status, we could also predict the success rate of inclusion in the reimbursement list in the Canadian context (as described above), this would create significant value for the industry.

[0128] Therefore, the system is a dynamic, evolving and integrated process that allows covering the entire clinical drug development process up to commercial success.

[0129] By leveraging machine learning methods and deep drug development risk network mapping with millions of data points from multivariate data sets enriched by manual labeling by domain experts, these methods and systems will provide predictive and prescriptive insights from drug clinical development (including clinical study initiation to market access and commercial success) based on a priori estimates of multidimensional risks associated with candidate drug development. Finally, the methods and systems will generate these insights and deploy them in an understandable way to guide strategic decisions in drug portfolio management.

[0130] Other and further aspects and advantages of the present invention will become apparent upon understanding the illustrative embodiments that will be described or pointed out in the appended claims, and various advantages not mentioned herein will occur to those skilled in the art when employing the invention in practice.

[0131] BRIEF DESCRIPTION OF THE DRAWINGS

[0132] The above and other objects, features and advantages of the present invention will become more apparent from the following description with reference to the accompanying drawings, in which:

[0133] Figure 1 is a schematic diagram for a deeper understanding and mapping of the drug development risk network.

[0134] Figure 2 Schematic diagram of different interest state transitions of a prediction model according to the principles of the present invention.

[0135] Figure 3 Schematic diagram of a high-level architecture of an embodiment of a system and method for generating predictive but also prescriptive information to support strategic decisions a priori and during the clinical drug development phase according to the principles of the present invention. The system includes three main interrelated components: data sources, systems, and applications.

[0136] Figure 4 yes Figure 1 Workflow diagram of the data source of the system.

[0137] Figure 5 This is a system architecture diagram based on artificial intelligence algorithms.

[0138] Figure 6 is an architectural diagram of the execution and deployment modules, which are implemented as analytical, predictive, and prescriptive applications to support strategic decisions in the prior and clinical drug development stages.

[0139] Figure 7 It is a flowchart of an embodiment of a drug combination strategic management decision support method based on artificial intelligence according to the principles of the present invention.

[0140] Detailed Description of the Preferred Embodiments

[0141] A novel method and system for drug combination strategic management decision support based on artificial intelligence will be described below. Although the present invention is described with specific illustrative embodiments, it should be understood that the embodiments described herein are only examples, and the scope of the present invention is not intended to be limited thereby.

[0142] The present invention describes a system, method and platform for optimizing strategic drug portfolio management decision support using artificial intelligence. The platform is trained on a variety of structured and unstructured data from open and private data sources to generate predictive but also prescriptive information to support strategic decision making during the clinical drug development phase. The preferred embodiments selected to illustrate the present invention do not limit the scope of the present invention.

[0143] See also Figure 3 , shows an overall high-level architectural view of an embodiment of an artificial intelligence-based drug combination strategic management decision support system 100. The system is usually implemented as an intelligent platform for generating predictive and prescriptive information to support strategic decisions in the clinical drug development stage. The architectural structure is divided into three parts: data source, system, and application.

[0144] The system 100 generally includes a data acquisition module 10, a data processing module 30, and one or more applications 50. The data acquisition module 10 includes a program or application to extract or obtain data records from one or more data sources 11. The program can use an application programming interface (API), a network protocol, or a data file to retrieve the data records. The data acquisition module 10 can be configured to connect to a public domain database 12, an industrial partner 16, information from the Internet 20, knowledge from experts 24, and / or a government or official database 27.

[0145] The data sources 11 may include open and non-open databases. In some embodiments, the public domain database 12 may include a clinical trial database 13, such as, but not limited to, www.clinicaltrials.gov, an online database belonging to the U.S. government that contains more than 440,000 clinical trials conducted worldwide. In yet another embodiment, the clinical trial database may include EudraCT (Europe) - EudraCT (European Medicines Regulatory Agency Clinical Trials Database) contains information about drug interventional clinical trials conducted in the European Union (EU) or the European Economic Area (EEA) that began after May 1, 2004. Clinical Trials Database (Canada) - Health Canada's clinical trial database is populated with information about each clinical trial after Health Canada issues a NOL.

[0146] The public domain databases 12 may also include regulatory agency databases 14, such as, but not limited to, the US FDA, Health Canada's National Register of Authorized Medicinal Products, EMA, the European Union, and the like.

[0147] Industry partner data sources 16 may include proprietary databases 17, such as, but not limited to, clinical research organization databases. Information from the Internet 20 may include research databases 21, such as Medline TM , company drug pipeline 22, such as pharmaceutical company pipeline. Knowledge from expert sources 24 may include CRO pharmaceutical company executive key opinion leaders (go / no-go decision makers) 25.

[0148] Once all planned clinical trials have yielded positive results for a particular drug indication, the complete clinical dossier for the drug under evaluation is submitted to the regulatory authorities for evaluation for marketing approval, depending on the region in which the sponsor is interested in marketing and selling the drug. With subtle differences between the levels of information contained in these regulatory databases, we can find information about drugs, indications, and approval dates.

[0149] Government or official databases 27 may include public reimbursement databases 28, such as, but not limited to, INESS and CADTH databases, and commercial databases 29, such as market research databases.

[0150] As mentioned above, after receiving marketing authorization from the regulatory agency, depending on the geographic region where the drug is planned to be sold, additional steps are required to bring the drug to market, such as the Health Technology Assessment (HTA) review process. Once a particular drug indication has received marketing authorization from the regulatory agency, HTA is a multidisciplinary process that summarizes information on the medical, social, economic and ethical issues related to the use of health technologies. A CADTH Canadian Agency for Drugs and Technologies in Health (CADTH) reimbursement review is a comprehensive evaluation of the clinical effectiveness and cost-effectiveness of a drug or class of drugs, as well as the patient and clinician perspectives. The results of the evaluation provide non-binding recommendations that help guide reimbursement decisions by the federal, provincial and territorial governments of Canada (except Quebec). The database containing the CADTH recommendations can be downloaded in CSV format and contains 1043 entries. The INESSS National Institute of Excellence in Health and Social Services (INESSS) is the HTA agency responsible for recommending reimbursement decisions to the Quebec Ministry of Health. Once the Ministry of Health's decision is announced, the drug will be listed on the public list of drugs under the general drug insurance plan in Quebec pharmacies or hospitals. Manual extraction of the database is required. As of February 2023, the entire database contains 5978 entries. pCPA Following the positive reimbursement recommendations issued by CADTH / INESSS, the negotiation process with the Pan-Canadian Pharmaceutical Alliance (pCPA) proceeds through four steps, involving all provincial jurisdictions interested in participating in the process: 1-Initiation, 2-Consideration, 3-Negotiation, 4-Completion. These steps come with different sets of deliverables and different statuses are published on the pCPA website. 754 entries were manually extracted into Excel files. European Reimbursement Landscape: After obtaining marketing authorization from the European Medicines Agency (EMA), the various HTA bodies in the EU Member States compare the additional therapeutic benefits of the product being evaluated with other new or existing treatments. The reimbursement rules for a drug may be different in various EU countries, and the extent of prescription can vary greatly. The HTA process is followed by price negotiations, which can take place at the central or regional level. In fact, it is crucial to fully consider the planning of the trial design and the correct selection of key trial outcomes, as some European jurisdictions do not accept certain clinical endpoints in certain therapeutic areas as effective. Some basic factors must be considered to ensure that market access and sales are correctly predicted.

[0151] Still refer to Figure 3, the system 100 is generally configured to organize and process data obtained from different sources. The system 100 is also configured to process, convert and / or prepare data obtained from the data source (also referred to as raw data) for output to the artificial intelligence algorithm. In some embodiments, the system 100 is configured to store the prepared structured data in a data source, such as but not limited to a database, a file, or any other storage device. In such an embodiment, the system 100 is configured to execute a program or application module 50 that implements the artificial intelligence algorithm and provide the prepared structured data to the program. The artificial intelligence program 50 is configured to generate predictive but also normative information to support a priori analysis and strategic decision-making as well as decision-making during clinical development in various application areas. The system 100 is primarily based on machine learning.

[0152] The data processing module 30 is configured to process the data collected or obtained by the data collection module 10. The processing module 30 is configured to classify the data into a plurality of repositories 31, such as but not limited to clinical data 32, regulatory data 33, economic data 34, molecular data 35 and internal data 37.

[0153] The system may include an internal data source that includes internal data 37. The internal data source can be a structured relational database (which includes data from collected and annotated databases) and can include relationships that allow the linking of dispersed data collected from various databases. For example, clinical trials are accessible from clinicalTrials.gov and are listed in Phase 1, Phase 2, and Phase 3. Clinical trials that progress from Phase 1 to Phase 3 are not accessible to laymen or even internal personnel. The present method allows clinical trials of the same drug to be linked to a given indication or treatment condition that is related. Therefore, the internal data source contains structured data. The structured data can be logically linked or can create relationships with other data, such as data reimbursement from other databases (e.g., INESSS, CADTH, commercial pipelines, etc.). Different links or relationships are generally intended to provide retrospective and prospective pipelines of clinical trial data.

[0154] The processing module 30 may also include a data management and integration subsystem 40, which includes modules: managing data access 41, data management 42, data security 43 and data operations 44. The data is fed to a natural language processing unit 45, a machine learning unit 46 and / or a deep learning unit 47. The natural language processing unit 45, the machine learning unit 46 and / or the deep learning unit 47 are configured to be trained using data obtained from different data sources 11 and classified in the repository 31.

[0155] In some embodiments, system 100 is configured to organize and structure at least some unstructured data sources, such as using natural language processing (NLP) techniques.

[0156] The system also includes an application module 50. The application module 50 is configured to execute instructions that implement the algorithm and generate various analyses. In such an embodiment, users of the system 100 will be able to obtain the following knowledge: predictions of clinical phase transitions (Phase I, Phase II, Phase III, and regulatory approval / pharmacoeconomic approval), estimates of commercial success, normative data for optimizing research behavior (risk factor classification, study design recommendations, recruitment rate estimates, etc.), better control of risk factors / events in the clinical trial process by real-time monitoring of clinical research progress, more efficient use of internal resources (e.g., AI proactively classifies risk factors for each project, which will allow clinical trials to be prioritized effectively). More efficient use of internal resources (e.g., AI proactively classifies risk factors for each project, which will allow resources for each project to be prioritized effectively and optimize return on investment), proactive management of clinical trial behavior (e.g., predicting recruitment needs, predicting impact events (such as compliance or withdrawal)), and careful analysis to evaluate investment opportunities (i.e., mergers and acquisitions, venture capital).

[0157] Still refer to Figure 3 The application 50 is configured to use the results of the processed data to experiment, test and fine-tune the data. The output of the experimentation, testing and fine-tuning is deployed to predict, recommend, optimize, analyze, discover and / or report based on the processed data and the artificial intelligence engine (45, 46 and 47).

[0158] refer to Figure 4 and Figure 5 , the application 50 can also be divided into a data processing engine 60 and a model engineering 70. The information collected in the data acquisition module 10 is sent to the data processing engine 60 to classify, process and / or standardize the data. The data processing engine 60 may include a conversion module 61, which is configured to convert the data into an acceptable format, a standardization / normalization module 62, an induction module 63 and an encoding module 64. The induction module 63 can allow for the processing of missing data and incomplete data sets and reduce the bias caused by missing data. The encoding module 64 can allow for the conversion of labeled data points into numerical variables for model training. Once the data is processed by one or more of the data processing modules 61, 62, 63 and / or 64, the data is sent to a data preprocessing pipeline 65. The data preprocessing pipeline 65 sends the processed data to a selection sampler 66. The selection sampler 66 is configured to divide the data into a test set 67 and a training set 68. The test set 67 is configured to test different models, and the training set 68 is configured to train the machine learning unit 46.

[0159] The model engineering 70 is configured to receive the test set 67 and / or the training set 68 as a basis for developing a plurality of models 71, and to select and validate the developed models 75. The model development module 72 is configured to develop machine learning models, such as but not limited to random forests, support vector machines, gradient boosted trees, logistic regression, and neural networks, etc. The model development module 72 may also be configured to explain and interpret the developed models.

[0160] The model selection and validation module 74 may be used to select the best model developed in the model development module 72. Thus, the model selection and validation module 64 may include selecting the best model 75 based on performance metrics (e.g., accuracy, precision, and recall performance metrics). The model selection and validation module 74 may also include generating explanations and descriptions 76 of the decisions used in the model. The model selection and validation module 74 may also include validating the generated explanations and descriptions 77 by domain experts. For example, validation may include checking sensitive parameters that have a significant impact on the prediction. The model selection and validation module 74 may also include searching for counterfactual scenarios 78. In some embodiments, the search for counterfactual scenarios may include a target maximum likelihood estimation model.

[0161] The selected model is then executed and deployed in production.

[0162] Reference now Figure 5 and Figure 6, once the model is validated by the model selection and validation module 74, the validated model is sent to the application module 50. The application module may include execution 51 and deployment 55. Execution 51 includes an experiment component 52, a test component 53, and an adjustment component 54. Therefore, the execution module 51 allows the model to be experimented, tested, and fine-tuned based on expected and counterfactual scenarios. The deployment module 55 can use the selected trained and tested model to make decisions based on new data. Therefore, the deployment module 55 may include a prediction component 56, which can predict the success rate of clinical trials for different states (e.g., "end", "end + positive results", "regulatory approval", "commercial success"). The deployment module 55 may also include a recommendation component 57, which can generate recommendations that can change its modifiable factors to affect the prediction of the prediction component 56. The deployment module 55 may also include an optimization component 58, allowing users to access a set of different training models for different clinical trial stages, while providing users with explanations of model decisions. The deployment module 55 may also include an analysis component 59, allowing users to evaluate influencing features, change these features and adjust them, and evaluate counterfactual success predictions. The deployment module 55 may also include a discovery component 81 that allows the user to discover the importance of specific features and reveal the impact of specific features on the final decision. The deployment module 55 may also include a reporting component 82 that is configured to display a summary of the predictions, predictors, models and counterfactual recommendations and explanations on a user interface.

[0163] In an embodiment having a platform, the platform can be constructed to integrate a data environment organized into major clusters. In such an embodiment, each cluster represents a module: a clinical risk module, a regulatory risk module, a commercial risk module, and a pharmacological risk module. Each module includes data that is typically derived from public or private databases. The nature of the data varies and includes information about clinical trials, regulatory approval decisions, economic and reimbursement data, pharmacological data, commercial data, and company data.

[0164] refer to Figure 7 , a flow chart of an embodiment of a drug combination strategic management decision support method 200 based on artificial intelligence is shown. The method 200 includes receiving a request 201 for a new treatment study. The request 201 for a new treatment study may include inputting one or more characteristics 202 related to the study. Characteristics related to the study may include, but are not limited to, characteristics of the study, characteristics of study participants, characteristics of the treatment to be studied, characteristics of the treatment study method, characteristics of the sponsors and collaborators of the study, etc. The input information is generally intended to collect raw data about the study.

[0165] The input data, also referred to as raw data, is input into a data preprocessing pipeline 203. The input data is standardized 204, summarized 205, and / or encoded 206. Thus, the data is now formatted or processed for input into an artificial intelligence (AI) program configured to execute an AI algorithm.

[0166] The preprocessed data is input into a multi-layer model 207 to calculate predictions about the success of the clinical trial of the proposed study. For example, a first layer model 208 can be configured to calculate intermediate predictions 209 of certain aspects of the success of the clinical trial. For example, the first layer model 208 can include calculating predictions 210 of target recruitment, calculating predictions 211 of protocol deviations, and calculating predictions of other factors. The predictions according to the different aspects represent intermediate predictions of the success of the clinical study.

[0167] For example, the first layer model 208 is the lowest level in the exemplary structure, and the prediction goal can correspond to the business needs of the contract research organization (CRO), specifically, predicting whether the clinical trial will be completed. Therefore, the method can include developing a machine learning model (layer 1) for predicting success based on the specific business needs of the CRO. The developed model identifies factors that have a greater impact on the prediction results of the developed model among all factors. The identified factors may include, but are not limited to, protocol deviations, the number of patients recruited, the duration of the clinical trial, the withdrawal rate, and / or the age of the patients in the trial. The identified factors can be the most important predictive contributors. Unlike hundreds of other variables in the database, the most important predictive contributors are unknown at the beginning of the clinical trial. Therefore, these variables can be used in the first layer model 208 to predict the success of the clinical trial.

[0168] The intermediate predictions 209 are input to the second layer model 212. The second layer model 212 is configured to make predictions for clinical trial success 213 based on the intermediate predictions 209 of the first layer model 208. It can be appreciated that the intermediate predictions 209, combined with the variables that contribute the most, should effectively predict the target of interest (completed or incomplete clinical trials). In the illustrated workflow, in the first layer 208, the model is expected to evolve with the predictions of subsequent targets until commercial success is achieved.

[0169] The calculated intermediate predictions 209 and the calculated clinical trial success predictions 213 are input into the explainability and illustrative module 220. The explainability and illustrative module 220 is generally intended to explain and illustrate the calculated intermediate predictions 209 and the calculated clinical trial success predictions 213. At each stage of the process, the method provides intermediate predictions 208. The intermediate predictions 209 identify the degree of influence of the variables being used in such models 208. It is understood that the range of influences includes positive, neutral or negative influences. Therefore, since the multi-layer model 207 can include multiple layers, each layer identifies the degree of influence of each variable used in each model of the layer. Therefore, the system 100 explains the predicted success of the clinical study by analyzing the different effects of the intermediate predictions and variables of such hierarchical models. Therefore, different intermediate predictions reveal the explanations of these variables at each stage of the multi-layer model 207 to provide explainability and illustrativeness.

[0170] In some embodiments, module 220 may include a global explanation subsystem 221 configured to generate a summary 223 of logical rules and rule logic 224 divided by attributes. And the module may also include a local explanation module 225 configured to generate or produce contribution attributes 226, comparisons 227 with similar studies, contrasting explanations 228, and counterfactual (hypothetical) scenarios 229. As described above, the global explanation subsystem 221 and the local explanation module 225 use the intermediate predictions 208 and the relevant variables output at each period of the multi-layer 207 model to generate logical rules or other contribution attributes. The explainability and illustrativeness module 220 can be configured to generate and present to the user an explanation or basis for the predictions made by the AI ​​algorithm. In this way, the user can make better and more informed decisions about the research at hand.

[0171] Method 200 also includes making a decision 230. Decision 230 can include using prediction 213 of research success, interpretation 220 of prediction, intermediate prediction 208, interpretation 231 of intermediate predictions and other predictions.

[0172] The use of multi-period models is generally intended to decompose complex decision-making processes into multiple periods, levels or stages. In addition, the use of multi-period model 207 reveals the specific impact of each level and related variables on the prediction of clinical trial success. In such embodiments, each period or level represents a specific aspect or consideration in the decision-making process, and one or more predictions are calculated for each period. Multi-period methods are generally intended to provide a systematic and comprehensive analysis. The prediction of each period or level is generally used as an input source for the next period or level. Therefore, the next period uses the results of the previous period to calculate another prediction, which is enhanced relative to the intermediate prediction. Therefore, the system can decompose the complex problem of predicting the success or failure of clinical trial research into multiple prediction calculations for smaller problems. The use of multi-period models further allows the overall performance of the engine to be improved.

[0173] In embodiments using multi-period models, method 200 is generally intended to unlock or compute predictions for specific situations or global predictive aspects by providing a more targeted and modular problem-solving approach. By breaking down complex problems into simpler steps, each intermediate model can focus on specific aspects and solve specific sub-problems. Interpreting the results and understanding the information provided by these models or periods is critical to understanding the problem being solved and effectively utilizing the information provided by the models. For example, a multi-period model can be used to predict the percent probability of success in reaching the target number of participants in a Phase 2 clinical study, or to assess the likelihood that a Phase 2 clinical study falls into a high risk area for protocol deviation, or to estimate the duration (in days) of a Phase 2 clinical study.

[0174] Each period of a multi-period model typically requires a separate training and evaluation process, as well as management of intermediate inputs and outputs. For example, a multi-period model may include periods associated with estimates of clinical study volunteer recruitment, periods associated with estimates of protocol deviations, periods associated with estimates of volunteer withdrawal from the study, or periods associated with estimates of study duration. It will be appreciated that any other number of aspects or periods may be considered within the scope of the present invention.

[0175] Although exemplary and presently preferred embodiments of the present invention have been described in detail above, it should be understood that the concepts of the present invention may be embodied and used in other various ways and that the appended claims are intended to be interpreted to include such variations unless limited by prior art.

[0176] References

[0177] CenterWatch.(nd).Retrieved June 2,2022,from https: / / www.centerwatch.com / articles / 16879

[0178] Babbs,C.F.(2014).Choosing inclusion criteria that minimize the timeand cost of clinical trials.World Journal of Methodology,4(2),109–122.https: / / doi.org / 10.5662 / wjm.v4.i2.109

[0179] Dhankhar,A.,Saptarshi,G.,Arvind,G.,&Michael,T.(2018).Expandinghorizons for risk management in pharma|McKinsey.https: / / www.mckinsey.com / business-functions / risk-and-resilience / our-insights / expand ing-horizons-for-risk-management-in-pharma

[0180] Dickson,M.,&Gagnon,J.P.(2004).Key factors in the rising cost of newdrug discovery and development.Nature Reviews.Drug Discovery,3(5),417–429.https: / / doi.org / 10.1038 / nrd1382

[0181] Fogel,D.B.(2018).Factors associated with clinical trials that failand opportunities for improving the likelihood of success:Areview.Contemporary Clinical Trials Communications,11,156–164.https: / / doi.org / 10.1016 / j.conctc.2018.08.001

[0182] Fouad,M.N.,Lee,J.Y.,Catalano,P.J.,Vogt,T.M.,Zafar,S.Y.,West,D.W.,Simon,C.,Klabunde,C.N.,Kahn,K.L.,Weeks,J.C.,&Kiefe,C.I.(2013).Enrollment ofpatients with lung and colorectal cancers onto clinical trials.Journal ofOncology Practice,9(2),e40-47.https: / / doi.org / 10.1200 / JOP.2012.000598

[0183] Getz,K.A.,Zuckerman,R.,Cropp,A.B.,Hindle,A.L.,Krauss,R.,&Kaitin,K.I.(2011).Measuring the Incidence,Causes,and Repercussions of ProtocolAmendments.Drug Information Journal:DIJ / Drug Information Association,45(3),265–275.https: / / doi.org / 10.1177 / 009286151104500307

[0184] Harrison,R.K.(2016).Phase II and phase III failures:2013-2015.NatureReviews.Drug Discovery,15(12),817–818.https: / / doi.org / 10.1038 / nrd.2016.184

[0185] Hoskyn,S.L.(n.d.).EXPLAINING PUBLIC REIMBURSEMENT DELAYS FOR NEWMEDICINES FOR CANADIAN PATIENTS.3.

[0186] Hwang,T.J.,Carpenter,D.,Lauffenburger,J.C.,Wang,B.,Franklin,J.M.,&Kesselheim,A.S.(2016).Failure of Investigational Drugs in Late-StageClinicalDevelopment and Publication of Trial Results.JAMA Internal Medicine,176(12),1826–1833.https: / / doi.org / 10.1001 / jamainternmed.2016.6008

[0187] Kwak,Y.,&Dixon,C.(2008).Risk management framework forpharmaceuticalresearch and development projects.https: / / doi.org / 10.1108 / 17538370810906255

[0188] Lamberti,M.J.,Mathias,A.,Myles,J.E.,Howe,D.,&Getz,K.(2012).Evaluatingthe Impact of Patient Recruitment and Retention Practices.DrugInformationJournal:DIJ / Drug Information Association,46(5),573–580.https: / / doi.org / 10.1177 / 0092861512453040

[0189] Pammolli,F.,Magazzini,L.,&Riccaboni,M.(2011).The productivity crisisinpharmaceutical R&D.Nature Reviews.Drug Discovery,10(6),428–438.https: / / doi.org / 10.1038 / nrd3405

[0190] Paul,S.M.,Mytelka,D.S.,Dunwiddie,C.T.,Persinger,C.C.,Munos,B.H.,Lindborg,S.R.,&Schacht,A.L.(2010a).How to improve R&D productivity:Thepharmaceutical industry’s grand challenge.Nature Reviews Drug Discovery,9(3),203–214.https: / / doi.org / 10.1038 / nrd3078

[0191] Paul,S.M.,Mytelka,D.S.,Dunwiddie,C.T.,Persinger,C.C.,Munos,B.H.,Lindborg,S.R.,&Schacht,A.L.(2010b).How to improve R&D productivity:Thepharmaceutical industry’s grand challenge.Nature Reviews.Drug Discovery,9(3),203–214.https: / / doi.org / 10.1038 / nrd3078

[0192] Guide.(n.d.).Retrieved June 14,2022,fromhttps: / / www.pmi.org / pmbok-guide-standards / foundational / PMBOK

[0193] Scannell,J.W.,Blanckley,A.,Boldon,H.,&Warrington,B.(2012).Diagnosingthe decline in pharmaceutical R&D efficiency.Nature Reviews.DrugDiscovery,11(3),191–200.https: / / doi.org / 10.1038 / nrd3681

[0194] Schlander,M.,Hernandez-Villafuerte,K.,Cheng,C.-Y.,Mestre-Ferrandiz,J.,&Baumann,M.(2021).How Much Does It Cost to Research and Develop a NewDrug?ASystematic Review and Assessment.PharmacoEconomics,39(11),1243–1269.https: / / doi.org / 10.1007 / s40273-021-01065-y

[0195] Schroen,A.T.,Petroni,G.R.,Wang,H.,Gray,R.,Wang,X.F.,Cronin,W.,Sargent,D.J.,Benedetti,J.,Wickerham,D.L.,Djulbegovic,B.,&Slingluff,C.L.(2010).Preliminary evaluation of factors associated with premature trialclosure andfeasibility of accrual benchmarks in phase III oncologytrials.Clinical Trials(London,England),7(4),312–321.https: / / doi.org / 10.1177 / 1740774510374973

[0196] Schuhmacher,A.,Gassmann,O.,&Hinder,M.(2016).Changing R&D modelsinresearch-based pharmaceutical companies.Journal of Translational Medicine,14(1),105.https: / / doi.org / 10.1186 / s12967-016-0838-4

[0197] Serebruany,V.L.,Oshrine,B.R.,Malinin,A.I.,Atar,D.,Michelson,A.D.,&Ferguson,J.J.(2005).Noncompliance in cardiovascular clinicaltrials.AmericanHeart Journal,150(5),882–886.https: / / doi.org / 10.1016 / j.ahj.2005.02.039

[0198] Smith,D.L.(2012).Patient Nonadherence in Clinical Trials:Could ThereBe aLink to Postmarketing Patient Safety?Drug Information Journal:DIJ / DrugInformation Association,46(1),27–34.https: / / doi.org / 10.1177 / 0092861511428300

[0199] Thoma,A.,Farrokhyar,F.,McKnight,L.,&Bhandari,M.(2010).How tooptimizepatient recruitment.Canadian Journal of Surgery,53(3),205–210.https: / / www.ncbi.nlm.nih.gov / pmc / articles / PMC2878987 /

[0200] Williams,R.J.,Tse,T.,DiPiazza,K.,&Zarin,D.A.(2015).Terminated Trialsinthe ClinicalTrials.gov Results Database:Evaluation of Availability ofPrimaryOutcome Data and Reasons for Termination.PloS One,10(5),e0127242.https: / / doi.org / 10.1371 / journal.pone.0127242

Claims

1. A system for predicting the success of a clinical trial of a drug product, the system comprising: A data source, wherein the data source includes data related to a clinical trial; A server, the server comprising: a data acquisition module that communicates data with a plurality of external data sources, the plurality of external data sources including data related to clinical trials, regulatory approvals, economic and reimbursement information, pharmacological information, and commercial and company information; a data processing engine configured to convert and normalize data from the data acquisition module using a natural language processor; A machine learning engine comprising a model trained using data processed by the data processing engine, the trained machine learning engine being configured to execute an algorithm to analyze data from a data source related to the clinical trial and to calculate a prediction of success of the clinical trial based on the analyzed data.

2. According to the system of claim 1, the model trained by the machine learning engine is a multi-layer model.

3. The system of claim 2, wherein the multi-layer model comprises a first-layer model configured to calculate a plurality of successful intermediate predictions.

4. The system of claim 3, wherein the multi-layer model comprises a plurality of first-layer models, each model being configured to compute an intermediate prediction of success.

5. The system of claim 4, wherein the plurality of first-layer models comprises at least one of the following models: a model for calculating target recruitment predictions for said clinical trial; a model for calculating predictions of protocol deviations for the clinical trial; and Models used to calculate other factors relevant to the clinical trial.

6. The system of claim 4, wherein each intermediate prediction of success of the first layer model is input into a second layer model to calculate a prediction of success of the clinical trial.

7. The system of claim 1, further comprising a module for interpreting and explaining the calculated prediction of success of the clinical trial.

8. The system of claim 7, the means for interpreting and explaining the calculated prediction of success of the clinical trial comprises generating logic rules for calculating the prediction.

9. The system of claim 7, wherein the means for interpreting and explaining the calculated prediction of success of the clinical trial comprises any of the following: the contributing attributes of the clinical trial; Studies for comparison with the clinical trials described; contrastive explanation; Scenarios that affect the expected degree of success of the clinical trial.

10. The system of claim 1, further comprising an application module configured to execute the machine learning engine using data related to the clinical trial.

11. The system of claim 1, wherein the acquired data sources are classified in a plurality of repositories.

12. The system of claim 11, wherein the repository comprises any of the following types of data: clinical data, regulatory data, economic data, molecular data, and MPP.

13. A computer-implemented method for predicting the success of a clinical trial of a pharmaceutical product, comprising: Acquire data from multiple external data sources, including data related to clinical trials, regulatory approvals, economic and reimbursement information, pharmacological information, and commercial and company information; Use natural language processors to process, transform and normalize the acquired data, Executing a machine learning model trained using the processed, transformed, and normalized data to analyze data related to the clinical trial and calculate a prediction of success of the clinical trial based on the analyzed data.

14. The method of claim 13, wherein the trained model is a multi-layer model, comprising a second-layer model and multiple first-layer models, and the method further comprises each of the first-layer models calculating an intermediate prediction of success of a particular aspect of the clinical study.

15. The method of claim 14, wherein each of the plurality of first-layer models calculates one of the following: prediction of target recruitment for said clinical trial; prediction of protocol deviations of the clinical trial; and Other factors relevant to the clinical trial.

16. The method of claim 15, wherein the second tier model uses each of the intermediate predictions calculated by the first tier model to calculate a prediction of success of the clinical trial.

17. The method of claim 13, further comprising monitoring real-time progress characteristics of the clinical study and using these characteristics to calculate a prediction of success of the clinical trial.

18. The method of claim 17, wherein the features include predictions of recruitment needs and identification of influencing events.

19. The method according to claim 13, executing the machine learning model to further calculate any of the following: clinical risk of the clinical trial, regulatory risk of the clinical trial, commercial risk of the clinical trial and pharmacological risk of the clinical trial.

20. The method of claim 13, executing the machine learning model to further generate normative data for optimizing research behavior.

21. The method of claim 13, further comprising developing multiple machine learning models for the clinical trial, training the developed models using the acquired data, and selecting one or more of the developed models based on performance metrics.

22. A computer readable medium storing instructions for executing the method of claim 13.

Images