Methods and systems for mitigating errors in a causal inference process

Abstract

Methods and server systems for mitigating errors in a causal inference process are described herein. The method performed by a server system includes accessing, for each of target and control entity, pre-treatment time series information and post-treatment time series information. Then, generating, by Machine Learning (ML) model, pre-treatment prediction time series information for target entity based on pre-treatment time series information of control entity. Then, computing a set of prediction error values by comparing pre-treatment prediction time series information and pre-treatment time series information of target entity. Then, generating, by ML model, post-treatment prediction time series information for target entity based on pre-treatment time series information of target entity and post-treatment time series information of control entity. Then, generating prediction range information based on post-treatment prediction time series information and set of prediction error values.

Description

METHODS AND SYSTEMS FOR MITIGATING ERRORS IN A CAUSALINFERENCE PROCESSTECHNICAL FIELD

[0001] The present disclosure relates to the field of causal inference for product impact evaluation and, more particularly, to electronic methods and complex processing systems for mitigating errors in a causal inference process being performed for determining the impact of an intervention.BACKGROUND

[0002] Nowadays, business decisions are becoming increasingly reliant on data-driven insights for maintaining their competitiveness in the marketplace. A commonly used technique for determining such data-driven insights is known as causal inference. The goal of causal inference is to understand the cause-and-effect relationship between different variables or decisions. In various instances, causal inference can be used to assess the impact of new product releases, marketing campaigns, political campaigns or initiatives, new pharmaceutical developments, new medicinal drugs, public policies, and so on. As would be apparent, causal inference plays a significant role in the field of data analytics, by empowering businesses or organizations in making informed decisions through the insights gained through the causal inferencing process.

[0003] Generally, causal inference is performed using Artificial Intelligence (Al) or Machine Learning (ML) algorithms that are designed to analyze complex datasets and learn the underlying causal relationships within these datasets. However, such an approach suffers from inherent challenges, particularly related to the accuracy of the algorithm being used for the inferencing process. For instance, an ML model may overestimate the impact of a new product offered by a business, due to errors in its predictions. Such an ML model will thereby generate incorrect results. As may be understood, such challenges arise from the intrinsic complexity of the input datasets, limited accuracy of the ML algorithm being used, data limitations, stochastic variability, and so on. As a result, inaccurate predictions may lead to biased estimates of the causal effect, undermining the reliability and validity of the causal inference results.

[0004] Thus, it is desirable to find technological solutions that mitigate the errors in a causal inference process, thereby improving the reliability and quality of the causal inferenceresults.SUMMARY

[0005] There exists a need for techniques to overcome one or more limitations stated above such as incorrect and unreliable results during a causal inference process due to errors within the AI / ML model or algorithm. Various embodiments of the present disclosure provide methods and systems for mitigating errors in a causal inference process being performed for determining the impact of an action such as the implementation of a new product.

[0006] Various embodiments of the present disclosure describe a computing device and a method that determines a distribution of errors present within the predictions of an ML model being used for undertaking a causal inference process. Further, this distribution of error can be used to determine the actual impact of an action accurately while eliminating the inaccuracies introduced due to the errors. With improved and accurate predictions, the decision-making process related to the causal inference task can be easily improved as well. Thus, leading to robust and reliable decision-making practices for businesses.

[0007] To achieve the above and other objectives of the present disclosure, in one aspect, a computer-implemented method for mitigating errors in a causal inference process is disclosed. The computer-implemented method is performed by a server system. The computer- implemented method includes accessing, for each of a target entity and a control entity, pretreatment time series information and post-treatment time series information from a database. The computer-implemented method further includes generating, by a Machine Learning (ML) model, pre-treatment prediction time series information for the target entity based, at least in part, on the pre-treatment time series information of the control entity.

[0008] The computer-implemented method further includes computing a set of prediction error values by comparing the pre-treatment prediction time series information and the pre-treatment time series information of the target entity. The computer-implemented method further includes generating, by the ML model, post-treatment prediction time series information for the target entity based, at least in part, on the pre-treatment time series information of the target entity and the post-treatment time series information of the control entity. The computer-implemented method further includes generating prediction range information for the target entity based, at least in part, on the post-treatment prediction time series information and the set of prediction error values.

[0009] An advantage of some embodiments is that by generating a set of predictionerror values, an initial set of errors present in the outcomes of the ML model can be ascertained and accounted for. Another advantage of some embodiments is that the prediction range information provides insights into a distribution of errors present within the predictions / outcomes of the ML model. Further, the prediction range information provides accurate causal inference results, since it allows the ML to rectify its outputs based on the detected errors.

[0010] In an aspect, the step of generating the prediction range information includes performing a set of operations iteratively for predefined iterations. The set of operations includes (A), (B) randomly selecting a test value from the distribution of MBE values, (C) computing at least one of an error increment value and an error decrement value for each outcome prediction in the post-treatment prediction time series information based, at least in part, on the post-treatment prediction time series information and the test value, and (D) computing a percentage change value for each outcome prediction based, at least in part, on an actual outcome from the post-treatment time series information of the target entity and at least one of the error increment value and the error decrement value computed for each outcome prediction in the post-treatment prediction time series information.

[0011] An advantage of some embodiments is that computing the MBE instead of other forms of error allows the server system to accurately capture the distribution of errors present within the predictions / outcomes of the ML model. Another advantage of some embodiments is that by computing the error increment value and the error decrement value, the technical problem associated with underfitting and overfitting the ML model can be avoided.

[0012] In an aspect, the step of generating the prediction range information further includes computing a mean percentage change value based, at least in part, on the percentage change value computed from each iteration of the predefined iterations. The step further includes storing the mean percentage change value as the prediction range information.

[0013] An advantage of some embodiments is that the mean percentage change value provides more accurate insights into the actual impact of an action such as the impact due to the introduction of a new product in the market.

[0014] In an aspect, the computer-implemented method further includes computing a confidence interval for the ML model based, at least in part, on a confidence score of the ML model.

[0015] An advantage of some embodiments is that the confidence interval helps todefine a range for causal inference. As may be appreciated, the selection of a good confidence interval helps to improve the results of the ML model. Since results present within the confidence interval have a greater likelihood of being accurate, selecting such results while determining the impact of an action will provide accurate results.

[0016] In an aspect, the computer-implemented method further includes determining a model impact factor for the ML model based, at least in part, on the confidence interval and the prediction range information. In an aspect, the step of determining the model impact factor further includes selecting a set of percentage change values within the confidence interval from the percentage change value computed from each operation. The step further includes setting the model impact factor based, at least in part, on the set of percentage change values.

[0017] In an aspect, the computer-implemented method further includes training the ML model based, at least in part, on the pre-treatment time series information of the target entity and the pre-treatment time series information of the control entity.

[0018] An advantage of some embodiments is that training the ML model based on pre-treatment time series information of the target and control entity teaches the model about the usual operational performance of these entities. This makes it easier for the model to learn of any discrepancies due to an action such as the implementation of a new product on the target entity by comparing the performance of the target entity with the pre-action performance and the post- action performance of a control entity where the action has not been performed.

[0019] As per another embodiment of the present disclosure, a server system is disclosed. The server system includes a communication interface and a memory including executable instructions. The server system also includes a processor communicably coupled to the memory. The processor is configured to execute the instructions to cause the server system, at least in part, to access, for each of a target entity and a control entity, pre-treatment time series information and post-treatment time series information from a database. The server system is further caused to generate, by a Machine Learning (ML) model, pre-treatment prediction time series information for the target entity based, at least in part, on the pretreatment time series information of the control entity. The server system is further caused to compute a set of prediction error values by comparing the pre-treatment prediction time series information and the pre-treatment time series information of the target entity. The server system is further caused to generate, by the ML model, post-treatment prediction time series information for the target entity based, at least in part, on the pre-treatment time seriesinformation of the target entity and the post-treatment time series information of the control entity. The server system is further caused to generate prediction range information for the target entity based, at least in part, on the post-treatment prediction time series information and the set of prediction error values.

[0020] As per yet another embodiment of the present disclosure, a non-transitory computer-readable storage medium is disclosed. The non-transitory computer-readable storage medium includes computer-executable instructions that, when executed by at least a processor of a server system, cause the server system to perform a method. The method includes accessing, for each target entity and a control entity, pre-treatment time series information and post-treatment time series information from a database. The method further includes generating, by a Machine Learning (ML) model, pre-treatment prediction time series information for the target entity based, at least in part, on the pre-treatment time series information of the control entity.

[0021] The method further includes computing a set of prediction error values by comparing the pre-treatment prediction time series information and the pre-treatment time series information of the target entity. The method further includes generating, by the ML model, post-treatment prediction time series information for the target entity based, at least in part, on the pre-treatment time series information of the target entity and the post-treatment time series information of the control entity. The method further includes generating prediction range information for the target entity based, at least in part, on the post-treatment prediction time series information and the set of prediction error values.

[0022] The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the drawings and the following detailed description.BRIEF DESCRIPTION OF FIGURES

[0023] For a more complete understanding of example embodiments of the present technology, reference is now made to the following descriptions taken in connection with the accompanying drawings in which:

[0024] FIG. 1 is an example representation of a conversational interaction environment, in accordance with various embodiments of the present disclosure;

[0025] FIG. 2 illustrates a simplified block diagram of a server system, in accordancewith an embodiment of the present disclosure;

[0026] FIG. 3 illustrates a schematic representation of the process for generating customized agent replies for an agent in response to a client email from a client, in accordance with an embodiment of the present disclosure;

[0027] FIG. 4 illustrates a representation of various Graphical User Interfaces (GUIs), in accordance with various embodiments of the present disclosure; and

[0028] FIG. 5 illustrates a flow diagram of a method of operating the server system for generating customized reply recommendations for an agent involved in a conversational interaction with a client, in accordance with an embodiment of the present disclosure.

[0029] The drawings referred to in this description are not to be understood as being drawn to scale except if specifically noted, and such drawings are only exemplary in nature.DETAILED DESCRIPTION

[0030] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, to one skilled in the art that the present disclosure can be practiced without these specific details. Descriptions of well-known components and processing techniques are omitted to not obscure the embodiments herein unnecessarily. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.

[0031] References in this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. The appearances of the phrase “in an embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Moreover, various features are described which may be exhibited by some embodiments and not by others. Similarly, various requirements are described which may be requirements for some embodiments but not for other embodiments.

[0032] Moreover, although the following description contains many specifics for thepurposes of illustration, anyone skilled in the art will appreciate that many variations and / or alterations to said details are within the scope of the present disclosure. Similarly, although many of the features of the present disclosure are described in terms of each other, or in conjunction with each other, one skilled in the art will appreciate that many of these features can be provided independently of other features. Accordingly, this description of the present disclosure is set forth without any loss of generality to, and without imposing limitations upon, the present disclosure.

[0033] Conditional language such as, among others, "can," "could," "might" or "may," unless specifically stated otherwise, are otherwise understood within the context as used in general to convey that certain embodiments include, while other embodiments do not include, certain features, elements and / or steps. Thus, such conditional language is not generally intended to imply that features, elements, and / or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without user input or prompting, whether these features, elements and / or steps are included or are to be performed in any particular embodiment.

[0034] Disjunctive language such as the phrase "at least one of X, Y, or Z" unless specifically stated otherwise, is otherwise understood with the context as used in general to present that an item, term, etc., may be either X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Thus, such disjunctive language is not generally intended to, and should not, imply that certain embodiments require at least one of X, at least one of Y, or at least one of Z to each be present.

[0035] Unless otherwise explicitly stated, articles such as "a" or "an" should generally be interpreted to include one or more described items. Accordingly, phrases such as "a server system configured to" are intended to include one or more recited server systems / processors. Such one or more recited devices can also be collectively configured to carry out the stated recitations. For example, "a processor configured to carry out recitations A, B, and C" can include a first processor configured to carry out recitation A working in conjunction with a second processor configured to carry out recitations B and C. The same holds true for the use of definite articles used to introduce embodiment recitations. In addition, even if a specific number of an introduced embodiment recitation is explicitly recited, those skilled in the art will recognize that such recitation should typically be interpreted to mean at least the recited number (e.g., the bare recitation of "two recitations," without other modifiers, typically means at least two recitations or two or more recitations).

[0036] It will be understood by those within the art that, in general, terms used herein, are generally intended as "open" terms (e.g., the term "including" or “comprising” should be interpreted as "including / comprising but not limited to," the term "having" should be interpreted as "having at least," the term "includes" or “comprises” should be interpreted as "includes / comprises but is not limited to," etc.).

[0037] For expository purposes, the term ‘causal inference’ refers to the process of determining the causal relationship between variables by analyzing observational or experimental data and accounting for potential confounding factors. Causal inference is a branch of statistical analysis and reasoning that aims to understand and quantify how a change in one variable causes a change in other variables within a system. Causal inference can be used in various fields, including economics, healthcare, social sciences, and business, as it allows decision-makers to identify the factors that drive outcomes and make informed decisions based on causally valid evidence.

[0038] As described earlier, causal inference plays a vital role in the decision-making process for businesses. A commonly used conventional technique for performing causal inference is known as synthetic control. The synthetic control technique aims to estimate the causal effect of an intervention such as a new product or service launch, a public policy, a political campaign, a marketing campaign, and so on, by comparing the outcomes of a target entity (or treated unit) (e.g., a region where a company released a new product) with those of a control entity (or control unit) (e.g., a region where the company didn’t release the product). In the synthetic control technique, a machine learning or forecasting model is employed to predict or forecast the values of the outcomes after the intervention has taken place in the target entity. Although synthetic control is considered the gold standard for performing causal inference of an intervention, it suffers from various problems. For instance, due to the complex nature of input data for the model and the complexity of the underlying relationships within the input data, the predictions generated by machine learning or forecasting models are prone to errors. The presence of these errors leads to incorrect results or assumptions for the causal inference process. Such incorrect results or assumptions may lead to invalid causal inference results. If an organization or business uses these invalid causal inference results to form their policy or business decisions, it might lead to flawed decision-making, financial loss, and suboptimal resource allocation. For instance, if a government performs a causal inference process to determine the impact of a new public policy, then invalid results may indicate that this public health policy is showing good results for the population, while in reality, the policymay be causing harm to public health. Therefore, it is crucial to improve the existing process of performing causal inference by mitigating such errors or inaccuracies.

[0039] To that end, various embodiments of the present disclosure aim to solve the above-mentioned technical problems by providing an approach for mitigating errors in the causal inference process. The approach of the present disclosure aims to reduce the impact of errors in the causal inference process to improve the accuracy of the causal inference results.

[0040] FIG. 1 is an example representation of an environment 100 for performing the causal inference process, in accordance with various embodiments of the present disclosure. The environment 100 includes a server system 102, a target entity 104, and a control entity 106, each coupled to, and in communication with (and / or with access to) a network 108.

[0041] The network 108 may include, without limitation, a Light Fidelity (Li-Fi) network, a Local Area Network (LAN), a Wide Area Network (WAN), a Metropolitan Area Network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an Infrared (IR) network, a Radio Frequency (RF) network, a virtual network, and / or another suitable public and / or private network capable of supporting communication among two or more of the parts or components illustrated in FIG. 1, or any combination thereof.

[0042] Various entities in the environment 100 may connect to the network 108 in accordance with various wired and wireless communication protocols, such as Transmission Control Protocol / Intemet Protocol (TCP / IP), User Datagram Protocol (UDP), 2nd Generation (2G), 3rd Generation (3G), 4th Generation (4G), 5th Generation (5G) communication protocols, Long Term Evolution (LTE) communication protocols, future communication protocols or any combination thereof. For example, the network 108 may include multiple different networks, such as a private network made accessible by the server system 102 and a public network (e.g., the Internet, etc.) through which the server system 102, the target entity 104, and the control entity 106 may communicate.

[0043] In an embodiment, the target entity 104 represents an entity, a marketplace, a region, a subject, a group of subjects, or individuals that are subjected to a specific intervention or treatment by an organization or business (not shown). The target entity 104 is the entity for which the causal inference process is performed to determine the impact of the said intervention. In various non-limiting examples, the intervention may refer to at least one of a new product launch, a public policy, a marketing campaign, a political campaign, an economic intervention, a medical treatment procedure, a new pharmaceutical drug, and so on. Forinstance, if a business wishes to perform the causal inference process to ascertain the impact of a new product category on their revenue in a particular region, they may deem the said region as the target entity 104. In another instance, if a pharmaceutical company wishes to know the impact of a new drug on the health of patients, they may select a group of patients enrolled in a pharmaceutical trial as the target entity 104. It is noted that the terms ‘target entity 104’, ‘treated unit’, ‘treatment unit’, and ‘target unit’ may be used interchangeably throughout the present disclosure.

[0044] In an embodiment, the control entity 106 represents an entity, a marketplace, a region, a subject, a group of subjects, or individuals that are not subjected to a specific intervention or treatment by an organization or business but are closely related to the target entity 104. The control entity 106 is the entity that acts as a comparison or baseline against which the effects of the intervention on the target entity 104 can be evaluated. In particular, the control entity 106 is carefully created such that it shares similar characteristics with the target entity 104 to ensure that a comparison between these entities may be performed. As may be appreciated, unlike the target entity 104, the control entity 106 does not undergo the intervention. During the causal inference process, the control entity 106 is not subjected to the intervention, thus ensuring that the data associated with it can be used as a baseline to determine the impact of the intervention on the target entity 104.

[0045] For instance, if a business wishes to perform the causal inference process to ascertain the impact of a new product category on their revenue in a particular region, they may deem another region with similar revenue and market conditions as the control entity 106. In another instance, if a pharmaceutical company wishes to know the impact of a new drug on the health of patients, they may select the remaining patients enrolled in a pharmaceutical trial apart from the previously selected patient group as the control entity 106. It is noted that the terms ‘control entity 106’, ‘control unit’, ‘control group’, and ‘reference group’ may be used interchangeably throughout the present disclosure.

[0046] As described earlier, to ensure the accuracy of the causal inference results, the errors in the existing approach of using machine learning and forecasting algorithms have to be mitigated or eliminated.

[0047] To solve this problem, an approach for mitigating errors in a causal inference process is required. To that end, to address the above-mentioned limitation, the present disclosure describes that the server system 102 mitigates errors from the causal inferenceresults produced by AI / ML models.

[0048] In one embodiment, the environment 100 may further include a database 110 coupled with the server system 102. In an example, the server system 102 coupled with the database 110 is embodied within a central server (not shown) associated with the organization or business that wishes to conduct the causal inference process. However, in other examples, the server system 102 can be a standalone component (acting as a hub) connected to the central server. The database 110 may be incorporated in the server system 102 or maybe an individual entity connected to the server system 102 or maybe a database stored in cloud storage. In one embodiment, the database 110 may store pre-treatment time series information, post-treatment time series information, a Machine Learning (ML) model 112, and other necessary machine instructions required for implementing the various functionalities of the server system 102 such as firmware data, operating system, and the like. It is noted that the pre-treatment time series information for the target entity 104 and the control entity 106, post-treatment time series information for the target entity 104 and the control entity 106, and Machine Learning (ML) model 112 have been explained in detail later in the present disclosure. In addition, the database 110 provides a storage location for data and / or metadata obtained from various operations performed by the server system 102.

[0049] In an embodiment, for performing the causal inference process, the server system 102 is configured to access, for each of a target entity 104 and a control entity 106, pretreatment time series information and post-treatment time series information from the database. The term ‘pre-treatment time series information’ refers to temporal data collected for either the target entity 104 or the control entity 106 before the implementation of an intervention by the organization or business. In particular, the pre-treatment time series information provides a historical record of various variables or metrics (such as revenue, attachment ratio, Forty feet Equivalent Unit (FEU), Twenty feet Equivalent Unit (TEU), etc., among other relevant entityspecific metrics) before the intervention is performed. In various non-limiting examples, pretreatment time series information may include historical revenue of a region before a new product is launched, historical sales for a product before the launch of a marketing campaign, patient vital signs before the administration of medical treatment, and so on.

[0050] The term ‘post-treatment time series information’ refers to temporal data collected for either the target entity 104 or the control entity 106 after the implementation of an intervention by the organization or business to the target entity 104. In particular, the posttreatment time series information provides a historical record of various variables or metrics(such as revenue, attachment ratio, Forty feet Equivalent Unit (FEU), Twenty feet Equivalent Unit (TEU), etc., among other relevant entity- specific metrics) before the intervention is performed to the target entity 104. In various non-limiting examples, post-treatment time series information may include revenue of a region after a new product is launched, sales for a product after the launch of a marketing campaign, patient vital signs after the administration of medical treatment, and so on. Here, it is pertinent to note that the post-treatment time series information of the control entity 106 simply refers to the information collected after the intervention is performed on the target entity 104. In other words, the intervention does not impact the post-treatment time series information of the control entity 106.

[0051] Then, the server system 102 is configured to generate pre-treatment prediction time series information for the target entity 104 based, at least in part, on the pre-treatment time series information of the control entity 106. In particular, the server system 102 utilizes AI / ML models such as the ML model 112 to generate the pre-treatment prediction time series information for the target entity 104. Thereafter, the server system 102 is configured to compute a set of prediction error values by comparing the pre-treatment prediction time series information and the pre-treatment time series information of the target entity 104. It is understood that since the pre-treatment time series information of the target entity 104 is already available, it can be used to determine the prediction error values associated with the ML model 112.

[0052] Further, the server system 102 is configured to generate, by the ML model 112, post-treatment prediction time series information for the target entity 104 based, at least in part, on the pre-treatment time series information of the target entity 104 and the post-treatment time series information of the control entity 106. Furthermore, the server system 102 is configured to generate prediction range information for the target entity 104 based, at least in part, on the post-treatment prediction time series information and the set of prediction error values. Herein, prediction range information represents or indicates the impact of the intervention on the target entity 104, i.e., the causal inference results. As may be appreciated, the prediction error values help the server system 102 to compensate for or mitigate the errors within the prediction range information, thus making the causal inference results accurate. This aspect has been described in detail later in the present disclosure.

[0053] Although in FIG. 1, the server system 102 is shown to be incorporated within the environment 100, in some embodiments, the server system 102 may be external to and in communication with the environment 100, for example, via the network 108. In someexamples, the server system 102 may be implemented in third-party external servers associated with the organization or business to perform the various operations described herein.

[0054] The number and arrangement of systems, devices, and / or networks shown in FIG. 1 are provided as an example. There may be additional systems, devices, and / or networks; fewer systems, devices, and / or networks; different systems, devices, and / or networks; and / or differently arranged systems, devices, and / or networks than those shown in FIG. 1. Furthermore, two or more systems or devices shown in FIG. 1 may be implemented within a single system or device, or a single system or device is shown in FIG. 1 may be implemented as multiple, distributed systems or devices. In addition, the server system 102 should be understood to be embodied in at least one computing device in communication with the network 108, which may be specifically configured, via executable instructions, to perform steps as described herein, and / or embodied in at least one non-transitory computer-readable media.

[0055] FIG. 2 illustrates a simplified block diagram of a server system 200, in accordance with an embodiment of the present disclosure. It is noted that the server system 200 may be similar to the server system 102 of FIG. 1. In one embodiment, the server system 200 is a part of the internal server operated by an organization or a business conducting the causal inference process to determine the impact of an intervention on the target entity 104. In some embodiments, the server system 200 is embodied as a cloud-based and / or Software as a Service (SaaS) based architecture.

[0056] The server system 200 includes a computer system 202 and a database 204. It is noted that the database 204 is identical to the database 110 of FIG. 1. The computer system 202 includes at least one processor 206 (herein, referred to interchangeably as ‘processor 206’) for executing instructions, a memory 208, a communication interface 210, a user interface 212 and a storage interface 214 that communicates with each other via a bus 216.

[0057] In some embodiments, the database 204 is integrated into the computer system 202. For example, the computer system 202 may include one or more hard disk drives as the database 204. The user interface 212 is an interface, such as a Human Machine Interface (HMI) or a software application that allows users such as administrators to interact with and control the server system 200 or one or more parameters associated with the server system 200. It may be noted that the user interface 212 may be composed of several components that vary based on the complexity and purpose of the application. Examples of components of the userinterface 212 may include visual elements, controls, navigation, accessibility features, etc.

[0058] A storage interface 214 is any component capable of providing the processor 206 with access to the database 204. The storage interface 214 may include, for example, an Advanced Technology Attachment (ATA) adapter, a Serial ATA (SATA) adapter, a Small Computer System Interface (SCSI) adapter, a RAID controller, a SAN adapter, a network adapter, and / or any component providing the processor 206 with access to the database 204. In one non-limiting example, the database 204 is configured to store a machine learning (ML) model 218, time series information 220, and the like. It is noted that the ML model 218 is identical to the ML model 112 of FIG. 1.

[0059] The processor 206 includes suitable logic, circuitry, and / or interfaces to execute operations for mitigating errors in the causal inference process, and the like. Examples of the processor 206 include, but are not limited to, an Application-Specific Integrated Circuit (ASIC) processor, a Reduced Instruction Set Computing (RISC) processor, a Graphical Processing Unit (GPU), a Complex Instruction Set Computing (CISC) processor, a Field-Programmable Gate Array (FPGA), and the like.

[0060] The memory 208 includes suitable logic, circuitry, and / or interfaces to store a set of computer-readable instructions for performing the various operations described herein. Examples of the memory 208 include a random-access memory (RAM), a read-only memory (ROM), a removable storage drive, a hard disk drive (HDD), and the like. It will be apparent to a person skilled in the art that the scope of the disclosure is not limited to realizing the memory 208 in the server system 200, as described herein. In another embodiment, the memory 208 may be realized in the form of a database server or a cloud storage working in conjunction with the server system 200, without departing from the scope of the present disclosure.

[0061] The processor 206 is operatively coupled to the communication interface 210, such that the processor 206 is capable of communicating with a remote device (i.e., to / from a remote device 222) such as third-party servers, or with the target entity 104, the control entity 106, or communicating with any other entity connected to the network 108 (as shown in FIG. 1).

[0062] It is noted that the server system 200 as illustrated and hereinafter described is merely illustrative of an apparatus that could benefit from embodiments of the present disclosure and, therefore, should not be taken to limit the scope of the present disclosure. It is noted that the server system 200 may include fewer or more components than those depictedin FIG. 2.

[0063] In one implementation, the processor 206 includes a time series generation module 224, a prediction module 226, and an error computation module 228. It should be noted that components, described herein, such as the time series generation module 224, the prediction module 226, and the error computation module 228 can be configured in a variety of ways, including electronic circuitries, digital arithmetic, and logic blocks, and memory systems in combination with software, firmware, and embedded technologies.

[0064] In an embodiment, the time series generation module 224 includes suitable logic and / or interfaces for generating the time series information 220 for the target entity 104 and the control entity 106. The time series information 220 refers to temporal data related to the various entities, i.e., the target entity 104 and the control entity 106. The temporal data may include data samples associated with various categories such as revenue, sales, FEU, TEU, attachment ratio, operation time, medical records, user adoption rate, user complaints, consumer disputes, customer engagement metrics (such as feedback, likes, and so on), and so on. The generation of time series information 220 may include various techniques such as data collection, indexing historical data records, data modeling, and so on. It is noted that these techniques are well-known in the art and are not explained herein for the sake of brevity. In various examples, the time series information 220 further includes pre-treatment time series information of the target entity 104, pre-treatment time series information of the control entity 106, post-treatment time series information of the control entity 106, and post-treatment time series information of target entity 104.

[0065] In an example, the pre-treatment time series information includes temporal data collected for either the target entity 104 or the control entity 106 before the implementation of an intervention by the organization or business. In particular, the pre-treatment time series information provides a historical record of various variables or metrics before the intervention is performed. In various non-limiting examples, pre-treatment time series information may include historical revenue of a region before a new product is launched, historical sales for a product before the launch of a marketing campaign, patient vital signs before the administration of medical treatment, and so on. In an example, the post-treatment time series information includes temporal data collected for either the target entity 104 or the control entity 106 after the implementation of an intervention by the organization or business to the target entity 104. In particular, the post-treatment time series information provides a historical record of various variables or metrics before the intervention is performed on the target entity 104. Invarious non-limiting examples, post-treatment time series information may include revenue of a region after a new product is launched, sales for a product after the launch of a marketing campaign, patient vital signs after the administration of medical treatment, and so on. As described earlier, it is noted that the post-treatment time series information of the control entity 106 simply refers to the information collected after the intervention is performed on the target entity 104. Upon generation of the time series information 220, the time series generation module 224 is configured to store the time series information 220 in the database 204.

[0066] In another embodiment, the time series generation module 224 is communicably coupled to the prediction module 226 and is configured to transmit or share the time series information 220 with the prediction module 226.

[0067] In an embodiment, the prediction module 226 includes suitable logic and / or interfaces for accessing the time series information 220 from the database 204. More specifically, the prediction module 226 is configured to access, for each of the target entity 104 and the control entity 106, pre-treatment time series information and post-treatment time series information from the database 204. Then, the prediction module 226 is configured to generate pre-treatment prediction time series information for the target entity 104 based, at least in part, on the pre-treatment time series information of the control entity 106. In various examples, the prediction module 226 is configured to utilize AI / ML models such as the ML model 218 to generate the pre-treatment prediction time series information for the target entity 104. In a nonlimiting implementation, the ML model 218 may be a regression ML model. Examples of ML model 218 may include a Linear Regression model, a Ridge Regression model, a Lasso Regression model, a Support Vector Regression (SVR) model, a Decision Tree Regression model, a Random Forest Regression model, a Gradient Boosting Regression model, an extreme Gradient Boost (XGBoost) regression model, and so on.

[0068] In another embodiment, the prediction module 226 is configured to train the ML model 218 based, at least in part, on the pre-treatment time series information of the target entity 104 and the pre-treatment time series information of the control entity 106. In particular, the ML model 218 is trained by considering the pre-treatment time series information of the control entity 106 as the input feature. The task of the model is to predict the outcomes in the pre-treatment time series information of the target entity 104. In other words, the ML model 218 is trained to learn the correlation and the underlying relationship between the various metrics of the control entity 106 and the target entity 104. This aspect has been described in detail later in the present disclosure.

[0069] In another embodiment, the prediction module 226 is configured to generate post-treatment prediction time series information for the target entity 104 based, at least in part, on the pre-treatment time series information of the target entity 104 and the post-treatment time series information of the control entity 106. In various examples, the prediction module 226 is configured to utilize AI / ML models such as the ML model 218 to generate the posttreatment prediction time series information for the target entity 104. In particular, the ML model 218 treats the post-treatment time series information of the control entity 106 as the input feature to predict the outcomes in the post-treatment time series information for the target entity 104. As may be understood, the ML model 218 is trained such that it understands the correlation between the metrics of the target entity 104 and the control entity 106. Therefore, the ML model 218 is capable of predicting the outcomes that would occur in the post-treatment time series information for the target entity 104 if no intervention takes place.

[0070] In another embodiment, the prediction module 226 is communicably coupled to the error computation module 228 and is configured to transmit or share pre-treatment prediction time series information and the post-treatment prediction time series information of the target entity 104 with the error computation module 228.

[0071] In an embodiment, the error computation module 228 includes suitable logic and / or interfaces for computing a set of prediction error values by comparing the pre-treatment prediction time series information and the pre-treatment time series information of the target entity 104.

[0072] In another embodiment, the error computation module 228 is configured to generate prediction range information for the target entity 104 based, at least in part, on the post-treatment prediction time series information and the set of prediction error values. The prediction range information for the target entity 104 indicates the impact of an intervention on the target entity 104. As may be understood, the post-treatment prediction time series information includes metrics corresponding to a scenario where no intervention has taken place whereas the post-treatment time series information of the target entity 104 includes the actual outcomes when the intervention has taken place. Therefore, there will exist a difference between the outcome predictions and the actual outcomes, this difference signifies the impact of the intervention. However, this difference between the outcome predictions and the actual outcomes also suffers from inaccuracies due to the inherent errors present in the predictions of the ML model 218. The set of prediction error values generated earlier, allows the error computation module 228 to compensate or mitigate the inaccuracies in the difference betweenthe outcome predictions and the actual outcomes. This in turn improves the causal impact results.

[0073] In particular, for generating the prediction range information, the error computation module 228 is configured to perform a set of operations iteratively for predefined iterations. Herein, the value of the predefined iterations may be defined by an administrator (not shown) of the server system 200. The set of operations may include computing a distribution of Mean Based Error (MBE) value based, at least in part, on the set of prediction error values. In various non-limiting examples, the MBE may refer to any or at least one of a Mean Error (ME), a Mean Absolute Error (MAE), a Mean Squared Error (MSE), a Root Mean Squared Error (RMSE), and so on.

[0074] Then, the set of operations may include randomly selecting a test value from the distribution of MBE values. Thereafter, the set of operations may include computing at least one of an error increment value and an error decrement value for each outcome prediction in the post-treatment prediction time series information based, at least in part, on the posttreatment prediction time series information and the test value. For instance, the error increment value is computed by adding the predicted outcome value with the selected MBE value. Similarly, the error decrement value is computed by subtracting the selected MBE value from the predicted outcome value.

[0075] Further, the set of operations may include computing a percentage change value for each outcome prediction based, at least in part, on an actual outcome from the posttreatment time series information of the target entity and at least one of the error increment value and the error decrement value computed for each outcome prediction in the posttreatment prediction time series information. In other words, either or both the error increment value and the error decrement value can be used to compute the percentage change value for each outcome prediction. This aspect allows for flexibility in the computation process. This aspect has been described in detail later in the present disclosure.

[0076] Further, the error computation module 228 is configured to compute a mean percentage change value based, at least in part, on the percentage change value computed from each iteration of the predefined iterations. In some instances, the error computation module 228 is configured to store the mean percentage change value as the prediction range information in the database 204.

[0077] Furthermore, the error computation module 228 is configured to compute aconfidence interval for the ML model based, at least in part, on a confidence score of the ML model 218. The confidence score refers to the level of certainty or reliability associated with the predictions of the ML model 218. The process for computing the confidence score is described later in the present disclosure. As may be understood, the range of the percentage change values can be wide. Therefore, to have the optimal impact numbers the range of the percentage change values has to be restricted. To achieve this, a prominent range in which most of the percentage change values lie has to be selected. Due to this, the confidence level of the trained ML model (such as ML model 218) is used to determine the confidence interval.

[0078] Thereafter, the error computation module 228 is configured to determine a model impact factor for the ML model 218 based, at least in part, on the confidence interval and the prediction range information. In particular, for determining the model impact factor, the error computation module 228 is configured to select a set of percentage change values within the confidence interval from the percentage change value computed from each operation. Then, the error computation module 228 is configured to set the model impact factor based, at least in part, on the set of percentage change values.

[0079] FIG. 3 illustrates a schematic representation 300 of the process flow for determining the model impact factor, in accordance with an embodiment of the present disclosure. In a particular non-limiting implementation, the various operations of the present disclosure can be divided into three stages, i.e., the model operation stage, the operation research stage, and the model impact determination stage.

[0080] In the model operation stage, at step 302, the server system 200 accesses the post-treatment time series information of the target entity 104 and the control entity 106.

[0081] At step 304, the server system 200 trains the ML model 218. In particular, the ML model 218 is trained using the pre-treatment time series information of the control unit 106 as an input feature. The goal of the training process is to predict the pre-treatment time series data of the target entity 104.

[0082] In one non-limiting implementation, the ML model 218 may be a regression algorithm-based ML model such as, but not limited to, linear regression, decision trees, random forests, gradient boosting, and so on. The selection of the ML model 218 may be done based on the type of time series information being used for performing the causal inference process. At first, the server system 200 generates relevant pre-treatment features using the pre-treatment time series information of the control entity 106 and the target entity 104. It is noted thatvarious feature generation techniques such as, but not limited to, time -based feature generation, binning, aggregated variable generation, dimensionality reduction, one hot encoding, frequency encoding, and so on, may be used for generating the described features. Since these feature generation techniques are well known in the art, they are not described here for the sake of brevity. In some instances, the pre-treatment time series information for both entities may be preprocessed by the server system 200 using known techniques to handle or address missing values, outliers, scaling, etc., among other data quality issues.

[0083] Further, the server system 200 utilizes the relevant pre-treatment features generated from the pre-treatment time series information of the control entity 106 as input features for training the regression ML model. As may be understood, the relevant features are selected such that they are predictive of the outcome for the pre-treatment time series information of the target entity 104. In particular, the model training process includes using the pre-treatment time series information of the control entity 106 as the training set while using the pre-treatment time series information of the target entity 104 as the validation set. In some instances, the pre-treatment time series information of both entities may be combined and later split into training and validation sets for performing the training process as well.

[0084] Further, the performance of the trained regression ML model may be evaluated using the validation set. In some instances, assessment metrics such as, but not limited to, mean squared error (MSE), root mean squared error (RMSE), or the coefficient of determination (R- squared) may be used to measure the model's accuracy in predicting the pre-treatment data of the target entity 104. Furthermore, the hyper-parameters of the regression ML model may be tuned (or fine-tuned), if required (based on the evaluation results), to optimize the performance of the model on the validation set. In some instances, hyper-parameters such as, but not limited to, regularization strength, tree depth, learning rate, and so on may be adjusted to improve the model's predictive accuracy. Once, the ML model 218 is trained, then at step 306, the server system 200 uses the ML model 218 to generate post-treatment prediction time series information for the target entity 104 using the post-treatment time series of the control entity 106 as the input feature.

[0085] Now, in the model operation stage, at step 308, a set of operations is performed iteratively for predefined iterations. In one exemplary implementation, the iterations, i.e., i, are initialized at i=0 and the predefined iterations are set at 1000. In other words, the set of operations (i.e., step 310 to step 316) are performed iteratively for 1000 iterations.

[0086] At step 310, the server system 200 computes a Mean Based Error (MBE) such as, but not limited to, Mean Error, MSE, RMSE, MAE, and the like using each data sample or outcome from the post-treatment prediction time series of the target entity 104 (i.e., the predicted outcomes) and the post-treatment time series of the target entity 104 (i.e., the actual outcomes). To that end, a distribution of MBEs is computed or generated by the server system 200 using the aforementioned time series information of the target entity 104. Further, the server system 200 randomly selects an MBE value from the distribution of MBEs, this selected MBE value may be called a test value.

[0087] At step 312, the server system 200 iterates over the outcome predictions in the post-treatment prediction time series information of the target entity 104. In some instances, a simulation with 1000 iterations may be performed to select random values from the distribution of mean errors and then computations are performed on the selected random values along with the post-treatment prediction time series information.

[0088] At step 314, the server system 200 computes an error increment value and an error decrement value for each outcome prediction in the post-treatment prediction time series information of the target entity 104. For instance, the error increment value is computed by adding the predicted outcome value with the randomly selected MBE value (i.e., the test value). Similarly, the error decrement value is computed by subtracting the randomly selected MBE value (i.e., the test value) from the predicted outcome value.

[0089] At 316, the server system 200 computes a percentage change value for each outcome prediction. In particular, the server system 200 selects a random test value between the error decrement value and the error increment value for each outcome prediction. Further, the server system 200 compares it to the actual outcome from the post-treatment time series information of the target entity 104.

[0090] At 318, the server system 200 stores the percentage change value generated during each iteration into a list (called, a percent change list). The list may then be stored within the database 204.

[0091] Now, in the model impact determination stage, at step 320, the server system 200 computes the confidence interval of the ML model 218 using the confidence score of the model. It is noted that this aspect has been described in detail later in the present disclosure.

[0092] At step 322, the server system 200 selects the percentage change value falling within the calculated confidence interval from the percent change list. For instance, if theconfidence interval is 20% then, the server system 200 selects the mean of the percentage change value and calculates 10% to the left and 10% to the right with respect to the mean of the percentage change value.

[0093] At step 324, the server system 200 sets the model impact factor for the ML model 218 based, at least in part, on the confidence interval and the percent change list. In particular, the server system 200 computes a mean percentage change value using the percent change list and stores it as the prediction range information. Later, the prediction range information and the confidence interval are used to set the model impact factor.

[0094] FIG. 4 illustrates a representation of various graphs associated with an exemplary implementation of the error mitigation process during causal inferencing using an ML model, in accordance with various embodiments of the present disclosure.

[0095] In an exemplary implementation, the server system 200 may be configured to perform the causal inference process for detecting the impact of a new product or service on the revenue of an organization. For performing the causal inference process, at first, a target entity such as target entity 104 is selected. The target entity 104 may correspond to a region or marketplace where the new product or service must be implemented. In an example, the organization may wish to implement an express delivery service for providing expedited shipping to their customers in the United States of America (USA). In this scenario, the marketplace of the USA is said to be the target entity 104. Now, for performing the causal inference process, a control entity such as control entity 106 has to be selected. The control entity 106 may correspond to a region or marketplace that is similar to the target entity 104. More specifically, the control entity 106 is selected such that it is a close match to the market conditions such as revenue, market size, customer pool, and so on, of the target entity 104. Referring to the previous example, it may be determined after comparative analysis that the marketplace in Canada has exceptional similarity to the marketplace in the USA. In such a scenario, the Canadian marketplace may be selected as the control entity 106.

[0096] In another exemplary implementation, the server system 200 may be configured to perform the causal inference process for implementing a pricing strategy of a shipping and logistics organization for some test corridors to increase the number of bookings. For performing this causal inference process, the test corridors for which the pricing strategy is implemented can be set as the target entity 104. Further, the control entity 106 is selected, called the control corridor, based on the pattern of bookings correlated with the test corridorsbefore implementing the new pricing strategy.

[0097] Further, the server system 200 is configured to train an ML model such as ML model 218 for performing the causal inference process. In particular, the ML model 218 may be trained to determine the impact of the express delivery service on the revenue of the organization for the target entity 104, i.e., the marketplace of the USA or the bookings of the test corridor. The server system 200 can train the ML model 218 with the pre-treatment time series information indicating the revenue information (i.e., daily, weekly, or monthly revenue information) of the control entity 106, i.e., the marketplace of Canada or the bookings of the control corridor as the input feature. The target of the ML model 218 is set to the pre-treatment time series information of the target entity 104. In a non-limiting implementation, the training process may be a supervised learning process where historical time series information is split into a training and a validation set. The training set is used to train the ML model 218 while the validation set is used to validate the predictions of the ML model 218. This learning or training is performed to ensure that the ML model 218 learns how the revenue of the target entity 104 is correlated to the control entity 106 before the new service is implemented. The server system 200 is configured to compute a set of prediction error values by comparing the actual pre-treatment time series information of the target entity 104 present in the validation set with the predicted pre-treatment time series information for the target entity 104 by the ML model 218.

[0098] Then, the ML model 218 is configured to predict the post-treatment time series information of the target entity 104 using the post-treatment time series information of the control entity 106 as the input feature. As described earlier, the post-treatment time series information of the control entity 106 helps the ML model 218 to understand what would have been the organic growth of revenue of the target entity 104 if the new service had not been implemented. Therefore, to ascertain the impact of the new service, i.e., express delivery service or the pricing strategy, the post-treatment time series information for the target entity 104 is compared with the post-treatment prediction time series information for the target entity 104. However, due to inaccuracies associated with the ML model 218, simply comparing the predicted post-treatment time series information with the actual post-treatment time series information will not yield accurate results. In other words, the computed causal impact of the new service or the pricing strategy will not be accurate.

[0099] To address such inaccuracies, the server system 200 computes a distribution of mean-based error (MBE) values such as a Mean Absolute Error (MAE) value for the ML model218 based on the set of prediction error values. As depicted, graph 400 represents the distribution of the MAE values of the ML model 218 trained using the pre-treatment time series information. It is noted that the results illustrated in Graph 400 are generated based on exemplary revenue data and the results may vary based on the selected data samples.

[0100] For the sake of explanation, it is assumed that the actual post-treatment time series information of the revenue for the target entity 104 may be [$2000, $3500, $4700, $3900, $4800] while the post-treatment prediction time series information for the target entity may be [$2200, $2900, $3600, $3400, $4500]. In this scenario, the server system 200 is configured to run a plurality of simulations / iterations (for instance, 1000 simulations / iterations) to randomly select test values from the distribution of MBE. Then, the server system 200 is configured to compute an error increment value and an error decrement value for each outcome prediction in the post-treatment prediction time series information using the selected random test value. For instance, if during the first simulation / iteration, the picked MBE is 70, then, the posttreatment prediction time series information becomes [$2270, $2970, $3670, $3470, $4570]. In another instance, if during the second simulation / iteration, the picked MBE is -110, then, the post-treatment prediction time series information becomes [$2090, $2790, $3490, $3290, $4390]. In other words, during each iteration, either the error increment value or the error decrement value can be computed for each outcome prediction in the post-treatment prediction time series information.

[0101] For each simulation / iteration, the server system 200 is configured to compute the percentage change value for each outcome by subtracting the sum of post-treatment prediction time series information and the sum of actual post-treatment time series information for the target entity 104. For instance, for the first iteration, the sum (post-treatment prediction time series information) is $16950, and the sum (actual post-treatment time series information) is $18900, so the percentage change value for the first iteration is 11.5%. In another instance, for the second iteration, the sum (post-treatment prediction time series information) is $16050, and the sum (actual post-treatment time series information) is $18900, so the percentage change value for the first iteration is 17.75%.

[0102] Now since this process is repeated for 1000 simulations / iterations, the server system 200 would generate 1000 percent change values. For instance, the 1000 percent change values may be [13.77, 21.0, 20.52, 12.29, 13.35, 11.81, 17.84, 22.45, 19.63, 20.33, 15.15, 20.79, 14.79, 19.0, 21.2, 12.34, 15.38, 16.0, 20.49, 21.11, 18.03, 14.79, 13.63, 14.98, 19.84, 15.56, 22.76, 19.02, 14.76, 20.9, 13.09, 24.09, 17.45, 10.57, 23.49,14.63, 14.22, 25.97, 10.34,19.09, 14.84, 22.04, 17.27, 13.25, 11.3, 19.97, 93, 19.05, 21.53, 14.65, 23.31, 12.7, 18.88, 18.95, 18.76, 23.57, 15.27, 15.91, 14.39, 17.42, 11.57, 15.44, 22.66, 15.57, 18.29, 15.57, . ]

[0103] For capturing the overall percent change, the server system 200 may compute a mean percentage change value using the percent change value from each iteration. Furthermore, the server system 200 is configured to compute the confidence interval based on the confidence score of the ML model 218, i.e., the model metric R2 score. In a non-limiting implementation, the confidence interval may be computed using the following equation:Confidence Interval (CI) = 100 - (R2 * 100) . . . Eqn.1

[0104] For instance, if the R2 score of the ML model 218 is 0.8, then the CI of the ML model 218 becomes 20%. Since, the confidence interval refers to an interval for which the ML model 218 will have high confidence in its outcomes / predictions thus, selecting the percentage change values within this interval will lead to improved causal inference results. Therefore, the server system 200 is configured to select a set of percentage change values within the confidence interval from the percentage change value computed during each iteration and the mean percentage change value using the set of percentage change values. As depicted, Graph 450 represents the distribution of the percentage change values and the confidence interval of the ML model 218. In the illustrated example, the mean percentage change value is computed to be approximately 26% to 28%. Then, the mean percentage change value is stored as the prediction range information for the target entity. As would be appreciated, the mean percentage change value represents the impact of the new service on the revenue of the organization. In other words, the impact of the express delivery service on the revenue of the organization for the marketplace of the USA is approximately 26% to 28%. Similarly, the impact of the new pricing strategy on the number of bookings may be computed as well.

[0105] FIG. 5 illustrates a flow diagram of a method 500 of operating the server system 200 for mitigating errors in a causal inference process, in accordance with an embodiment of the present disclosure. The method 500 depicted in the flow diagram may be executed by, for example, the server system 200. The sequence of operations of the method 500 may not be necessarily executed in the same order as they are presented. Further, one or more operations may be grouped and performed in the form of a single step, or one operation may have several sub-steps that may be performed in parallel or in a sequential manner. Operations of the method 500, and combinations of operations in the method 500 may be implemented by, for example,hardware, firmware, a processor, circuitry, and / or a different device associated with the execution of software that includes one or more computer program instructions. The plurality of operations is depicted in the process flow of the method 500. The process flow starts at operation 502.

[0106] At 502, the method 500 includes accessing, for each of a target entity such as target entity 104 and a control entity such as control entity 106, pre-treatment time series information and post-treatment time series information from a database such as database 204.

[0107] At 504, the method 500 includes generating, by a Machine Learning (ML) model such as ML model 218, pre-treatment prediction time series information for the target entity 104 based, at least in part, on the pre-treatment time series information of the control entity 106.

[0108] At 506, the method 500 includes computing a set of prediction error values by comparing the pre-treatment prediction time series information and the pre-treatment time series information of the target entity 104.

[0109] At 508, the method 500 includes generating, by the ML model 218, posttreatment prediction time series information for the target entity 104 based, at least in part, on the pre-treatment time series information of the target entity 104 and the post-treatment time series information of the control entity 106.

[0110] At 510, the method 500 includes generating prediction range information for the target entity 104 based, at least in part, on the post-treatment prediction time series information and the set of prediction error values.

[0111] The disclosed method with reference to FIG. 5, or one or more operations of the server system 200 may be implemented using software including computer-executable instructions stored on one or more computer-readable media (e.g., non-transitory computer- readable media, such as one or more optical media discs, volatile memory components (e.g., DRAM or SRAM), or nonvolatile memory or storage components (e.g., hard drives or solid- state nonvolatile memory components, such as Flash memory components) and executed on a computer (e.g., any suitable computer, such as a laptop computer, netbook, Web book, tablet computing device, smartphone, or other mobile computing devices). Such software may be executed, for example, on a single local computer or in a network environment (e.g., via the Internet, a wide-area network, a local-area network, a remote web-based server, a client-server network (such as a cloud computing network), or other such networks) using one or more1 network computers.

[0112] Additionally, any of the intermediate or final data created and used during the implementation of the disclosed methods or systems may also be stored on one or more computer-readable media (e.g., non-transitory computer-readable media) and are considered to be within the scope of the disclosed technology. Furthermore, any of the software-based embodiments may be uploaded, downloaded, or remotely accessed through a suitable communication means. Such suitable communication means include, for example, the Internet, the World Wide Web (WWW), an intranet, software applications, cable (including fiber optic cable), magnetic communications, electromagnetic communications (including RF, microwave, and infrared communications), electronic communications, or other such communication means.

[0113] Although the invention has been described with reference to specific exemplary embodiments, it is noted that various modifications and changes may be made to these embodiments without departing from the broad scope of the invention. For example, the various operations, blocks, etc., described herein may be enabled and operated using hardware circuitry (for example, Complementary Metal Oxide Semiconductor (CMOS) based logic circuitry), firmware, software, and / or any combination of hardware, firmware, and / or software (for example, embodied in a machine-readable medium). For example, the apparatuses and methods may be embodied using transistors, logic gates, and electrical circuits (for example, Application Specific Integrated Circuit (ASIC) circuitry and / or Digital Signal Processor (DSP) circuitry).

[0114] Particularly, the server system 200 and its various components may be enabled using software and / or using transistors, logic gates, and electrical circuits (for example, integrated circuit circuitry such as ASIC circuitry). Various embodiments of the invention may include one or more computer programs stored or otherwise embodied on a computer-readable medium, wherein the computer programs are configured to cause the processor or the computer to perform one or more operations. A computer-readable medium storing, embodying, or encoded with a computer program, or similar language, may be embodied as a tangible data storage device storing one or more software programs that are configured to cause the processor or computer to perform one or more operations. Such operations may be, for example, any of the steps or operations described herein. In some embodiments, the computer programs may be stored and provided to a computer using any type of non-transitory computer- readable media. Non-transitory computer-readable media includes any type of tangible storagemedia.

[0115] Examples of non-transitory computer-readable media include magnetic storage media (such as floppy disks, magnetic tapes, hard disk drives, etc.), optical magnetic storage media (e.g. magneto-optical disks), Compact Disc Read-Only Memory (CD-ROM), Compact Disc Recordable (CD-R), compact disc rewritable (CD-R / W ), Digital Versatile Disc (DVD), BLU-RAY® Disc (BD), and semiconductor memories (such as mask ROM, programmable ROM (PROM), (erasable PROM ), flash memory, Random Access Memory (RAM), etc.). Additionally, a tangible data storage device may be embodied as one or more volatile memory devices, one or more non-volatile memory devices, and / or a combination of one or more volatile memory devices and non-volatile memory devices. In some embodiments, the computer programs may be provided to a computer using any type of transitory computer- readable media. Examples of transitory computer-readable media include electric signals, optical signals, and electromagnetic waves. Transitory computer-readable media can provide the program to a computer via a wired communication line (e.g., electric wires, and optical fibers) or a wireless communication line.

[0116] Various embodiments of the invention, as discussed above, may be practiced with steps and / or operations in a different order, and / or with hardware elements in configurations, which are different than those which, are disclosed. Therefore, although the invention has been described based on these exemplary embodiments, it is noted that certain modifications, variations, and alternative constructions may be apparent and well within the scope of the invention.

[0117] Although various exemplary embodiments of the invention are described herein in a language specific to structural features and / or methodological acts, the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as exemplary forms of implementing the claims.

Claims

CLAIMS1. A computer- implemented method, comprising: accessing, for each of a target entity and a control entity, pre-treatment time series information and post-treatment time series information from a database; generating, by a Machine Learning (ML) model, pre-treatment prediction time series information for the target entity based, at least in part, on the pre-treatment time series information of the control entity; computing a set of prediction error values by comparing the pre-treatment prediction time series information and the pre-treatment time series information of the target entity; generating, by the ML model, post-treatment prediction time series information for the target entity based, at least in part, on the pre-treatment time series information of the target entity and the post-treatment time series information of the control entity; and generating prediction range information for the target entity based, at least in part, on the post-treatment prediction time series information and the set of prediction error values.

2. The computer- implemented method as claimed in claim 1, wherein generating the prediction range information, comprises: performing a set of operations iteratively for predefined iterations, the set of operations comprising: computing a distribution of Mean Based Error (MBE) values based, at least in part, on the set of prediction error values; randomly selecting a test value from the distribution of MBE values; computing at least one of an error increment value and an error decrement value for each outcome prediction in the post-treatment prediction time series information based, at least in part, on the post-treatment prediction time series information and the test value; and computing a percentage change value for each outcome prediction based, at least in part, on an actual outcome from the post-treatment time series information of the target entity and at least one of the error increment value and the errordecrement value computed for each outcome prediction in the post-treatment prediction time series information.

3. The computer- implemented method as claimed in claim 2, further comprising: computing a mean percentage change value based, at least in part, on the percentage change value computed from each iteration of the predefined iterations; and storing the mean percentage change value as the prediction range information.

4. The computer-implemented method as claimed in claim 1, further comprising: computing a confidence interval for the ML model based, at least in part, on a confidence score of the ML model.

5. The computer-implemented method as claimed in claim 4, further comprising: determining a model impact factor for the ML model based, at least in part, on the confidence interval and the prediction range information.

6. The computer-implemented method as claimed in any of claims 1 to 5, wherein determining the model impact factor, comprises: selecting a set of percentage change values within the confidence interval from the percentage change value computed from each operation; and setting the model impact factor based, at least in part, on the set of percentage change values.

7. The computer-implemented method as claimed in claim 1, further comprising: training the ML model based, at least in part, on the pre-treatment time series information of the target entity and the pre-treatment time series information of the control entity.

8. A server system, comprising: a communication interface; a memory configured to store instructions; and a processor in communication with the communication interface and the memory, the processor configured to execute the instructions stored in the memory and thereby cause the server system to perform at least in part to:access, for each of a target entity and a control entity, pre-treatment time series information and post-treatment time series information from a database; generate, by a Machine Learning (ML) model, pre-treatment prediction time series information for the target entity based, at least in part, on the pre-treatment time series information of the control entity; compute a set of prediction error values by comparing the pre-treatment prediction time series information and the pre-treatment time series information of the target entity; generate, by the ML model, post-treatment prediction time series information for the target entity based, at least in part, on the pre-treatment time series information of the target entity and the post-treatment time series information of the control entity; and generate prediction range information for the target entity based, at least in part, on the post-treatment prediction time series information and the set of prediction error values.

9. The system as claimed in claim 8, wherein to generate the prediction range information, the server system is caused, at least in part, to: perform a set of operations iteratively for predefined iterations, the set of operations comprising: computing a distribution of Mean Based Error (MBE) values based, at least in part, on the set of prediction error values; randomly selecting a test value from the distribution of MBE values; computing at least one of an error increment value and an error decrement value for each outcome prediction in the post-treatment prediction time series information based, at least in part, on the post-treatment prediction time series information and the test value; and computing a percentage change value for each outcome prediction based, at least in part, on an actual outcome from the post-treatment time series information of the target entity and at least one of the error increment value and the error decrement value computed for each outcome prediction in the post-treatment prediction time series information.

10. The system as claimed in claim 9, wherein the server system is further caused, at least in part, to: compute a mean percentage change value based, at least in part, on the percentage change value computed from each iteration of the predefined iterations; and store the mean percentage change value as the prediction range information.

11. The system as claimed in claim 8, wherein the server system is further caused, at least in part, to: compute a confidence interval for the ML model based, at least in part, on a confidence score of the ML model.

12. The system as claimed in claim 11, wherein the server system is further caused, at least in part, to: determine a model impact factor for the ML model based, at least in part, on the confidence interval and the prediction range information.

13. The system as claimed in any of claims 8 to 12, wherein to determine the model impact factor, the server system is caused, at least in part, to: select a set of percentage change values within the confidence interval from the percentage change value computed from each operation; and set the model impact factor based, at least in part, on the set of percentage change values.

14. The system as claimed in claim 8, wherein the server system is further caused, at least in part, to: train the ML model based, at least in part, on the pre-treatment time series information of the target entity and the pre-treatment time series information of the control entity.

15. A non-transitory computer-readable storage medium comprising computer-executable instructions that, when executed by at least a processor of a server system, cause the server system to perform operations according to any one of the previous claims.

Images