Deep causal learning for e-commerce content generation and optimization

Abstract

A system for optimizing business objectives for e-commerce content may include a memory and a processor coupled to the memory. The processor may receive one or more hypotheses for multivariate comparison of content. The content may be provided to a user of the e-commerce system. The processor may repeatedly generate self-organizing experimental units (SOEUs) based on one or more hypotheses. The processor may inject the SOEU into the online system to generate quantitative reasoning regarding the content. The processor may identify at least one confidence interval within the quantization reasoning in response to the injection SOEU. The processor may iteratively modify the SOEU based on the at least one confidence interval to identify at least one causal interaction of the e-commerce content within the system. Other methods and apparatus are described.

Description

[0001] This application is a divisional application of patent application (international filing date August 26, 2019, application number 201980070005.2, invention title "Deep Causal Learning for E-commerce Content Generation and Optimization"). Technical Field

[0002] This invention relates to determining the effectiveness of e-commerce content and optimizing content allocation to enhance business objectives, and more specifically, to performing these operations simultaneously. Background Technology

[0003] E-commerce is a rapidly growing retail channel. Suppliers can tailor their product marketing to consumers by changing the content presented to them as they browse and transact on e-commerce websites. By altering this content, suppliers can influence consumer responses and, by extension, gain insights into how they can impact transactions for corresponding services or products. Effectively managing the presented content, understanding consumer reactions to it, and continuously optimizing it are key elements for suppliers to maximize their e-commerce objectives—i.e., increasing sales and / or profits. Summary of the Invention

[0004] This article discloses systems, devices, software, and methods for optimizing e-commerce content to maximize business objectives.

[0005] In one embodiment, a system for optimizing business objectives of e-commerce content is described, the system having a memory and a processor coupled to the memory, wherein the processor is configured to: (a) receive one or more hypotheses for randomized multivariate comparisons of content to be provided to users of the system; (b) repeatedly generate self-organizing units of experiment (SOEUs) based on the one or more hypotheses; (c) inject SOEUs into the system to generate quantitative inference about the content; (d) in response to injecting SOEUs, identify at least one confidence interval within the quantitative inference; and (e) iteratively modify the SOEUs based on the at least one confidence interval to identify at least one causal interaction of the e-commerce content within the system.

[0006] In another embodiment, a computer-implemented method for optimizing e-commerce content for business objectives is described, comprising: receiving one or more hypotheses for multivariate comparisons of content, including content to be provided to users of the system; repeatedly generating self-organizing experimental units (SOEUs) based on the one or more hypotheses; injecting SOEUs into the system to generate quantitative inference about the content; identifying at least one confidence interval within the quantitative inference in response to the injection of SOEUs; and iteratively modifying the SOEUs based on the at least one confidence interval to identify at least one causal interaction of the e-commerce content within the system.

[0007] These and other aspects will become apparent in the following detailed description. However, in no event should this broad summary be construed as a limitation on the subject matter protected by the claims, whether such subject matter is presented in the claims of the original filing, in the claims of the amended filing, or otherwise in the course of the application. Attached Figure Description

[0008] In accompanying drawings that are not necessarily drawn to scale, similar numbers may describe similar parts in different views. Similar reference numerals with different letter suffixes may indicate different instances of similar parts. Some embodiments are shown in the drawings by way of example rather than limitation, wherein:

[0009] Figure 1 This is a diagram illustrating systems for e-commerce content generation and optimization based on various examples;

[0010] Figure 2 It is a block diagram of the software modules and core processes of a system based on various examples; and

[0011] Figure 3 It is a flowchart of computer-implemented methods for generating and optimizing e-commerce content, based on various examples. Detailed Implementation

[0012] For the terminology listed below, unless a different definition is provided elsewhere in the claims or description, these definitions shall apply throughout the application.

[0013] Glossary

[0014] Certain terms used throughout the specification and claims, while mostly well-known, may require some interpretation. It should be understood that, as used in this specification and the appended embodiments:

[0015] Unless otherwise expressly specified, the singular forms “a” and “the” include multiple referents. As used in this specification and the accompanying embodiments, unless otherwise expressly specified, the term “or” is generally used in its meaning including “and / or”.

[0016] The terms "independent variable (IV)" and "external variable (EV)" are commonly used to refer to variables manipulated by the user and variables not controlled by the user, respectively. Independent variables can be discrete or continuous. External variables are typically continuous.

[0017] The term "level," used in conjunction with experimental units, is typically used to describe the state of a characteristic or option of an independent variable (IV). For example, if two levels are defined for a characteristic, the first level means that the characteristic is active in the experimental unit, and the second level would be defined as its inactivity. Additional states or conditions of an IV can subsequently be defined as either active or inactive.

[0018] The term "repetitively" is often used to describe something that occurs continuously, with or without a specific sequence. For example, a process may follow a set of steps continuously or iteratively in a specified order (e.g., if a process contains steps 1-5, then the process performs steps 1, 2, 3, 4, 5 in that order or in reverse order - steps 5, 4, 3, 2, 1), or it may follow these steps randomly or non-sequentially (e.g., 1, 3, 5, 4, 2, or any combination thereof).

[0019] "Exchangeable" or "commutability" is often deployed as a result that is statistically equivalent to the content distribution outcome.

[0020] The term "causal effect" or "causal relationship / interaction / reasoning" is a positive or negative indication of the existence, non-existence, change, or modification of specific content that has an impact on other content, and its ability to influence user interaction (i.e., purchasing a specific product).

[0021] "Positive" is usually defined as the probability of occurrence or selection that is not less than zero or non-zero.

[0022] The term "confounding factors" includes the Hawthorne effect, order / lag effects, demand characteristics, external variables, and / or any other factors that may change systematically with the level of the independent variable.

[0023] The numerical range limited by the endpoints includes all values ​​contained in that range (e.g., 1 to 5, including 1, 1.5, 2, 2.75, 3, 3.8, 4 and 5).

[0024] Unless otherwise specified, all figures for expressions or measurements of components, properties, etc., used in this specification and embodiments should in all cases be understood to be modified by the term "about". Therefore, unless stated to the contrary, the numerical parameters shown in the foregoing specification and the appended list of embodiments may vary according to the desired properties sought by those skilled in the art using the teachings of this disclosure. At a minimum, and without attempting to limit the application of the doctrine of equivalence to the embodiments protected by the claims, each numerical parameter should be interpreted at least according to the number of significant digits reported and by applying customary rounding.

[0025] Various exemplary embodiments of the present disclosure will now be described with specific reference to the accompanying drawings. Various modifications and alterations may be made to the exemplary embodiments of the present disclosure without departing from the spirit and scope of the present disclosure. Therefore, it should be understood that the embodiments of the present disclosure are not limited to the exemplary embodiments described below, but are subject to the limiting factors shown in the claims and any equivalents.

[0026] Generally, humans and many machine learning implementations make decisions under conditions of probabilistic uncertainty. Identifying patterns, inferences, or connections within a dataset through passive observation without introducing intentional or unintentional biases or disordered assumptions is challenging. Datasets can introduce additional challenges because they can introduce 1) selection or sampling bias, 2) confounding variables, and 3) a lack of directional evidence. Controlled or adaptive experiments aim to eliminate bias by introducing aspects of randomization, blocking, and balancing, but are still hampered by the large amount of prior knowledge required to provide tangible results (i.e., ensuring high internal and external validity) and the rigid constraints imposed by real-world decision-making. Adaptive experiments perform one or more steps sequentially and often need to end previous steps before subsequent steps can be resolved. The technique described in this paper overcomes passive observation and adaptive experiments by transforming controlled or adaptive experiments into a non-sequential process of repeatedly analyzing and optimizing data through self-organizing experiments. Self-organizing processes rationally utilize the temporal, sequential, and natural variability in parameters of decisions to automatically compute and explicitly infer causal relationships. The advantages of self-organizing adaptive learning systems and methods over existing adaptive experimentation techniques include the ability to operate on impoverished inputs when conditions or interactions are initially unknown, incomplete, or estimated as hypotheses and learned over time. Another advantage of adaptive learning systems and methods is their robustness to erroneous assumptions, including the effects of time, the duration of content conditions that should or can be analyzed, and external factors (e.g., consumer fashion or trends, seasonal changes, natural or man-made disasters, etc.). A further advantage over existing systems is the iterative application of spatially discontinuous causal relationships, as the location of content and its comparative impact on arrays of other content are crucial for understanding and optimizing the most effective content for e-commerce systems.

[0027] This system and method enable real-time understanding and quantification of causal effects, while providing fully automated operational control and comprehensive multi-objective optimization. The self-organizing system and method behave robustly and scalably, and operate effectively on complex real-world systems, including those with biases in spatial-temporal relationships and product diversity (i.e., e-commerce systems).

[0028] In modern e-commerce systems, suppliers can influence different types of consumer behavior based on displayed and interactive content elements. The challenges experienced in e-commerce systems involve indicating measurable consumer responses to content. Some suppliers focus on easily measurable and understandable consumer responses, which may include click-through rates or responses to surveys / questionnaires. For example, a consumer “click” is one type of consumer response that analyzes products, images, or links that consumers browse or interact with on an e-commerce website. These are measures of interest that may or may not lead to actual product sales. Often, understandable and easily measurable consumer responses may not accurately reflect parameters that provide strategic direction to suppliers, such as sales, revenue, and profit. For example, a consumer might click a link because an image caught their attention, but they may not intend to buy the corresponding product. In this example, a supplier seeking only to optimize the number of “clicks” on an e-commerce website to sell their products might miss opportunities to select content that directly increases sales and profits. “Clicks” are variable, and consumer behavior varies in terms of what they represent and how it is interpreted as a conversion (i.e., indicating that specific content influenced sales). For example, a consumer might already know they want to buy a product from an e-commerce website and will click once and then make a purchase. Before actually purchasing a product, another consumer may actively browse multiple e-commerce websites once or multiple times on the same day or over a period of several days or weeks. Systems designed to maximize relevance must understand what content directly leads to a sale and when to confirm it.

[0029] In many e-commerce systems, manually selecting displayed or interactive content options to achieve business goals (i.e., increasing sales and profits) is expensive and time-consuming / labor-intensive. For e-commerce websites managing multiple products, such manual selection becomes increasingly difficult. Furthermore, optimizing the sales of a single product may gain market share from competitors, but could lead to the obsolescence of similar products from suppliers. For example, a supplier might sell multiple furnace filters with many different options and profit margins. The supplier wants furnace filters to outsell competitor brands, but simultaneously wants furnace filters with higher profit margins to be purchased rather than those with lower profit margins. By merely optimizing sales, the supplier may miss opportunities to optimize profits or revenue. Generally, each product is managed individually without considering its interaction with other products. Another advantage of self-organizing adaptive learning systems and methods is their ability to assess and address product obsolescence, and more generally, optimize the product portfolio.

[0030] The implementation includes methods and systems for optimizing business objectives on e-commerce platforms. System inputs may include candidate content elements (e.g., fragments of text and / or images) and constraints (e.g., a 200-character limit for product titles or descriptions) to illustrate how and why content can be combined to present to consumers. Inputs may also include initial assumptions about, for example, business objectives, historical context and previous discoveries / learning, the time lag between viewing content and making a purchase decision, and system constraints. The system according to the implementation can specify protocols for assembling content elements. Methods according to some implementations can identify causal relationships between served content elements and purchasing behavior while optimizing revenue and profit. The system can be configured for any objective represented by human behavior. As described in more detail below, causal effects are measured by calculating the statistical significance of the presence (relative to non-existence) of content elements on or within a set of self-organizing experimental units. The assessment of statistical significance is accomplished by calculating confidence intervals, which quantify the expected value of the effect of the content element and its surrounding uncertainty (and represent a measure or degree of inference). In this case, the calculation of unbiased confidence intervals is relatively straightforward due to random sampling / randomization. The ability to interpret and adaptively use confidence intervals to automatically understand and apply the specific effects of content inclusion, placement, and duration, as well as their self-organizing comparisons with other content (to eliminate confounding effects of covariates), similar to deep learning, is a significant advantage over current solutions. The calculation of one or more confidence intervals allows for risk-adjusted optimization, as they quantify the expected effects and their surrounding range (i.e., the quantification of best- and worst-case scenarios). Depending on the implementation, the method and system can identify and adjust for erroneous inputs (e.g., incorrect assumptions) that confound causal knowledge and limit optimization outcomes, as well as monitor and apply changes in the causal relationship between content and consumer behavior.

[0031] Figure 1 This is an illustration of a system 100 for e-commerce content generation and optimization, according to various examples. System 100 includes a memory 102 and a processor 104 coupled to the memory 102. The processor 104 can receive input from a user interface 110, including one or more hypotheses 106 comprising multivariate comparisons of the content. Hypotheses 106 can also be retrieved from the memory 102. The input may also include content elements, which may also be stored in or accessed from the memory 102. As described earlier herein, the content will be provided to and optimized on an e-commerce system 114 to maximize business objectives.

[0032] Processor 104 and memory 102 may be part of user system 116, which includes a user interface 110 for inputting hypothesis 106. For example, user system 116 may be a mobile device (e.g., a smartphone, laptop, etc.) or a fixed device (i.e., a desktop computer) running an application on a device or in a cloud environment, displaying user interface 110 and connected to e-commerce system 114 via a wired or wireless network. In another embodiment, processor 104 and memory 102 may operate on e-commerce user system 118. E-commerce user system 118 receives input from user interface 110 operating on a mobile or fixed device running an application on a device or in a cloud environment. Hypothesis 106, including content elements, will be stored directly in or processed within e-commerce user system 118. User system 116 and e-commerce user system 118 may also operate simultaneously, meaning data can be stored and processed interchangeably between them.

[0033] Processor 104 can repeatedly generate self-organizing experimental units (SOEUs) 112 based on one or more hypotheses 106. SOEUs 112 (which will be referred to below as...) Figure 3 (And related tables describe in more detail) Quantify the reasoning within and between content.

[0034] At least one SOEU 112 may include the duration for which the corresponding SOEU 112 will be active in the system (e.g., e-commerce system 114). Processor 104 may generate multiple SOEUs 112 whose durations are randomly selected based on a uniform distribution, Poisson distribution, Gaussian distribution, binomial distribution, or any distribution supported on bounded or unbounded intervals. In one embodiment, the duration may be the longest duration of all generated SOEUs, and all intermediate durations are recorded simultaneously. Processor 104 can then select the duration that maximizes statistical significance from all recorded durations. Processor 104 may also dynamically modify (i.e., increase or decrease) the potential durations between SOEUs 112 until the lag effect of SOEU 112 on subsequent SOEUs 112 is reduced or completely eliminated, meaning the effect is fully reversible. Processor 104 may assess external validity based on quantitative inference or adapted causal evaluation (i.e., by comparing the application to a baseline, where the baseline may be relative to...). Figure 2 (Average of all possible content options defined in more detail) positive or negative results to increase or decrease the duration of at least one SOEU 112.

[0035] E-commerce system 114 may include online shopping or product sales portals, websites, or mobile applications. E-commerce system 114 may be, for example, an enterprise content management system that optimizes business-to-business (B2B) objectives or directs consumers to private or public portals displaying and trading products (e.g., Amazon, Target, Home Depot, Walmart, etc.). It may also include intranets or internet search engines (e.g., Google, Yahoo, Bing, etc.) as consumers / users use them to explore products, compare prices, and read consumer reviews. Each SOEU 112 may represent a product or a variation of content specific to a product. Processor 104 may group SOEU 112 into blocks or clusters based on quantitative inference of the variation in content effects between experimental groups. Quantitative inference is based on characteristics of the content contained in each SOEU and between experimental groups, such as product, year, geographic location, etc. Processor 104 may identify different causal interactions for each cluster and select the best content for each cluster based on individual causal inference for each cluster.

[0036] Once generated, processor 104 can continuously inject SOEU 112 into e-commerce system 114, according to the following description relative to... Figure 3 The method and standard iteratively modify SOEU 112 and identify at least one causal interaction of content within the e-commerce system 114. Processor 104 can initially and iteratively assign content to SOEU 112 in a manner that is proportional to the amount of evidence of relative expected utility quantified by confidence intervals, albeit less uniformly. Processor 104 can generate at least one set of SOEU 112 based on a uniform probability distribution of included experimental units associated with at least one hypothesis 106, using processing as defined below.

[0037] Assumption 106 may include objectives for the e-commerce system 114. These objectives may include performance metrics for system risk-adjusted optimization. Examples include, but are not limited to: revenue, top-line or bottom-line sales, gross profit, profit margin, cost of goods sold (COGS), inventory management / level, price, shipping / shipping costs, market share, or combinations thereof.

[0038] Assumption 106 may include content elements that identify product attributes or specific details. Examples include, but are not limited to: product title, description, purpose, size, price, or combinations thereof.

[0039] Assumption 106 may include time constraints or specific constraints on the content. Time constraints concern the time and duration during which the content will be active, inactive (i.e., appropriate only at specific times of a day or year), or displayed in the system. Constraints on the content include the presence or absence of product images or videos, standardization of product brand names or labels, empty or blank text, repeated text, use of symbols, the maximum number of characters that can be used, or combinations thereof.

[0040] Hypothesis 106 can be initially defined when additional information becomes available or when the system is analyzed and causal reasoning is optimized, and then updated manually or automatically over time.

[0041] User interface 110 is a web-based or application-based portal that users access to input assumptions 106 for the system. User interface 110 can be presented as a graphical user window on a monitor or smartphone display. Users input assumptions 106 using a keyboard or virtual keyboard on a device used to access the system.

[0042] Components of system 100 can operate on fixed devices (e.g., desktop computers or servers) and / or mobile devices (i.e., smartphones) while connected to the e-commerce system via a local, group, or cloud-based network. One or more components of system 100 can also operate on fixed devices and / or mobile devices after the e-commerce system 114 receives a connection and direction.

[0043] Figure 2 It is a block diagram of the software modules and self-organizing core processes of the e-commerce content generation and optimization system 100 executed by the processor 104.

[0044] The software modules and self-organizing process include: a target module 202; a content element module 204; a standard data module 206; a maximum / minimum time-accessible range data module 208; and a content constraint module 210. The target module 202, content element module 204, standard data module 206, maximum / minimum time-accessible range data module 208, and content constraint module 210 can provide sufficient structure to begin generating SOEU 112 ( Figure 1 (and does not require detailed, specific details and precision).

[0045] Human supervisors or artificial intelligence (AI) agents 211 may adjust content elements and constraints at any time before, during, and after the implementation of the method, or when it is reasonable to adjust content elements and constraints. For example, when the system and method operate at the maximum value of boundary conditions (as defined by the constraints) and the effects have not yet plateaued. In some implementations, processor 104 may provide (e.g., to a display) instructions for potential actions to be taken by the human supervisor or AI agent. Feedback or updates on hypotheses or objectives may also be received manually or automatically from the human processor or AI (i.e., consumer comments or trends received from social media websites).

[0046] Processor 104 may additionally prompt or enable the user to provide a continuously prioritized list or queue of candidate content options. If this queue is provided to processor 104, processor 104 can reasonably introduce new options when introducing them will not adversely affect optimality. Similarly, when processor 104 detects that content options have little or no benefit, it can remove these content options, thereby prompting the human operator to review these content options for removal.

[0047] The processor 104 can also adjust for the fact that the cost of changing content may not be zero. The cost of content change can be part of the target and utility measured by the processor 104, thereby identifying a resource allocation optimization problem in which the cost (usually known) is balanced with the perceived potential value (not yet quantified).

[0048] Target module 202 receives, stores, displays, and modifies one or more conversion performance metrics of the e-commerce system that it will optimize. These targets can range from simple metrics (e.g., sales, revenue, gross profit, COGS, etc.) to weighted combinations of multiple metrics or any other functional conversion (e.g., considering complex cost factors, supply chain issues, inventory availability, etc.). If specified, the metrics and their corresponding user-assigned weights (i.e., importance values) are combined into a multi-objective utility function. User-assigned weights can be expressed as numbers or percentages. In some implementations, weight values ​​are non-negative and non-zero, and can be less than, equal to, or greater than 1%, 5%, 10%, 15%, 20%, 25%, 30%, 35%, 40%, 45%, 50%, 55%, 60%, 65%, 70%, 75%, 80%, 85%, 90%, 95%, or 99%. In other implementations, the weight values ​​can be numerical and can be less than, equal to, or greater than 1, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, or 99. The multi-objective utility function can be modified or refined at any point in time (or place) when the business objective changes (i.e., aggressive market penetration to maximize revenue).

[0049] Content Element Module 204 receives, stores, displays, and modifies user-provided content options, including a complete array of possible content combinations within the search space. A content element is a specific instance of text, images, videos, etc., that defines a service or product in technical or marketing language. Other examples of content elements include: consumer reviews of products obtained through e-commerce systems or other web pages or websites, payment for products or services, the use of financial incentives (i.e., discounts), and inventory levels / management. It should be noted that content elements can be fine-grained to control (e.g., phrases / words), image elements, etc. Content elements can be displayed through a user interface (i.e., Figure 1 The user interface 110 allows for manual input or updating, or automatic pasting, importing, copying, or uploading from another application or platform (e.g., Microsoft, LinkedIn, Pinterest, Facebook, Amazon.com, other social media websites, etc.) using natural language processing, sentiment analysis, generative adversarial networks, etc. Importantly, content elements can be updated (e.g., adding and / or deleting) without affecting what the system has already learned.

[0050] Standard data module 206 receives, stores, modifies, and represents past or historical conversion performance metrics (corresponding to defined targets) describing the performance of e-commerce products before implementing a system for a set of services or products. This data can optionally be used to calibrate the system and its initial decision changes. It also includes prior findings or inferences learned by the user or system during previous implementations. Standard data can be accessed through a user interface (i.e., Figure 1 The user interface 110) allows manual input, or automatic import, copying, or uploading from another program or platform (e.g., ORACLE, MICROSOFT, TURBOTAX, SAP, etc.) into the system.

[0051] The Maximum / Minimum Time-Accessible Range Data Module 208 receives, stores, modifies, and represents the initial estimates of the maximum and minimum ranges of diffusion and decay of the causal effects of content changes in behavior / decision throughout the e-commerce system. In this example, decay refers to the amount of time an experimental unit is deactivated before another experimental unit is activated. It refers to the amount of time it takes for the results of a specific content allocation to clear the system (i.e., to become undetectable). System-defined or user-defined durations (whether time or time percentages) may also exist between the active and inactive states of an experiment. This module is used to define the initial search space and to generate orthogonal self-organizing experimental units.

[0052] Content constraint module 210 involves a set of content rules provided by the user or e-commerce system that restricts the overall combinatorial search space of possibilities. Content constraint module 210 receives, stores, modifies, and represents user- or system-defined constraints. These include user-defined rules or e-commerce system-specified deterministic models that define the boundaries (or limitations) of content. Constraints can be “soft,” meaning the system will adhere to them until evidence is provided that the assumption defining them is incorrect, or constraints can be “hard,” meaning the system will adhere to them (i.e., never violate them) without deviating from or considering other evidence. Constraints include, but are not limited to: the location of content that can be applied within the e-commerce platform (e.g., product titles versus detailed product descriptions); constraints on multiplicity and co-occurrence (e.g., content options cannot be used together if they can be repeated); and constraints specified by the e-commerce platform (e.g., maximum character length for product titles). Constraints can be updated during implementation because reasoning can be quantified to explore the impact on utility at or near boundaries. Content elements and constraints provide opportunities for human agents to manage risk and reward by constraining or expanding the range of system options.

[0053] The core algorithmic approach and process 212 uses the target module 202, content element module 204, standard data module 206, maximum / minimum time-accessible range data module 208, and content constraint module 210 to generate a content specification protocol 214 defining real-world content for application at any given point in time. The core algorithmic approach and process 212 can be initialized by humans, another machine learning method (e.g., for initializing relevance inference), other statistical methods (e.g., for defining initial sampling probability distributions for experimental units and content elements), or a combination thereof. The core algorithmic approach and process 212 includes the following: experimental unit generation process 216; processing allocation process 218; exploration / application management process 220; baseline monitoring process 222; data inclusion window management process 224; and clustering in the experimental unit process 226.

[0054] The generation of experimental unit procedure 216 is based on inputs received from core modules 202, 204, 206, 208, and 210 to identify statistically equivalent spatial-temporal units (i.e., where experimental conditions are equivalent and the duration of the unit is Pareto optimized to minimize lag effects while maximizing statistical power). An ideal experimental unit is characterized by a minimum spatial / temporal range that prevents lag effects from degrading the generated causal knowledge. In one implementation, this can be identified through a systematic exploration of the spatial / temporal range of experimental units to discover the optimal unit size corresponding to the average effect size located at a 95% confidence interval (p = 0.05) from asymptotically averaged effect over a large spatial-temporal range. The generation of experimental unit procedure 216 identifies interchangeable experimental units (i.e., forming clusters of interchangeable experimental units) and optimizes the spatial and temporal characteristics of the experimental units within each cluster by minimizing lag effects while maximizing statistical power (i.e., the number of EUs). Examples of the generation and execution of experimental units, the selection and use of independent and dependent variables, and the allocation of spatial / temporal conditions are described, for example, in jointly owned U.S. Patent No. 9,947,018 (Brooks et al.) and U.S. Patent Publication No. 2016 / 0350796 (Arsenault et al.).

[0055] The assignment process 218 provides controlled random assignments (such as randomization without replacement, reverse balancing, and block-based randomization) of content elements to experimental units at assignment frequencies following a uniform or predefined probability distribution (i.e., historical or normal operation) until differences in utility are detected, explored, and applied. Within each cluster of exchangeable experimental units, the assignment of independent variable (IV) levels can follow a fully factorial design, partial factorial design, block design, or Latin square design that allows multiple block-based factors. Independent variables (IVs) are assigned such that the relative frequencies of the assignments match the relative frequencies specified by the exploration / application management process (described below). Block-based assignment involves balancing assignments among external factors (i.e., confounding factors), while clustering involves isolating assignments for each confounding factor. The choice between block-based and clustering depends on the strength of the covariates and the statistical power (i.e., clustering only begins once sufficient SOEUs have accumulated). When the number of external factors is large and both are part of a “self-organizing” process, they can coexist.

[0056] The lag effect of content allocation within the experimental unit is operablely and adaptively controlled. A lag effect means that an effect of content allocation contaminates the measured effect of the next treatment. To eliminate lag effects, the duration of the treatment allocation must match the maximum / minimum temporal reach of the effect. For example, if min = 0 and max = 4, the optimal duration could be 4 with a frequency of 1 / 8 (using the last 4 days within an 8-day timeframe). In another example, if min = 4 and max = 4, the optimal duration could be 1 with a frequency of 1. This can also depend on whether the effect is persistent (i.e., stable over time within the duration of the experiment) or transient (i.e., varies over time within the duration of the experiment).

[0057] The exploration / application management process 220 analyzes confidence interval (CI) overlap to explore frequency using probability matching, rational choice theory, or other techniques, where smaller overlap between CIs leads to more frequent use of the level associated with maximum utility. For each experimental unit, the system needs to decide whether to allocate the experiment to make the best decision with the highest probability, or to improve the accuracy of the probability estimate (i.e., CI). The system can modify the radicalness of the application allocation and place itself under experimental control to find the radicalness that maximizes utility (including minimizes regret) relative to the exploration allocation determined by baseline monitoring, where the baseline is defined as the average of all levels (i.e., exploration). The system monitors the gap between application and exploration, thus providing an objective measure of regret. Regret is the expected reduction in utility / reward due to initiating the exploration process instead of using the application process for optimization. When the cost (including opportunity cost) of performing a treatment is not uniform across the levels of the independent variable, a Bonferroni-corrected confidence interval (or inference) is calculated, indicating that more evidence is needed to apply the more expensive treatment.

[0058] The baseline monitoring process 222 continuously analyzes the baseline in real time through periodic random assignment to provide an unbiased measure of utility improvement. Baselines can be assigned based on the required quantification value; their default states can be assigned for investigation or application. In addition to the experimental units assigned as described above, the system continuously determines the number of baseline experimental units needed to monitor the performance differences between these baseline trials and treatment assignments through statistical power analysis. Baseline experimental units are randomly sampled based on standard operating range data. The differences between baseline trials and investigation / application trials provide an unbiased measure of the utility of internal parameters (including clustering, data inclusion window, investigation / application radicalism), allowing for objective adjustment of such parameters. Baseline trials also ensure that the entire search space is constrained by the investigation.

[0059] The Data Inclusion Window (DIW) management process 224 uses factor analysis of variance (ANOVA) or other methods (i.e., normality tests) to analyze the impact of time variance on the strength and stability of the interaction between the selected independent variables and the utility function, thereby analyzing the extent to which the data represents the current state of the e-commerce system to provide real-time decision support. For each independent variable, it determines the Pareto optimal data inclusion window that maximizes both the experimental power (across all experimental unit clusters and the entire decision search space) and statistical significance of the causal effect. This prevents the process from overfitting the data and allows it to remain highly responsive to dynamic changes in the underlying system's structure. Confidence intervals are calculated on the Pareto optimal data inclusion window to provide a trade-off between precision (narrow confidence intervals) and accuracy as conditions change over time. This DIW can be initially defined by the user based on input constraints. Generally, the system operates based on the assumption of instability (i.e., it is not 100% stable) and adjusts dynamically.

[0060] The clustering process of experimental unit procedure 226 conditionally optimizes SOEU injection and content allocation based on external factors outside experimental control to provide honest or impartial evidence of causal interactions. Clustering is used to manage dimensions in the system by learning how to conditionally assign independent variable levels based on the effects of independent variable levels and the interactions between experimental unit attributes that cannot be manipulated by the system (e.g., seasonal or weather effects, content demand, location on an e-commerce website, etc.). The system's dimension / granularity (i.e., the number of clusters) is always commensurate with the amount of available data. Therefore, there is no limit to how many external factors can or should be considered. External factors with large effects are identified and clustered first, while other factors are managed through blockization. The more characteristics of the experimental units are known, the more effective the process is in eliminating confounding and effect correction factors. Confounding is generally addressed by randomization, and effect correction factors are eliminated by clustering. Initial hypotheses include which characteristics should be considered based on prior knowledge or evidence that they are indeed important. Hypotheses can be added or removed over time as needed. Adding more characteristics does not necessarily increase dimensionality, as they will be ignored until evidence supports the need for clustering. It is achieved by merging experimental units into clusters that have the greatest intra-cluster similarity and the greatest inter-cluster difference in the impact of independent variables on utility. The number of clusters is optimized using two related mechanisms: 1) Using techniques including factor ANOVA, independence tests, conditional inference trees, etc., to find factors that explain the largest amount of variance between clusters, and using stepwise statistical power analysis to select multiple factors that produce clusters with sufficient statistical power to find applicable utility; and 2) By continuously testing clustering decisions and using baseline monitoring to objectively explore and apply their impact on utility, clustering decisions are placed under experimental control.

[0061] Table 1 shows the core algorithm methods and processes once the e-commerce system is implemented. Figure 2 Each core algorithm method and process in ) can be operated in stages. Stages are defined as initiation, exploration / application, cluster initiation, and continuous cluster optimization. Once input and hypothesis 106 ( Figure 1 The initiation phase occurs when the system begins analyzing the data contained in the target, content elements, standard data, maximum / minimum time-accessible range data, and content constraints modules. Figure 2-2 Data from 02, 204, 206, 208, and 210 are used to define variables and experimental unit widths. The exploration / application phase uses statistical probability matching to repeatedly evaluate the data, adjusts experimental unit durations to investigate the search space definition, and determines cluster assignments. The cluster initiation phase proactively analyzes one or more assigned clusters and their potential impact on confidence intervals for repeated computations. The continuous cluster optimization phase computes cluster variability to identify causal inferences between confidence intervals.

[0062] Table 1: Core Algorithm Methods and Process Implementation by Stage

[0063]

[0064] The Point-of-Sale (POS) Business Data Module 228 receives, stores, and accesses data related to consumer transactions, including payments for products or services, the use of financial incentives (i.e., discounts), inventory levels, and supply chain management. Information uploaded and used in the Point-of-Sale (POS) Business Data Module 228 can provide additional context for generating and iterating SOEUs and identifying causal inferences. POS data can be received daily, weekly, monthly, yearly, etc., and its reception is primarily based on the structure and requirements of the e-commerce website.

[0065] The causal knowledge module 230 systematically executes the core algorithmic methods and procedures 212 (previously defined) to calculate confidence levels of relative effects distributed around different content, thereby representing the expected value of the effect on the multi-objective optimization function and the uncertainty surrounding that estimate, while minimizing confounding from external or internal factors, exploring / applying causal reasoning, and optimizing the objectives based on the initial definition or refinement. Confidence intervals for each level of the independent or dependent variable, or a combination of levels of independent variables, are calculated in the causal knowledge module 230. This is calculated by taking the difference between the average effects when the variable is activated and when it is deactivated on the data inclusion window, thus providing an estimate of the causal effect. Exemplarily, in some implementations, confidence intervals for each duration can be calculated simultaneously or sequentially if the data inclusion window satisfies a normality test (i.e., the Shapiro-Wilk test) with the largest p-value (i.e., 0.05) for each duration. Alternatively, the duration with the largest statistical power (or alternatively, the smallest t-test p-value) on each corresponding data inclusion window can be selected. Each variable and each cluster may have a specific data inclusion window (i.e., they can all be the same or different). The execution of processes does not need to be sequential, and they are advantageously operated independently and frequently as needed to improve optimization capabilities. Continuous evaluation of the incremental values ​​of learning and application (i.e., how many more values ​​need to be captured probabilistically?) includes the potential impact of adding, editing, or removing independent variables (i.e., expanding the search space). Causal inference requires: 1) commutativity between experimental units, meaning they are commutative at any time during the analysis and the results will not change; 2) independence between experimental units (i.e., no hysteresis effects); 3) consistency in treatment allocation and management; 4) reversibility of effects; and 5) positivity in selection.

[0066] The continuous optimization module 232 further enhances the effectiveness of probability matching to stimulate clustering for identifying, monitoring and improving experimental unit processes 226 and to explore / apply management processes 220.

[0067] Figure 3 This is a flowchart of a computer-implemented method 300 for content generation and optimization, based on various examples. The operation of method 300 can be performed by elements of system 100 or by... Figure 2 The element is executed, and reference system 100 or Figure 2 Elements within. Figure 3 The steps outlined and the computer-implemented method 300 may be performed simultaneously in different orders, or may include steps that are not explicitly identified.

[0068] Let's use an illustrative example to explain method 300. In the illustrative example, a supplier wants to optimize the sales of two products offered on an e-commerce website. These two products are named PR01 and PR02.

[0069] See Figure 3 And using the example scenario outlined above, method 300 for content generation and optimization begins with operation 302, where processor 104 ( Figure 1 The system receives one or more hypotheses for randomized multivariate comparisons of the content. This content is provided by the provider to the e-commerce system 114. Figure 1 Assumptions include, for example, descriptive content and constraints on the content provided by content element 204 and content constraint module 210. Constraints include time constraints (e.g., provided by maximum / minimum time availability data module 208) or constraints on content type, or other constraints or combinations thereof. Assumptions include, for example, objectives for e-commerce system 114 received by one or more target modules 202.

[0070] In this example, the target (consisting of one or more target modules 202) Figure 2 Management includes optimizing sales of both products and through the user interface 110 ( Figure 1 The content element is input into user system 116. (Content element module 204) Figure 2 Management includes, for example, product titles and identified descriptive characteristics. Standard data (converted from historical data module 206) Figure 2 Management includes the reporting and collection of historical sales data for both products. Maximum / minimum time-accessible range data (from the minimum / maximum time-accessible range data module 208) Figure 2 The management includes data on how quickly consumers purchase a product after being exposed to its content. For example, compared to PR01, 95% of consumers can purchase a product within 1-3 days of its content being exposed on an e-commerce website. A summary of these assumptions is shown in Table 2. A constraint is defined, limiting the number of characters, including both text and numbers, that can be used for descriptive features.

[0071] Table 2: Hypothetical Reception

[0072] name title Feature A Feature B Feature C price Maximum / Minimum Time Availability PR01 title Feature A Feature B Feature C price time PR02 title Feature A Feature B Feature C price time

[0073] The supplier then provides content options that best convey or express information about the product title or descriptive features that can increase interest and lead to sales. Example content options for both products are shown in Table 3. <Blank> indicates that no text is provided as an option or that the content option is not defined. Variables (Heading 1, Heading 2, A1, A2, B1, B2, and C1) represent any textual and numerical text specifying the feature (such as "durable," "superior performance," or "available in multiple colors"). Some feature options are similar across both products, while others are different. For example, feature option C is the same for both products, while feature options A and B are different.

[0074] Table 3: Product Content Options

[0075] name Title Options Feature A option Feature B Feature C option PR01 Title 1 or Title 2 <Blank> or A1 or A2 <Blank> or B1 <blank> or C1 PR02 Title 1 or Title 2 <Blank> or A2 <Blank> or B1 or B2 <blank> or C1

[0076] Method 300 continues with operation 304, where processor 104 repeatedly generates SOEU 112 for inference between quantized contents based on one or more hypotheses. In an exemplary example, SOEU 112 includes repeatedly generating and iterating core algorithm methods and procedures 212 (…). Figure 2 ).

[0077] Experimental Unit Procedure 216 ( Figure 2 We generated assignment variables and randomized content options to begin analyzing their effects on an e-commerce system. Table 4 shows the captured hypotheses and variable assignments of content options based on this example. EV represents external variables. IV represents independent variables. RV represents response variables to content assignments (e.g., level dependent variables within independent variables).

[0078] Table 4: Variable Allocation for Experimental Units

[0079] variable definition EV1 sales revenue 1 PR01's historical sales speed EV1 sales 2 PR02's historical sales speed IV1 Level 1 Title 1 IV1 Level 2 Title 2 IV2 Level 1 <Blank> IV2 Level 2 A1 IV2 Level 3 A2 IV3 Level 1 <Blank> IV3 Level 2 B1 IV3 Level 3 B2 IV4 Level 1 <Blank> IV4 Level 2 C1 RV1 The effect of IV1 RV2 The effect of IV2 RV3 The effect of IV3 RV4 The effect of IV4

[0080] In some implementations, processor 104 can generate experiments with durations randomly selected based on a specific statistical distribution. Several factors influence or lead to the choice of statistical distribution, and often involve a trade-off between efficiency and computational duration. If there is no prior knowledge indicating that one duration is superior to another, the statistical distribution can be uniform, can be normally distributed around historical estimates, or can be any distribution supported on bounded or unbounded intervals. Speed ​​and accuracy of the analysis are important. Computational quality causal inference can take longer. As previously mentioned, statistical distributions include: uniform distribution, Poisson distribution, Gaussian distribution, binomial distribution, or any distribution supported on bounded or unbounded intervals. Without loss of generality, a uniform distribution is chosen in this example. Table 5 shows exemplary randomized experimental units generated, where their double-blind randomization assignments are not replaced. Duration is defined as the length of time an experimental unit remains active in an e-commerce system where T1, T2, and T3 represent different time intervals. The randomized experimental units create content specification protocol 214 ( Figure 2 The e-commerce system will execute this content specification protocol to quantify causal reasoning. Content probability distributions are initially based on historical data and / or constraints (if any, otherwise uniform), and over time based on content discovered through exploration / application of management. Product probability distributions are based on block decomposition and also on clustering over time.

[0081] Table 5: Exemplary Experimental Units

[0082] EU product Duration EV1 IV1 IV2 IV3 IV4 1 PR01 T1 Sales revenue 1 Level 1 Level 2 Level 2 Level 2 2 PR02 T2 Sales revenue 2 Level 1 Level 2 Level 2 Level 1 3 PR02 T2 Sales revenue 2 Level 2 Level 2 Level 1 Level 1 4 PR01 T3 Sales revenue 1 Level 1 Level 2 Level 1 Level 1 N PR01 T2 Sales revenue 1 Level 1 Level 1 Level 1 Level 1

[0083] Processing allocation procedure 218 ( Figure 2 The baseline is defined as the average of all combinations of variables, and a portion of the generated experimental units is assigned to the baseline. As method 300 continues, baseline monitoring procedure 222 ( Figure 2 Baselines were assigned for exploration and the baseline definition (i.e., exploration frequency) was further refined. Evaluations were conducted based on the SOEU definition, and an initial block and cluster were assigned.

[0084] Method 300 continues with operation 306, wherein processor 104 continuously injects self-organizing experimental units (SOEUs) into e-commerce system 114 to generate quantitative inference about content. Processor 104 follows the content specification protocol 214 ( Figure 2Instructions are injected into the experimental unit. Once injected into the e-commerce system, SOEU is initiated and executed. As the experimental unit ends, the next available unexecuted (i.e., assigned to a different block) experimental unit begins. POS data 228 is collected as a result of executing SOEU on e-commerce system 114, and this POS data is received by the core algorithm process 212 to compute causal knowledge process 230. Figure 2 The sales difference, confidence interval, and causal interaction of ().

[0085] Method 300 continues with operation 308, where processor 104 identifies one or more confidence intervals in the injected SOEUs. When the experimental unit ends, the confidence intervals are repeatedly calculated to represent the inference the experiment has regarding the sales of the two products. For each SOEU, the processor 104 calculates the resulting sales of both products. Table 6 illustrates how two SOEUs in the SOEUs generate a response variable that expresses the resulting sales of either product (RS1 or RS2). Note: The calculation of the response variable occurs across all SOEUs, and for simplicity, this example is limited to only two SOEUs.

[0086] Table 6: Calculation of Response Variables

[0087] product EV1 IV1 IV2 IV3 IV4 DV1 DV2 DV3 DV4 PR01 Sales revenue 1 Level 1 Level 2 Level 2 Level 2 RS1 RS1 RS1 RS1 PR02 Sales revenue 2 Level 1 Level 2 Level 2 Level 1 RS2 RS2 RS2 RS2

[0088] Calculate the difference between the response variables of the two products at different levels. Note that for this example, only IV4 satisfies the requirement of a single level difference. The difference (Δ) is calculated as |RS2-RS1|. Most commonly, the difference between adjacent levels is calculated on “similar” (i.e., interchangeable) experimental units (e.g., “ON” relative to “OFF” or “Level 1” relative to “Level 2”). They can also be calculated as the average of one level relative to all other levels (if more than one). Then calculate the confidence interval (CI) with respect to the mean and standard deviation of the sampling distribution (see Equation 1), where μ represents the mean and σ represents the standard deviation. A coefficient of 1.96 provides a 95% confidence interval.

[0089]

[0090] This process is repeated for all SOEUs that still operate under the normality assumption due to the central limit theorem (tested for normality using the Shapiro-Wilk test), which produces one or more confidence intervals representing the direction and magnitude of the causal effect caused by the content elements.

[0091] Method 300 continues with operation 310, wherein processor 104 iteratively modifies the SOEU based on at least one confidence interval to identify at least one causal interaction of content within the system. (Exploration / Application of Management Processes) Figure 2 Identify the variation between the calculated confidence intervals to determine which level has greater utility than others. Experimental unit procedure 226 performs cluster search and identifies the variance within the confidence intervals relative to external variables and the identified effect correction factor. Continuous optimization procedure 232... Figure 2 Cluster assignment can be further improved by performing statistical analysis (e.g., ANOVA) to facilitate cluster identification. This is done by aggregating the differences between response variables across all levels and performing a time-series evaluation. Once this clustering has occurred, the calculated differences are specific to that cluster and no longer represent the effect among all SOEUs.

[0092] If no relationship is found between different SOEUs 112 and sales differences (or other parameters), the above operation can continue indefinitely. However, if a potential causal relationship exists, processor 104 will identify causal interactions within the e-commerce system 114. The benefit of optimizing the duration of SOEUs 112 is to adjust the time interval for the duration effect of consumer responses and purchasing patterns. If the duration of SOEUs 112 is too short, the consumer effect from SOEUs 112 will persist after the product switches to the next SOEU 112, which violates the requirement of independent causal inference. This contaminates the attributes of sales differences for product content and weakens the detection of the effect. On the other hand, if the duration of SOEUs 112 is too long, the effect is explicit, but system 100 wastes statistical power by failing to maximize the number of SOEUs that system 100 can execute over time. Therefore, to optimize SOEUs 112, processor 104 adaptively modifies the duration of at least one SOEU 112 until the lag effect of SOEU 112 on subsequent SOEUs 112 decreases. In some implementations, processor 104 can perform probability matching on the SOEU 112 duration, allowing processor 104 to try longer / shorter durations to verify that the SOEU 112 duration is appropriately regulated. If the duration remains stable, the clusters will become smaller as long as there is a continued opportunity to increase homogeneity within the cluster and increase heterogeneity between clusters. At this point, for each IV, cluster, and level pair difference (time series), normality is tested, and it is determined that the data inclusion window should be modified to ensure honest / unbiased confidence intervals representing true causal interactions. Data inclusion window management process 224 ( Figure 2) Manage normality testing and updates. Processor 104 modifies existing assumptions to update durations (e.g., T1, T2, or T3) and content variable levels, resulting in a new SOEU defined as operation 310 in method 300. As the SOEU expires and is regenerated, a new content specification protocol is generated by processor 104 and submitted to e-commerce system 114. Causal knowledge process 230 ( Figure 2 Repeated analysis leads to more accurate confidence intervals and identification of causal interactions within each cluster, thereby effectively identifying content options that have the greatest impact on sales of both products.

Claims

1. A system for optimizing business objectives of e-commerce content, the system comprising: Memory; and A processor, coupled to the memory, is configured to: Receive one or more hypotheses for randomized multivariate comparisons of content that will be provided to the system's users; Self-organizing experimental units (SOEUs) are repeatedly generated based on one or more of the aforementioned hypotheses; The SOEU is injected into the system to generate quantitative inference about the content; In response to the injection of the SOEU, at least one confidence interval within the quantified inference is identified, wherein the at least one confidence interval is calculated by obtaining the difference between the average effects when the variable is activated and when the variable is deactivated, thereby providing an estimate of the causal effect; The SOEU is iteratively modified based on the at least one confidence interval to identify at least one causal interaction of the e-commerce content within the system; as well as The new content option is introduced when it has no adverse effect on optimality.

2. The system of claim 1, wherein baseline monitoring determines the number of previously injected SOEUs used to identify at least one confidence interval.

3. The system of claim 1, wherein the assumptions include constraints on the content.

4. The system of claim 3, wherein the constraint includes at least one time constraint among time constraints.

5. The system according to any one of claims 1-4, further comprising: User input device; and wherein the processor is further configured to: Receive user input, including updated content; as well as A subsequent SOEU is generated based on the updated content.

6. The system according to any one of claims 1-5, wherein the assumption includes an objective for the system.

7. The system of claim 6, wherein the objective includes at least one of sales revenue, profit margin, market share, or inventory management.

8. The system of claim 7, wherein the objective represents a weighted combination of two sales figures, two profit margins, or two inventory management methods.

9. The system according to any one of claims 1-8, wherein at least one SOEU includes the duration for which the corresponding SOEU will be active in the system.

10. The system of claim 9, wherein the processor is further configured to: Generate multiple SOEUs with durations randomly selected based on a probability distribution.

11. The system according to any one of claims 1-10, wherein the processor is further configured to: The duration of at least one SOEU is adaptively modified until the hysteresis effect of the SOEU on subsequent SOEUs is reduced.

12. The system according to any one of claims 1-11, wherein the processor is further configured to: Assign one or more processes to the SOEU; Identifying individual causal interactions based on one or more of the aforementioned processes; and The optimal content for the one or more processes is selected based on the individual causal interactions.

13. The system of claim 12, wherein the one or more processes are assigned based on block grouping, clustering, or any combination thereof.

14. The system according to any one of claims 1-13, wherein the processor is further configured to: At least one content option for one or more SOEUs is assigned based on the variance in the calculated confidence interval.

15. The system of claim 14, wherein the aggressiveness of the application of variance is determined by baseline monitoring.

16. The system according to any one of claims 1-15, further comprising: User display; and said processor is further configured to: Provide the user's display with at least one representation of the causal interaction of the content.

17. A computer-implemented method for optimizing e-commerce content to achieve business objectives, the method comprising: Receive one or more hypotheses for multivariate comparisons of content, including content to be provided to system users; Self-organizing experimental units (SOEUs) are repeatedly generated based on one or more of the aforementioned hypotheses; The SOEU is injected into the system to generate quantitative inference about the content; In response to the injection of the SOEU, at least one confidence interval within the quantified inference is identified, wherein the at least one confidence interval is calculated by obtaining the difference between the average effects when the variable is activated and when the variable is deactivated, thereby providing an estimate of the causal effect; The SOEU is iteratively modified based on the at least one confidence interval to identify at least one causal interaction of the e-commerce content within the system; as well as The new content option is introduced when it has no adverse effect on optimality.

18. The method of claim 17, wherein baseline monitoring determines the number of previously injected SOEUs used to identify at least one confidence interval.

19. The method of claim 17, wherein the assumptions include constraints on the content.

20. The method of claim 19, wherein the constraint includes at least one time constraint among time constraints.

21. The method according to any one of claims 17-20, further comprising: Receive user input, including updated content; as well as A subsequent SOEU is generated based on the updated content.

22. The method according to any one of claims 17-21, wherein the assumptions include objectives for the system.

23. The method of claim 22, wherein the objective includes at least one of sales revenue, profit margin, market share, or inventory management.

24. The method of claim 23, wherein the objective represents a weighted combination of two sales figures, two profit margins, or two inventory management methods.

25. The method according to any one of claims 17-24, wherein at least one SOEU includes the duration for which the corresponding SOEU will be active in the system.

26. The method of claim 25, further comprising: Generate multiple SOEUs with durations randomly selected based on a probability distribution.

27. The method according to claims 17-26, further comprising: The data inclusion window of at least one SOEU is adaptively modified until the hysteresis effect of the SOEU on subsequent SOEUs decreases.

28. The method according to any one of claims 17-27, further comprising: Assign one or more processes to the SOEU; Identify individual causal interactions based on one or more of the aforementioned processes; as well as The optimal content for the one or more processes is selected based on the individual causal interactions.

29. The method of claim 28, wherein the one or more processes are assigned based on block grouping, clustering, or any combination thereof.

30. The method according to any one of claims 16-29, further comprising: At least one content option for one or more SOEUs is assigned based on the variance in the calculated confidence interval.

31. The method of claim 30, wherein the radicalness of the application of variance is determined by baseline monitoring.

32. The method according to any one of claims 17-31, further comprising: The content is displayed as a representation of at least one causal interaction.

Images