Process flow behavior prediction

Abstract

This provides a method, system, and storage medium for realizing process flow behavior prediction, including process flow prediction based on partial flow information using machine learning. [Solution] The method includes: acquiring data corresponding to a completed process flow associated with a resolved incident; generating one or more labels to serve as target variables for training a machine learning model based on the data corresponding to the completed process flow; extracting a subset of data corresponding to a predetermined time range cutoff; generating multiple features derived from the data corresponding to the completed process flow for training a machine learning model; and training the machine learning model to predict the future behavior of an ongoing process flow using the multiple features and one or more labels associated with the data corresponding to the completed process flow.

Description

Technical Field

[0001] Embodiments discussed in the present specification relate to process flow behavior prediction including, for example, process flow prediction based on partial flow information using machine learning.

Background Art

[0002] Process flows are used across various industries and fields to describe a sequence of events associated with a specific incident. In some examples, process flows are designed to represent tasks and actions aimed at achieving a goal. These flows provide a structured framework for capturing and organizing events and interactions within a process, enabling clear communication and analysis. While process flows can also guide task execution, their descriptive nature allows organizations to document and understand the progression of events, decision points, and outcomes associated with a particular scenario or incident. Process flows are often visualized using diagrams, charts, or text descriptions that show each step, decision point, and their interconnections, enabling streamlined operations, error reduction, and improved productivity.

[0003] The subject matter claimed in the present specification is not limited to embodiments that solve any drawbacks or embodiments that operate only in environments such as those described above. Rather, this background is provided only to illustrate one example of a technical field in which some embodiments described in the present specification may be implemented.

Summary of Invention

[0004] According to one aspect of one embodiment, the operation may include acquiring data corresponding to a completed process flow, the completed process flow being associated with a resolved incident. The operation may further include generating one or more labels representing the characteristics of the incident, which serve as target variables for training a machine learning model based on the data corresponding to the completed process flow. In addition, the operation may further include extracting a subset of data that represents one or more events that occurred during a predetermined time range cutoff defined within the total time of the completed process flow. Furthermore, the operation may include generating a plurality of features representing identifiable attributes derived from the acquired data corresponding to the completed process flow in order to train a machine learning model based on the subset of data corresponding to the predetermined time range cutoff. Furthermore, the operation may include training a machine learning model to predict the future behavior of an ongoing process flow using the plurality of features and one or more labels associated with the data corresponding to the completed process flow.

[0005] The objectives and advantages of the embodiments are realized and achieved at least by the elements, features, and combinations particularly indicated in the claims. It should be understood that both the above general description and the following detailed description are illustrative and descriptive, and not limitations of the claimed invention. [Brief explanation of the drawing]

[0006] Exemplary embodiments are described and illustrated with further specifics and details through the use of the attached drawings below.

[0007] [Figure 1] This describes an environment related to training machine learning systems to predict the future behavior of existing or ongoing process flows.

[0008] [Figure 2]This document presents an exemplary environment that includes a system designed to generate labels and features for training machine learning models.

[0009] [Figure 3A] A flowchart illustrating an exemplary method for generating duration-based features as input to a machine learning model is shown.

[0010] [Figure 3B] A flowchart illustrating an exemplary method for generating activity-based features as input to a machine learning model is shown.

[0011] [Figure 3C] A flowchart illustrating an exemplary method for generating graph-based features as input to a machine learning model is shown.

[0012] [Figure 4] This document describes an exemplary environment that includes a machine learning system designed to generate predictions of the behavior of an ongoing process flow based on ongoing process flow data.

[0013] [Figure 5] An exemplary flowchart illustrates an exemplary method for training a machine learning model to predict behavior associated with one or more ongoing process flows.

[0014] [Figure 6] A block diagram of an exemplary computing system, all arranged according to one or more embodiments of this disclosure, is shown. [Modes for carrying out the invention]

[0015] Many organizations and systems use process flows, for example, to illustrate the progression of events or activities related to a particular incident. By capturing the temporal or event-driven dynamics of a process, process flows provide organizations with a descriptive framework for modeling and visualizing workflows, identifying interdependencies, gaining insights into behavioral patterns, and taking action based on the process flow. Widely adopted across industries, process flows enable organizations to track how an incident unfolds over time, providing clarity to complex systems without prescribing rigid execution paths. This versatility makes process flows crucial for improving transparency, resolving incidents, and adapting to the unique needs of diverse operating environments.

[0016] While process flows effectively describe events and activities associated with an incident in a descriptive manner, they often fail to provide accurate predictive insights into the outcomes or behavior of subsequent process flows in a prescriptive manner. Traditional solutions have attempted to predict or forecast future behavior related to process flows by statistically analyzing historical data linked to similar process flows. However, such historical statistical analysis is costly due to the need for significant human resources and is limited to identifying general trends and patterns that may not accurately reflect the nuances of specific process flows within the same category.

[0017] For example, consider a process flow related to a customer issue concerning a specific product. By analyzing historical data from process flows for similar product issues, we can determine that the average resolution time for such cases is approximately two weeks. Based on this statistical insight, we can predict that the current process flow will similarly take approximately two weeks to complete.

[0018] However, this approach relies on overly simplistic assumptions that project past trends onto ongoing and future process flows. This historical statistical approach ignores considering the unique factors that influence individual process flows, such as specific events, activities, and contexts. The inability to incorporate these variables often leads to inaccurate and unhelpful predictions, limiting their usefulness in real-world applications. Even when historical analysis can provide useful insights, the time and expertise required to consider specific relevant variables are sometimes prohibitively expensive.

[0019] As detailed below, one or more embodiments of this disclosure relate to training a machine learning system to predict future behavior associated with an ongoing process flow. In some embodiments, the machine learning model may analyze and incorporate a vast array of variables and relationships within the process flow, including event sequences, contextual factors, and external influences that conventional statistical methods cannot effectively capture. Unlike static statistical approaches, machine learning systems dynamically learn and adapt to patterns in the data, allowing their models to consider nonlinear relationships and subtle interactions between variables.

[0020] In some embodiments, the machine learning system may be trained using data associated with completed process flows. As used in the present disclosure, a completed process flow refers to a process flow that describes a resolved incident. In some embodiments, data from completed process flows may be used to generate labels for training a machine learning model. By leveraging completed process flows, the labels provide more accurate ground truth or correct output for training the machine learning system than data associated with incomplete or ongoing process flows. Therefore, in some examples, the trained machine learning system can achieve improved accuracy by comparing prediction results to these more accurate training labels generated using data associated with one or more completed process flows.

[0021] In some embodiments, the machine learning system may be trained to generate predictions using a subset of data extracted from data associated with one or more completed process flows. In some embodiments, the subset of data may be defined by a predetermined time range cutoff that represents a portion of the total time taken to resolve an incident associated with the completed process flow. For example, a completed process flow may take two weeks to reach resolution, and the subset of data may indicate one or more events, activities, etc. that occurred during the first week of the total two-week period.

[0022] In some embodiments, a subset of data may be used to generate features that can be represented as corresponding feature vectors. In some embodiments, feature vectors may be employed to train machine learning models, neural networks, and the like to predict behavior associated with one or more ongoing process flows. In some embodiments, by training a machine learning model using features generated based on a subset of data corresponding to completed process flows, the trained machine learning model can more accurately predict the future behavior of similar ongoing process flows. In some embodiments, the predicted future behavior may correspond to an ongoing process flow that has reached a point in time similar to that defined by a predetermined time range cutoff.

[0023] In some embodiments, a machine learning system may include a plurality of machine learning models, each trained using a different subset of data associated with a different predetermined time range cutoff associated with completed process flows. In some embodiments, by using different predetermined time range cutoffs for training, each model can specialize in behavior prediction associated with an ongoing process flow at a particular stage. For example, an ongoing process flow may correspond to an incident that has not been resolved for two weeks, then three weeks, then four weeks. Three different machine learning models may be trained with a first time range cutoff of two weeks, a second time range cutoff of three weeks, and a third time range cutoff of four weeks. Continuing with this example, by using three different models, the machine learning system may be better equipped to accurately predict the behavior of an ongoing process flow based on the amount of time that has elapsed without resolving the incident associated with the ongoing process flow.

[0024] In some embodiments, machine learning models can be improved by enabling them to perform analyses that were previously impossible or inaccurate. These improvements can be achieved by addressing the challenges of decomposing, labeling, and extracting data from completed process flows in a way that facilitates the transformation and / or generation of training data for machine learning models. In some embodiments, the transformation and / or generation of training data based on completed process flow data may enable machine learning models trained using the training data to predict the outcomes of ongoing and future process flows with improved accuracy.

[0025] In some embodiments, the system can solve the problem of converting completed historical process flow data into actionable training data for machine learning models. By decomposing the completed process flow, assigning labels to relevant components, and extracting relevant data within a specified time range, the system can generate a rich dataset for training.

[0026] In particular, in some embodiments, the training data used to train the machine learning models described herein may undergo specific improvements and transformations to enable more accurate predictions of future behavior in the ongoing process flow. For example, improvements may include data cleaning to remove noise and inconsistencies, feature engineering to create more informative attributes, data augmentation to increase the diversity of training samples, and normalization techniques to standardize the scale of different features. Additionally or alternatively, the training data may be temporally segmented to capture time-dependent patterns, and relevant historical context may be incorporated via a sliding window approach or a recurrent neural network architecture. These transformations to the training data may enable the machine learning models to better capture complex relationships and temporal dynamics within the process flow, potentially leading to more robust and accurate predictions of future behavior.

[0027] In some embodiments, transforming and / or generating training data in the manner described herein enables machine learning models to learn from past incidents and apply that knowledge to predict outcomes in ongoing or future process flows, potentially improving incident resolution efficiency and decision-making processes.

[0028] Additionally or alternatively, in some embodiments, the transformation and / or generation of training data may be performed via one or more specialized techniques that enable or can enable a computing system to acquire general process flow data (e.g., completed process flow data) and transform it into training data. In some embodiments, the specialized techniques described herein may utilize a combination of rule-based algorithms and statistical methods to identify patterns and relationships within the process flow data. The computing system may apply these algorithms to parse the data, identify significant events or milestones, and extract meaningful features that represent the underlying structure of the process flow. Furthermore, the techniques may incorporate domain-specific knowledge to ensure that the generated training data accurately captures the nuances and complexities of the particular ongoing process flow being analyzed, and / or predictable behavior of the ongoing process flow.

[0029] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0030] Figure 1 shows an environment 100 related to training a machine learning model 108 to predict the future behavior of an existing or ongoing process flow, according to one or more embodiments of the present disclosure. In the illustrated example, the environment 100 may include a process flow pipeline 104 and a machine learning model 108. In some embodiments, the process flow pipeline 104 may be configured to receive completed process flow data 102 and generate training data 106 for training the machine learning model 108.

[0031] The completed process flow data 102 may include any data associated with the completed process flow. As used in this disclosure, a completed process flow is a description of an event corresponding to a resolved incident.

[0032] In some embodiments, the completed process flow data 102 may include data associated with an incident, which may refer to any category of occurrence, condition, or situation that can be resolved using an action or intervention. In some embodiments, an incident may include a category that describes multiple occurrences. For example, in the context of customer service, an incident may include a customer's device that may not be functioning properly. Continuing this example, the customer's device may have multiple anomalies requiring service (e.g., a broken screen, a missing or broken sensor), but the overall incident for the customer's device will not be resolved until the device is repaired or replaced. Additionally or alternatively, an incident may refer to a single situation or event. For example, in the context of pedestrian transport using a scooter, an incident may include a single ride on the scooter from a starting point to a destination. As an additional example, an incident may include a rental car, in which case the incident may include any events, state changes, activities, etc., that may occur from the pickup of the rental car to the drop-off of the rental car.

[0033] In some embodiments, the completed process flow data 102 may include data related to incident resolution. In some embodiments, incident resolution may be defined according to a predetermined set of principles or criteria established to ensure consistency and effectiveness. For example, in the context of customer service involving a malfunctioning device, resolution may be explicitly defined as the device being restored to proper function or being replaced with a new, different, or modified unit. In some embodiments, incident resolution may involve more complex principles. For example, a predetermined principle may incorporate customer satisfaction metrics, such as verifying that the resolution met or exceeded customer expectations.

[0034] Additionally or alternatively, resolution criteria may be defined by a time threshold, in which case an incident may be considered resolved if a specified period of time has elapsed without further action or recurrence of the problem. For example, in an automated system that monitors operational stability, an incident may be considered resolved if no errors or alerts are triggered within a defined observation window, such as 72 hours after the last corrective action.

[0035] In some embodiments, the process flow data 102 may further include data associated with an event or activity corresponding to the incident. The event or activity may include any action, change, or occurrence that takes place in relation to the incident. In some embodiments, the event may include an action taken to address the incident, such as a diagnostic test, corrective repair, or stakeholder communication.

[0036] In some embodiments, events and / or activities associated with an incident may move the incident toward or toward resolution. Additionally or alternatively, activities and / or events may not affect whether an incident moves toward or toward resolution; instead, activities or events may describe events that may have occurred in connection with the incident.

[0037] For example, an incident may include a technical problem related to a software program. Continuing this example, events or activities associated with an incident may include performing system diagnostics, applying software patches, and / or replacing defective hardware components.

[0038] As an additional example, an incident could include customer-related issues, such as a problem a customer may have had during a call with a representative. Continuing this example, an event and / or activity could include contacting the customer for additional details, escalating the issue to a specialist team, and / or issuing a refund.

[0039] In some embodiments, an event may further include a state change, which may indicate a transition in the status or condition of an incident. In some embodiments, a state change can provide a higher level view of progress, such as moving from an “open” state to an “in progress” or “resolved” state. For example, a technical issue initially flagged as “unresolved” might transition to “diagnosed” after root cause analysis, and then to “resolved” once a fix is ​​implemented. Similarly, as another example, a customer complaint might progress from “submitted” to “under consideration,” and then to “closed” after the issue has been addressed to the customer’s satisfaction.

[0040] In some embodiments, a state change may be accompanied by one or more events. Additionally or alternatively, a state change may not be accompanied by an event. For example, an incident may transition from a “pending” state to a “terminated” state due to inactivity.

[0041] While some embodiments are described with reference to exemplary industries, completed process flow data 102, including, for example, incidents, events, activities, and state changes, can occur in a variety of industries, fields, and / or environments, including, but not limited to, technical systems, business processes, security contexts, and customer interactions.

[0042] In some embodiments, completed process flow data 102 may be associated with one completed process flow, for example, one process flow describing a single resolved incident. In some embodiments, completed process flow data 102 may include data associated with multiple completed process flows, for example, a database that stores data associated with multiple completed process flows. In some embodiments, completed process flow data 102 may be sent to and / or retrieved by a process flow pipeline 104.

[0043] The process flow pipeline 104 may include any suitable system, apparatus, or device configured to receive or otherwise acquire completed process flow data 102. In some embodiments, the process flow pipeline 104 may be a standalone system. Additionally or alternatively, the process flow pipeline 104 may be included in one or more other systems. Additionally or alternatively, the process flow pipeline 104 may instruct one or more other systems to perform operations.

[0044] In some embodiments, the process flow pipeline 104 may be configured to receive or otherwise acquire completed process flow data 102. In some embodiments, the process flow pipeline 104 may receive completed process flow data 102 from one or more other systems. Additionally or alternatively, the process flow pipeline system 104 may be configured to access and / or extract completed process flow data 102 from one or more databases and / or other data repositories.

[0045] In some embodiments, the process flow pipeline 104 may be configured to perform one or more preprocessing operations on the completed process flow data 102. In some embodiments, one or more preprocessing operations may include cleaning, transforming, and / or organizing the completed process flow data 102 into a structured format suitable for training a machine learning model 108. In some examples, preprocessing may improve data consistency and reliability and reduce errors or inconsistencies that could impair the final accuracy of the completed process flow data 102, the analysis results, or the machine learning model 108.

[0046] In some embodiments, preprocessing operations may include data cleaning, which may include identifying and correcting errors, inconsistencies, and / or incomplete entries in the completed process flow data 102. In some embodiments, data cleaning may include removing duplicate records, filling in missing values, correcting anomalies, or ensuring consistent units of measurement. In some embodiments, data cleaning may also include eliminating or reducing outliers that may distort downstream analysis. Additionally or alternatively, data cleaning may include integrating and aligning heterogeneous datasets by normalizing data formats, such as resolving schema differences, addressing duplicate or conflicting entries, and / or increasing the integration of units across sources.

[0047] In some embodiments, the preprocessing operation may include one or more transformation operations. In some examples, the transformation operation may include rescaling numerical values, encoding categorical variables, and generating derived attributes such as ratios or mean. In some embodiments, standardization and normalization techniques can improve the consistency of the completed process flow data 102 by adjusting data values ​​to a common range or distribution. In some embodiments, one or more noise reduction operations may remove irrelevant and / or redundant information, such as smoothing out fluctuations in time series data.

[0048] Additionally or alternatively, preprocessing operations may include feature selection and engineering, which may focus on identifying and constructing relevant attributes contained in the completed process flow data 102 in order to improve the training and final performance of the machine learning model 108. In some embodiments, feature selection and engineering may include creating interaction terms between variables, aggregating data over time, and filtering less useful features using statistical metrics. In these or other embodiments, preprocessing operations including the feature selection and engineering process may be further described and / or illustrated in this disclosure, for example, with respect to a process flow pipeline 204 (Figures 2-3C).

[0049] In some embodiments, the process flow pipeline 104 may be configured to generate training data 106 that can be sent to the machine learning system 108. In some embodiments, the training data 106 may include data that is structured and / or configured to train the machine learning model 108.

[0050] In some embodiments, the training data 106 may include features and / or labels that may have been generated in the process flow pipeline 104. In some embodiments, the training data 106 may include one or more features, or data representing one or more features, such as numerical representations of generated features, feature vectors, and other data structures.

[0051] Additionally or alternatively, the training data 106 may include one or more labels or other target variables. In some embodiments, the training data 106 may include data representing one or more labels or target variables that can be used as ground truth for training the machine learning model 108. In some examples, the training data 106 may be tagged or labeled, and for example, the machine learning model 108 may be trained using supervised learning. In some other examples, the training data 106 may not be tagged and / or preprocessed, and for example, the machine learning model 108 may not use supervised learning. In some embodiments, the training data 106 and the generation of the training data 106 may be further described and / or illustrated in this disclosure, for example with respect to Figures 2-5, etc.

[0052] In some embodiments, the training data 106 may be sent to the machine learning system 108.

[0053] The machine learning model 108 may include a system that can be trained to process input data and make predictions based on features and labels. The machine learning model 108 may be designed to operate using one or more algorithms such as regression, classification, or neural networks. In some embodiments, the machine learning model 108 may function independently or be integrated with other models in a larger framework. In some embodiments, the machine learning model 108 may include an architecture that includes one or more layers, nodes, and / or parameters that can be tuned based on the training process, enabling the machine learning model 108 to learn complex patterns contained in data, such as training data 106.

[0054] In some embodiments, the machine learning model 108 may include architectures and neural networks specifically tuned to make predictions about process flow. For example, recurrent neural networks (RNNs) and long short-term memory (LSTM) networks may be configured to analyze sequential data to identify patterns and trends in operational workflows. Similarly, in some embodiments, one or more reinforcement learning models may be applied to simulate decision-making within a dynamic process, thereby enabling the model / system to recommend or implement strategies that improve efficiency or minimize risk. Additionally or alternatively, the machine learning model 108 may include one or more graph neural networks (GNNs) that can be employed to model complex relationships within a process flow, such as dependencies between interconnected components or nodes.

[0055] In some embodiments, the machine learning model 108 may be configured to be trained using training data 106 which may be generated based on completed process flow data 102. In some embodiments, training may be performed according to any one or more classes of machine learning techniques, including, but not limited to, classes such as: supervised training, semi-supervised training, unsupervised training, self-learning, reinforcement learning, associative learning, transfer learning, feature learning (including principal component and cluster analysis), multilinear subspace learning, manifold learning, representation learning (including pre-dictionary learning), rule-based machine learning, anomaly detection, and any other variations or combinations thereof.

[0056] In some embodiments, the machine learning model 108 may be trained to predict the behavior associated with one or more ongoing process flows. In some embodiments, by training the machine learning model 108 with data associated with completed process flow data 102, the machine learning model 108 may be configured to produce more accurate predictions about the future behavior of the ongoing process flows than predictions made using, for example, one or more static statistical modeling methods or techniques.

[0057] In some embodiments, training the machine learning model 108 may involve exposing it to labeled data to improve its ability to make accurate predictions on new data, such as data corresponding to future ongoing process flows. In some embodiments, the training process may involve iteratively adjusting the internal parameters of the machine learning model 108 based on the difference between the predictions of the machine learning model 108 and known correct outputs associated with the training data 106, such as labels. In some embodiments, as the machine learning model 108 processes more training examples using the training data 106, it may learn to recognize patterns and relationships in completed process flow data 102, which may enable the machine learning model 108 to generalize to new inputs, such as future ongoing process data.

[0058] In some embodiments, during the training process, the machine learning model 108 may undergo internal modifications to improve its predictive ability. The training process may include adjusting the parameters, weights, and biases included in the machine learning model 108 based at least on training data 106, which includes corresponding labels and features. In some embodiments, the training and / or adjustment process may involve iteratively transforming and / or refining the internal representation and decision-making mechanism of the machine learning model 108.

[0059] In some embodiments, the machine learning model 108 may use and / or include various optimization algorithms that can function to reduce the difference between the predictions made by the machine learning model 108 and the actual target variables represented by labels. In some embodiments, these optimization processes may include techniques such as gradient descent that can iteratively update the parameters of the machine learning model 108, which can help reduce prediction errors. As training progresses, the machine learning model 108 may develop a more sophisticated internal representation of the input features and the relationship between the features and the target variables represented by labels contained in the training data 106.

[0060] In some embodiments, training the machine learning model 108 can improve its ability to generalize from the training data 106 to new examples. This improvement may be reflected in the internal structure of the machine learning model 108, which can become more refined and better suited to capturing underlying patterns and relationships within the completed process flow data 102. In some embodiments, by training the machine learning model 108, it can be better equipped to learn from the characteristics and results of completed process flows contained in the completed process flow data 102 to make accurate predictions about future ongoing process flows.

[0061] Figure 2 shows an exemplary environment 200, which includes a system designed to generate labels and features for training a machine learning model 208, according to one or more embodiments of the present disclosure. The exemplary environment 200 may include a process flow pipeline 204, which can be configured to generate training data 206 based on completed process flow data 202 for training the machine learning model 208. As described and / or illustrated in Figure 2, the completed process flow data 202, training data 206, and machine learning model 208 are the same and / or similar as the completed process flow data 102, training data 106, and machine learning model 108 further described and / or illustrated in the present disclosure, for example with respect to Figure 1.

[0062] The process flow pipeline 204 may include any suitable system, apparatus, or device configured to receive or otherwise acquire completed process flow data 202 and / or generate training data 206 for training a machine learning model 208. In these embodiments or other embodiments, the process flow pipeline 204 may be the same as and / or similar to the process flow pipeline 104 further described and / or illustrated in this disclosure, for example with respect to Figure 1.

[0063] The process flow pipeline 204, in an exemplary environment 200, includes a matching and selection module 210, a label generation module 212, a time cutoff module 214, and a feature generation module 216, each configured to perform one or more operations. In some embodiments, one or more of these modules may be implemented using hardware including one or more processors, a central processing unit (CPU), a graphics processing unit (GPU), a data processing unit (DPU), a parallel processing unit (PPU), a microprocessor (e.g., for performing or controlling the performance of one or more operations), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an accelerator (e.g., a deep learning accelerator (DLA)), and / or other processor types. In some examples, one or more of these modules may be implemented using a combination of hardware and software. In this disclosure, operations described as being performed by each module may include operations that each module can instruct the corresponding computing system to perform. In these embodiments or other embodiments, one or more of these modules may be implemented by one or more computing devices, as described in more detail with respect to Figure 6.

[0064] In some embodiments, the process flow pipeline 204 may receive and / or otherwise acquire the completed process flow data 202. One or more modules (e.g., matching and selection module 210, label generation module 212, time cutoff module 214, and feature generation module 216) may perform one or more operations on the completed process flow data 202 to generate training data 206.

[0065] For example, a module may be configured to perform one or more preprocessing operations on completed process flow data 202. These operations may include cleaning, organizing, and transforming the data to prepare it for analysis, or enabling the generation of labels and features for training one or more machine learning models. Preprocessing can improve the consistency and reliability of the data and reduce errors or inconsistencies that could impair the final accuracy of the completed process flow data 202, the analysis results, and / or the machine learning models 208. In some embodiments, preprocessing operations that may be performed by one or more modules may be further described and / or illustrated in this disclosure, for example, with respect to the process flow pipeline 104 in Figure 1.

[0066] In some embodiments, the matching and selection module 210 may be configured to perform one or more operations on the completed process flow data 202. In some embodiments, the one or more operations may include one or more matching operations and / or selection operations.

[0067] In some embodiments, the matching operation may include a process of aligning and integrating completed process flow data 202 from different sources or formats to create a more unified and consistent dataset.

[0068] In some embodiments, the matching operation may include, among other preprocessing operations, identifying relationships between heterogeneous data elements, resolving duplication, and improving compatibility across data structures. In some embodiments, the matching operation may include a process aimed at aligning and integrating data from diverse sources or formats to create a more unified and consistent dataset or multiple datasets. In some embodiments, the matching operation may include identifying and aligning relationships between heterogeneous data elements, which can improve the harmonization of data and information from various sources into a consistent structure. For example, matching may include resolving duplication by identifying and merging redundant records, standardizing data representations to ensure compatibility across different data structures, and resolving inconsistencies or conflicts within the completed process flow data 202.

[0069] In some embodiments, the matching operation may also include schema alignment, where fields or attributes from different sources are mapped into a common framework. Additionally or alternatively, the matching operation may include establishing relationships between events, activities, or other characteristics of completed process flow data202 that may be inherently connected but appear fragmented across the dataset. For example, sequential steps in a manufacturing or business workflow may be linked to create a consistent representation of the corresponding process.

[0070] In some embodiments, the matching process may improve the interoperability and usefulness of the completed process flow data 202. Additionally or alternatively, the matching operation may improve the preparation of the completed process flow data 202 for downstream tasks such as feature extraction, labeling, and / or analysis.

[0071] In addition, the matching and selection module 210 may be configured to perform one or more operations related to data selection. For example, in the context of completed process flow data 202, the selection process may include identifying and retaining important data points or features directly relevant to the task of training the machine learning model 208. For example, if the completed process flow data 102 includes task completion timestamps, machine utilization, and error logs, the matching and selection module 210 may prioritize selecting features such as average task duration, utilization trends during peak times, and error frequency under specific conditions. Continuing this example, the matching and selection module 210 may select these features because they may be among those that could affect the performance of the machine learning model 208.

[0072] In some embodiments, the selection process may reduce noise in the completed process flow data 202, for example, by reducing or eliminating redundant information. For example, if the completed process flow data 202 includes duplicate records of process completion from both manual and automated systems, the matching and selection module 210 may select automated entries that can reduce noise associated with human error.

[0073] Additionally or alternatively, the matching and selection module 210 may be configured to filter irrelevant data, such as process flow data for unrelated workflows or tasks. In some embodiments, filtering and removing irrelevant data may increase the likelihood that the completed process flow data 202 focuses on behaviors that may be relevant to the use of the machine learning model 208.

[0074] For example, if the goal is to train a machine learning model 208 to predict delays in a particular production line, the matching and selection module 210 may be configured to exclude data from other production lines or irrelevant quality control logs that do not directly affect the process flow of interest. In some examples, by curating a streamlined, task-specific dataset, the selection process performed by the matching and selection module 210 can improve the quality of the completed process flow data 210 for final training, which can lead to improved accuracy and reliability of the machine learning model 208 that can be trained using the resulting data. In some embodiments, the matching and selection module 210 may prepare the completed process flow data 202 for one or more additional operations, such as feature and / or label generation, which may be performed by a label generation module 212, a time cutoff module 214, and / or a feature generation module 216.

[0075] The label generation module 212 may be configured to generate and / or determine one or more labels corresponding to completed process flow data 202. In the context of completed process flow data 202, one or more labels may represent the outcome or characteristics of the associated process that can serve as target variables for training the machine learning model 208. In some embodiments, the labels generated using the label generation module 212 may be important to enable the machine learning model 208 to learn patterns in the process flows associated with the completed process flow data 202 and to make predictions about similar data corresponding to one or more future ongoing process flows.

[0076] For example, one or more labels that may be derived from completed process flow data 202 may include one or more classifications, such as binary classifications, such as whether a particular process flow resulted in “successful completion” or “process failure.” In some embodiments, the labels may include multiclass labels that can classify the process flow into states such as “on time,” “delayed,” or “stopped due to unavailable resources.”

[0077] In some embodiments, the labels may include numerical labels that can quantify one or more metrics associated with the completed process flow data 202. For example, the labels may include the total time taken to complete the process, the number of errors or interruptions encountered, or the workflow efficiency score. As another example, in the context of an assembly line, the forecast target may be an estimated time to resolve the disruption, expressed as a continuous or stepped continuous value based on an analysis of the corresponding completed process flow data 202.

[0078] In some embodiments, the label generation module 212 may directly use raw data from the completed process flow data 202. Additionally or alternatively, the label generation module 212 may determine, generate, and / or synthesize labels through one or more computational and / or analytical operations that can be performed on the data contained in the completed process flow data 202. For example, a label indicating "average task efficiency" may be determined from start and end timestamps across events associated with the completed process flow data 202.

[0079] In some embodiments, the label generation module 212 may be configured to generate one or more labels based on content used to train a machine learning model 208 to recognize or predict. For example, in the context of completed process flow data 202 including data associated with rental cars, each incident may be defined as a single trip in the rental car, and “Total Trip Time” may be a label that can be generated using the label generation module 212. Continuing this example, Total Trip Time may be a characteristic of each incident that the machine learning model 208 can be trained to predict. Thus, the label generation module 212 may generate a label associated with Total Trip Time, for example, to train the machine learning model 208 to predict Total Trip Time in a future similar ongoing process flow.

[0080] In some embodiments, the label generation module 212 may be configured to perform one or more technical processes to generate labels from completed process flow data 202. In some embodiments, these technical processes may include identifying key characteristics, outcomes, or resolution types that can serve as meaningful target variables for a machine learning model 208. In some examples, one or more processes involved in label generation may utilize natural language processing techniques to extract relevant information from text descriptions and may also perform statistical analysis of numerical data associated with the incident. Additionally or alternatively, domain-specific knowledge and predetermined classifications may be incorporated into one or more processes involved in label generation to ensure that the labels accurately represent key aspects of the incident resolution process. These generated labels may then be used, along with the extracted features, to train a machine learning model 208 to predict the future behavior of the ongoing process flow.

[0081] For example, one such process might involve analyzing incident resolution details corresponding to completed process flow data 202 to identify one or more key characteristics that can serve as target variables for a machine learning model 208. In some embodiments, these characteristics may be extracted from various data fields associated with the completed process flow data 202, such as incident type, resolution time, resources used, or customer satisfaction metrics. In some embodiments, the extracted characteristics may be encoded into a standardized format suitable for machine learning, such as one-hot coding of categorical variables or normalization of numerical values.

[0082] Another technical process in label generation may include a temporal analysis of completed process flow data 202. This temporal analysis may involve examining the timeline of events within the incident resolution process to identify milestones or decision points. In some embodiments, these time markers may be used to create time-based labels, such as whether an incident was resolved within a specific time threshold or whether a particular action was performed within a given time window. The resulting time-based labels may be added to a training dataset to provide a temporal context for a machine learning model 208 to predict future behavior in the ongoing process flow.

[0083] In some embodiments, the label generation module 212 may be configured to communicate, transmit, or otherwise package the generated labels together with the training data 206. Additionally or alternatively, one or more other systems may be configured to generate labels using the training data 206. For example, in a context of generating labels corresponding to one or more customer service processes, one or more of the labels may include "customer satisfaction," which can be self-reported by one or more customers independently of the process flow pipeline 204 and / or the label generation module 212. Additionally or alternatively, the label generation module 212 may be configured to instruct one or more other systems to generate, package, and / or communicate labels associated with the training data 206.

[0084] The time cutoff module 214 may be configured to sort, filter, and / or limit the data contained in the completed process flow data 202 from which features can be generated. In some embodiments, the time cutoff module 214 may extract a subset of the data contained in the completed process flow data 202. In some embodiments, the subset of data may be associated with one or more events contained in the completed process flow, one or more events which may occur within a predetermined time range cutoff. In some embodiments, the predetermined time range cutoff may be defined as a subset of the total time from the start of an incident to the resolution of the incident.

[0085] For example, in the context of a rental car where an incident corresponds to the rental period from the start of the rental until the return of the rental car, the total rental period (e.g., from pickup to drop-off) could be 7 days. Continuing this example, the completed process flow data 202 associated with the incident might include data associated with the total rental period of 7 days. Further continuing this example, the time cutoff module 214 could be configured to extract a subset of data that may correspond to the first 3 days of the total rental period corresponding to the rental period.

[0086] In some embodiments, the time cutoff module 214 may be configured to extract a subset of data based on a time range of interest. Additionally or alternatively, the time cutoff module 214 may be configured to extract a subset of data based on the number of events, activities, stages, etc. included in the completed process flow data, the number of events, activities, stages, etc. being less than the total number of events, activities, stages, etc. that could correspond to the completed process flow.

[0087] In some embodiments, the time cutoff module 214 may extract multiple subsets of data contained in completed process flow data 202. In some embodiments, the time cutoff module 214 may extract multiple subsets of data associated with a single completed process flow data 202. In some embodiments, the time cutoff module 214 may extract a subset of data associated from the start of an incident up to a specific cutoff time.

[0088] For example, again in the context of a process flow associated with a rental car, where the incident described by the process flow is the rental period, multiple subsets of data can be extracted using the time cutoff module 214. Continuing this example, the time cutoff module 214 may be configured to extract data corresponding to each subsequent day of the rental period, e.g., data corresponding to the first day, second day, third day, etc., of the rental period. Additionally or alternatively, the time cutoff module 214 may extract data corresponding to multiple shares of the total time. For example, in the same context of a process flow associated with a 7-day rental period, the time cutoff module 214 could extract data associated with the first day of the rental period, the first two days of the rental period, the first three days of the rental period, etc.

[0089] In some embodiments, a predetermined time range cutoff or a plurality of predetermined time range cutoffs may be determined based on what the machine learning model 208 can be trained to perform. For example, the machine learning model 208 may be trained to predict the behavior of a process flow that has reached a particular time range, event range, and / or activity range cutoff. For example, the machine learning model 208 may be trained to predict the behavior associated with a process flow that has reached a particular time range cutoff, such as one day into the rental period for a rental car.

[0090] In some embodiments, a subset of data that may represent events, activities, stages, etc., associated with a process flow can be extracted, and the extracted subset of data can be used to generate one or more features that can be used to train a machine learning model 208. In some embodiments, one or more features can be generated using a feature generation module 216.

[0091] The feature generation module 216 may be configured to generate one or more features corresponding to the completed process flow data 202 using one or more subsets of the data extracted by the time cutoff module 214. In some embodiments, the features may represent measurable or identifiable attributes derived from the extracted data of the completed process flow data 202. In some embodiments, the generated features may function as building blocks for downstream analysis or predictive models. For example, the features may include statistical summaries (e.g., mean, standard deviation), categorization, time markers, or any other characteristics that convey relevant information about the process flow.

[0092] In some examples, features may be represented as numerical or categorical values ​​within a structured dataset. In some embodiments, features may correspond to aspects, characteristics, or dimensions of the underlying data and may be organized in the form of feature vectors. In the context of completed process flow data 202, the feature generation module 216 may transform a subset of the data extracted by the time cutoff module 214 into a set of feature vectors. Each feature vector may encode and / or encapsulate multiple characteristics or attributes corresponding to the subset of data extracted by the time cutoff module 214, transforming complex, often unstructured, data into a format that can be configured for ingestion and / or analysis by a machine learning model 208. In some embodiments, feature vectors may encode information such as timestamps, process duration, operational state, or any other relevant attributes of the process flow.

[0093] In some embodiments, feature vectors can capture relevant information in a compact and standardized form. Representing data as vectors can reduce the dimensionality and complexity of the input, potentially improving the efficiency and performance of the learning process. In some embodiments, feature vectors can also enable the machine learning model 208 to identify patterns, correlations, and relationships between different attributes, which may be essential for making accurate predictions or classifications. Additionally or alternatively, the use of feature vectors can facilitate the application of various mathematical operations and distance metrics, which may be fundamental to many machine learning algorithms that may be used in the machine learning model 208.

[0094] The feature generation module 216 may be configured to generate one or more features in a wide variety of ways and using various methods, but this disclosure describes in more detail some examples of feature generation. Some examples of feature generation methods may include direct methods, time delta methods, calendar-based methods, duration-based methods, activity-based methods, and / or state graph-based methods.

[0095] In some embodiments, one or more direct methods for feature generation may be employed. In some embodiments, a direct method for feature generation may include directly extracting attributes from completed process flow data 202. This approach allows features to be derived directly from the data by focusing on key characteristics inherent in the dataset. For example, events, activities, stages, etc. associated with a subset of data from which features can be generated may be classified based on common characteristics shared among the events, activities, stages, etc. Continuing this example, the most common events, activities, stages, etc. may be recorded as features. Additionally or alternatively, the most common events, activities, stages, etc. based on their proportion of the total number of events, activities, stages, etc. may be recorded as features.

[0096] In some embodiments, one or more time delta methods for feature generation may be employed. In some embodiments, the time delta method for feature generation may involve deriving features based on calculating the difference or interval between temporal events in a subset of data contained in the completed process flow data 202. In some embodiments, the time delta method may highlight one or more temporal relationships between events or other data points, which may capture the dynamic aspects of the subset of data contained in the completed process flow data 202.

[0097] In some embodiments, one or more time delta methods can calculate the elapsed time between consecutive events, the duration of events, and / or the intervals between events and / or stages. Additionally or alternatively, time delta values ​​may be used to construct feature vectors that can provide insights into the timing, ordering, or pacing of actions.

[0098] In some embodiments, one or more calendar-based methods for feature generation may be employed. In some embodiments, the calendar-based method for feature generation may include generating features from a subset of data contained in completed process flow data 202 that can describe time patterns associated with specific calendar attributes such as day of the week, month, quarter, or season. In some examples, calendar-based feature generation may be useful for capturing seasonality or recurring trends in process flows that may be influenced by time-dependent factors. For example, features may be generated to reflect whether a process occurred on a weekday or weekend, during a holiday season, or in a particular accounting quarter. Additional calendar-based features may include a numerical representation of a month (e.g., 1 for January, 2 for February), or a binary indicator that signals whether an event coincides with a business peak season.

[0099] In some embodiments, by incorporating seasonal and temporal contexts, calendar-based feature generation methods may enable the identification of patterns that can correlate with external time-based influences such as market demand fluctuations, workforce deployment cycles, or environmental factors. Additionally or alternatively, calendar-based feature generation methods may consider cyclical behaviors included in process flows.

[0100] In some embodiments, one or more duration-based methods for feature generation may be employed. In some embodiments, a duration-based feature generation method may include deriving features by determining the length of time associated with a particular event, activity, stage, or transition within a process. In some embodiments, a duration-based method may focus on quantifying how long various events associated with a particular process flow may take.

[0101] For example, features may include the total duration of a process, the time spent at each event, or the elapsed time between specific events and / or stages. In some cases, by generating features based on duration, this method can highlight or determine bottlenecks, inefficiencies, and / or variability in the timing between events, activities, state changes, etc., in a process flow. Additionally or alternatively, duration-based features can help identify patterns or correlations between process duration and external factors such as resource allocation or workload.

[0102] An example flowchart illustrating an exemplary implementation of a duration-based method for generating one or more features may be described and / or illustrated with respect to Figure 3A.

[0103] Figure 3A shows a flowchart of an exemplary method 300 for generating duration-based features as input to a machine learning model, according to one or more embodiments of the present disclosure. Method 300 can be carried out by any suitable element of the system, such as the process flow pipeline 104 in Figure 1, the process flow pipeline 204 in Figure 2, and / or the feature generation module 216. Although shown as separate steps, the various blocks of Method 300 may be divided into additional blocks, combined into fewer blocks, or deleted, depending on the desired implementation. Furthermore, the execution order of the different steps may vary depending on the required implementation.

[0104] An exemplary method 300 may include block 302, which may include generating a time-sorted list of events contained in a set of data. In some embodiments, the generated list may include a sequential grouping of events that may have occurred during a particular incident. In some embodiments, the list may include a sequential grouping of events that may have been contained in a subset of data associated with a completed process flow. In some embodiments, for example, the time-sorted list of events may include each event corresponding to a particular incident up to a cutoff time, such as the one described with respect to Figure 2.

[0105] In some embodiments, the exemplary method 300 may further include a block 304 that creates an empty list of time-delta features. In some embodiments, the generated empty list may be generated so that one or more duration-based features are populated.

[0106] In some embodiments, exemplary method 300 may further include block 306. In block 306, it may be determined whether the incident associated with the time-sorted list of events is the last incident in the set of data. If it is determined that the incident associated with the time-sorted list of events is the last incident in the set of data, method 300 may proceed to block 320 and the method may terminate. If it is determined that the incident associated with the time-sorted list of events is not the last incident in the set of data, method 300 may proceed to block 308.

[0107] In block 308, events associated with state changes included in the data set may be filtered based on their corresponding timestamps. Once the timestamps corresponding to each event associated with a state change in the process flow have been determined, method 300 can proceed to block 310.

[0108] In block 310, the time delta between consecutive events associated with a state change in the process flow may be calculated and / or determined by other means. Once the time delta between each consecutive event associated with the state change has been determined or calculated, the method may proceed to blocks 312 and / or 314.

[0109] In block 312, the maximum time delta between events associated with a state change may be determined. In some embodiments, the maximum time delta between events associated with a state change may determine which state the corresponding incident remained in the longest. In some embodiments, the state associated with the maximum time delta may be recorded as a feature. Additionally or alternatively, the maximum time delta between events associated with a state change may also be recorded as a feature. In some embodiments, a feature that may correspond to the maximum time delta between events may improve the identification of bottlenecks, outliers, and / or periods of inactivity in a particular process flow. In some embodiments, method 300 may proceed to block 318.

[0110] Returning to block 314, in block 314, a cumulative time delta may be calculated for each event associated with a state change. Block 312 may correspond to determining the time delta between events associated with a state change, while block 314 identifies the time delta between a timestamp associated with the start of a particular process flow and each event associated with the state change. In some embodiments, determining the cumulative time delta between the start of an incident and each event associated with the state change may improve the tracking of overall progress and / or understanding how time may be distributed across a sequence of events. In some embodiments, once the cumulative time delta has been calculated for each event associated with a state change, method 300 may proceed to block 316.

[0111] In block 316, the maximum time delta and the corresponding event may be selected and / or recorded as one or more features. In some embodiments, the maximum cumulative time delta may be selected and recorded as a feature. Additionally or alternatively, a state associated with the maximum cumulative time delta may be selected and recorded as a feature. In some embodiments, once a feature associated with the cumulative time delta has been selected and recorded, the decision can proceed to block 318.

[0112] In block 318, one or more of the features determined and / or calculated in method 300 may be included in the list of time delta features. In some embodiments, one or more features may include a state associated with the maximum time delta, a maximum time delta between events associated with a state change, a maximum cumulative time delta, and / or a state associated with the maximum cumulative time delta, and may be added to the list of time delta features.

[0113] In some embodiments, after adding a feature to the list of time delta features, method 300 can proceed to block 306. If it is determined that the incident associated with the time-sorted list of events is the last incident in the set of data, method 300 can proceed to block 320 and the method may terminate. If it is determined that the incident associated with the time-sorted list of events is not the last incident in the set of data, method 300 can proceed to block 308, where events associated with another incident may be filtered, and method 300 may continue iterating until each incident has been analyzed.

[0114] Modifications, additions, or omissions may be made to Method 300 without departing from the scope of this disclosure. For example, the outlined steps and operations are provided only as examples, and some of the steps and operations may be optional, combined with fewer steps and operations, or extended to additional steps and operations without impairing the essence of the disclosed embodiments.

[0115] Returning to Figure 2, the feature generation module 216 may be further configured to generate one or more features using an activity-based method. An activity-based method for feature generation may involve deriving features from completed process flow data 202 or a subset of completed process flow data 202 by analyzing events contained in a process flow or part of a process flow. In some embodiments, the activity-based method may focus on capturing the occurrence, frequency, sequence, or type of events that may define a process flow or part of a process flow. For example, features may be generated to reflect the total number of events, the possible order in which events occurred, or the relative frequency of one event compared to other events. Additional features may include indicators of whether a particular critical or rare event may have occurred.

[0116] An example flowchart illustrating an exemplary implementation of an activity-based method for generating one or more features may be described and / or illustrated with respect to Figure 3B.

[0117] Figure 3B shows a flowchart of an exemplary method 325 for generating activity-based features as input to a machine learning model, according to one or more embodiments of the present disclosure. Method 325 may be carried out by any suitable elements of the system, such as the process flow pipeline 104 in Figure 1, the process flow pipeline 204 in Figure 2, and / or the feature generation module 216. Although shown as separate steps, the various blocks of Method 325 may be divided into additional blocks, combined into fewer blocks, or deleted, depending on the desired implementation. Furthermore, the execution order of the different steps may vary depending on the required implementation.

[0118] An exemplary method 325 may include a block 322 that generates a time-sorted list of events contained in a set of data, for example, a subset of completed process flow data 102 and / or process flow data 202 described and / or illustrated with respect to Figures 1 and 2. In some embodiments, the generated list may include a sequential grouping of events that may have occurred during a particular incident. In some embodiments, the list may include a sequential grouping of events that may have been contained in a subset of data associated with a completed process flow.

[0119] In some embodiments, exemplary method 325 may further include block 324 which creates an empty list of activity-based features. In some embodiments, the generated empty list may be generated so that one or more activity-based features are populated.

[0120] In some embodiments, exemplary method 325 may further include block 326. In block 326, method 325 may determine whether the incident associated with the time-sorted list of events is the last incident included in the set of data. If it is determined that the incident associated with the time-sorted list of events is the last incident included in the set of data, method 325 may proceed to block 348 and terminate. If it is determined that the incident associated with the time-sorted list of events is not the last incident included in the set of data, method 325 may proceed to block 328.

[0121] In block 328, the timestamp corresponding to the first event associated with the time-sorted list of events may be set to zero. In some embodiments, the first event does not have to be set to zero; instead, the first event may be recorded or set at any timestamp, insofar as the first event is noted or otherwise set as a starting point for an activity-based feature generation method. In some embodiments, method 325 may proceed to block 330 or block 338.

[0122] In block 330, all events associated with a particular incident may be sorted and / or assigned a corresponding timestamp. In some embodiments, the timestamp assigned to each event may be based on a set timestamp corresponding to a first event.

[0123] In block 332, time from zero can be calculated for all events associated with a particular incident. In some embodiments, the determined time can be determined from a set timestamp of a first event associated with the particular incident.

[0124] In block 334, time from zero or time from a set timestamp of the first event may be recorded. In some embodiments, time from zero may be recorded together with the corresponding event.

[0125] In block 336, the most active period can be determined. In some embodiments, the most active period can be determined by the number of events that may occur within a set period. For example, the set period may be a 24-hour period. Continuing this example, the most active period may be characterized as a 24-hour period containing the maximum number of events. In some embodiments, the most active period can be determined by the number of events having shared characteristics that occur within a set period. For example, the most active period may be defined by a set period containing the maximum number of events corresponding to state changes associated with an incident.

[0126] In some embodiments, both the number of events included in the most active period and the number of events as a percentage of the total number of events occurring during the incident may be individual features. In some embodiments, both the number of events corresponding to a state change and the states before and after the most active state change event may be determined to be features. In some embodiments, each of these features may be included in one or more feature vectors that can be used to train a machine learning model 208.

[0127] In some embodiments, the most active period of an event may be determined to be one or more features associated with the incident.

[0128] In block 338, the least active period of time may be determined. In some embodiments, similar to the determination of the most active period, the least active period may be determined as the set period with the fewest number of events, which can function as a characteristic associated with the incident. In some embodiments, the least active period may be determined by the absence of events with shared characteristics that occur within the set period. For example, the least active period may be defined as a set period containing the fewest events corresponding to the state changes associated with the incident.

[0129] In block 340, the features determined in blocks 336 and 338 may be added to the list of activity features. Once features are added to the list of activity features, method 325 can proceed to block 326.

[0130] In block 326, method 325 may determine whether the incident associated with the time-sorted list of events is the last incident in the set of data. If it is determined that the incident associated with the time-sorted list of events is the last incident in the set of data, method 325 may proceed to block 348 and terminate. If it is determined that the incident associated with the time-sorted list of events is not the last incident in the set of data, method 325 may proceed to block 328, where events associated with other incidents may be filtered, and method 325 may continue iterating until each incident has been analyzed.

[0131] Modifications, additions, or omissions may be made to Method 325 without departing from the scope of this disclosure. For example, the outlined steps and operations are provided only as examples, and some of the steps and operations may be optional, combined with fewer steps and operations, or extended to additional steps and operations without impairing the essence of the disclosed embodiments.

[0132] Returning to Figure 2, the feature generation module 216 may be further configured to generate one or more features using a state graph-based method. In some embodiments, the state graph-based method for feature generation may involve deriving features from completed process flow data 202 or a subset of data contained in completed process flow data 202 by representing the process as a graph of interconnected states and transitions. In some embodiments, the state graph-based method may focus on capturing the structural and relational aspects of the process, where nodes in the graph may represent unique states, and one or more edges between nodes may represent transitions or events that result in state changes corresponding to the process flow.

[0133] In some embodiments, features generated through a state graph-based method may include metrics such as the number of visited states, the frequency of specific transitions, common transition paths, and / or identification of critical nodes (e.g., bottleneck states or high-frequency states). Additionally or alternatively, one or more features may include graph-based properties such as path length, state transition probability, or network-centric criteria that may reflect connectivity and flow associated with the process flow.

[0134] For example, in the context of a customer service process flow represented as a state graph, the number of states visited can correspond to various stages that a customer interaction goes through, such as "Initial Inquiry," "Agent Assignment," "Problem Resolution," and "Feedback Collection." Continuing this example, the frequency of a particular transition can capture how often a customer moves from "Initial Inquiry" to "Problem Resolution" without going through "Agent Assignment," potentially demonstrating the effectiveness of a self-service channel. Furthermore, common transition paths can reveal that most interactions follow a sequence from "Initial Inquiry" to "Agent Assignment" and then to "Problem Resolution," while the identification of critical nodes can highlight "Agent Assignment" as a bottleneck where delays may occur. In some examples, graph-based properties such as path length can measure the average number of transitions that may occur before problem resolution. In some examples, one or more network-centric criteria can help identify one or more states that may play a significant role in the overall process flow.

[0135] An example flowchart illustrating an exemplary implementation of a state graph-based method 350 for generating one or more features may be described and / or illustrated with respect to Figure 3C.

[0136] Figure 3C shows a flowchart of an exemplary method 350 for generating graph-based features as input to a machine learning model, according to one or more embodiments of the present disclosure. Method 350 can be implemented by any suitable element of the system, such as the process flow pipeline 104 in Figure 1, the process flow pipeline 204 in Figure 2, and / or the feature generation module 216. Although shown as separate steps, the various blocks of Method 350 may be divided into additional blocks, combined into fewer blocks, or deleted, depending on the desired implementation. Furthermore, the execution order of the different steps may vary depending on the required implementation.

[0137] An exemplary method 350 may include a block 352. Block 352 generates a time-sorted list of events that are included in a set of data, for example, a subset of completed process flow data 102 and / or process flow data 202 described and / or illustrated with respect to Figures 1 and 2. In some embodiments, the generated list may include a sequential grouping of events that may have occurred during a particular incident. In some embodiments, the list may include a sequential grouping of events that may have been included in a subset of data associated with a completed process flow.

[0138] An exemplary method 350 may include a block 354 that generates a list of graph features contained in a set of data, for example, a subset of completed process flow data 102 and / or process flow data 202 described and / or illustrated with respect to Figures 1 and 2. In some embodiments, the generated list may include a sequential grouping of events that may have occurred during a particular incident. In some embodiments, the list may include a sequential grouping of events that may have been contained in a subset of data associated with a completed process flow.

[0139] In some embodiments, exemplary method 350 may further include block 356. In block 356, it may be determined whether the incident associated with the time-sorted list of events is the last incident in the set of data. If it is determined that the incident associated with the time-sorted list of events is the last incident in the set of data, method 350 may proceed to block 384 and terminate. If it is determined that the incident associated with the time-sorted list of events is not the last incident in the set of data, method 350 may proceed to block 358.

[0140] In block 358, events associated with state changes included in the data set may be filtered based on their corresponding timestamps. Once the timestamps corresponding to each event associated with a state change in the process flow have been determined, method 350 can proceed to block 360.

[0141] In block 360, a list of edges may be calculated or determined by other means. In this context, edges can represent connections or relationships between nodes in a graph, where nodes correspond to entities, states, events, activities, etc. In some embodiments, edges can specifically indicate relationships or dependencies between events that are part of an incident, such as causal links, temporal sequences, or hierarchical associations. In some embodiments, nodes may represent and / or correspond to only one or more states and / or state changes. In these embodiments, edges can specifically indicate relationships or dependencies between states or state changes. For example, edges may represent a progression from one event to another on a timeline, or a dependency between two related events.

[0142] In some embodiments, the determined edges may provide a structural framework for analyzing incidents as a graph, which may enable the identification of patterns, correlations, or anomalies within interconnected events. Once a list of edges is generated, method 350 can proceed to one or more of blocks 362, 364, and 376, each of which will be described in turn.

[0143] In block 362, features may be computed directly or determined by other means. In some embodiments, features may be derived directly from the structure and properties of the graph, represented by a list of edges. In some embodiments, these features may be computed using a computational tool such as NetworkX. Examples of direct features include node degree (the number of connections a node has), edge weights (representing the strength or frequency of relationships), clustering coefficients (indicating the tendency of nodes to form densely connected groups), and graph centrality metrics (such as mediation or proximity, which quantify the importance of nodes in the graph). In some embodiments, the generated features may be added to a list of graph features in block 382.

[0144] In block 364, a directed graph may be created or otherwise generated. In some embodiments, the directed graph may be constructed using a list of edges, each edge having an assigned direction that can represent a flow or sequence of relationships between events, states, activities, etc., which can be considered “nodes”. In some embodiments, the effective properties of the directed graph may enable the representation of asymmetric relationships such as cause-and-effect links, hierarchical dependencies, or temporal progression. For example, in the context of an incident, the directed graph may capture a sequence of events corresponding to the incident. In some embodiments, a directed graph can be generated using a tool such as NetworkX or a similar graph library, which can incorporate attributes such as edge weights, timestamps, or other labels.

[0145] In block 366, one or more components associated with a directed graph may be determined. In some embodiments, a component may refer to a node or characteristic in the directed graph that exhibits a particular structural or relational property. For example, strongly connected nodes may be identified, in which case all nodes in the subgraph are reachable from all other nodes and reflect tightly interconnected relationships. Additionally or alternatively, weakly connected nodes may be determined, in which case nodes are connected via edges without regard to direction. In some embodiments, both strongly connected and weakly connected nodes provide valuable insights into graph organization and clustering, such as identifying isolated processes, interdependent events, or key segments of an incident.

[0146] In block 370, one or more subgraphs may be generated based on strongly connected nodes. In some embodiments, one or more subgraphs may be created to isolate a particular region of interest within the graph, such as the strongly connected nodes identified in block 366. In some embodiments, one or more subgraphs may be particularly useful for analyzing localized interactions, detecting self-contained workflows, or identifying segments of systems operating independently.

[0147] In block 372, the diameter, radius, and maximum value may be determined based on one or more subgraphs. In some embodiments, the diameter, radius, and maximum value may provide one or more features based on one or more subgraphs. In some embodiments, the diameter may represent the longest shortest path between any two nodes in the subgraph and provide a measure of its maximum reach or range. In some embodiments, the radius may be the shortest and maximum distance from a single node to all other nodes in the subgraph and identify the most centrally located node or "center". In some embodiments, the maximum value may refer to the maximum value of a particular graph property, such as node degree, edge weight, or another feature of interest. In some embodiments, the diameter, radius, and maximum value may be features that can be added to the list of graph features in block 382.

[0148] In block 376, an empty list of visited edges may be created. The generation of the empty list of visited edges can be performed using a data structure such as an array, a linked list, or a set. This data structure may be configured to store information about the edges traversed in the graph-based feature selection process. In some embodiments, the empty list of visited edges may be initially created without any elements, allowing visited edges to be added later as the graph traverse progresses. In some embodiments, the empty list of visited edges may include, for example, one or more data structures such as dynamic arrays, linked lists, and / or other set data structures.

[0149] In some embodiments, the empty list of visited edges may be configured to support a variety of actions, including, for example, adding newly visited edges to the list, determining whether a particular edge has already been visited, and deciding to remove an edge from the list if backtracking is performed by a graph traversal algorithm. In some embodiments, the data structure corresponding to the empty list of visited edges may also be designed to store additional metadata about each visited edge, such as visit time or any associated weights or attributes. In some examples, the additional metadata may be useful for more complex graph-based feature selection algorithms, for example, which may consider one or more edge properties in a decision-making process.

[0150] In some embodiments, creating an empty list of visited edges may also be done to efficiently use memory, especially in the case of large graphs with a large number of edges. In some embodiments, a bit vector and / or Bloom filter can be used to compactly represent the set of visited edges.

[0151] In block 378, edges can be traversed. In some embodiments, traversing edges may involve exploring connections between nodes in the graph in order to collect relevant information or to perform specific tasks such as pathfinding or connectivity analysis. In some embodiments, during traversing, each edge may be followed from its source node to its target node, and the traversing may proceed in a specific order, such as depth-first or width-first.

[0152] In block 380, visited edges may be added to the list of visited edges. In some embodiments, each edge included in the graph may be visited and added to the list of visited edges. In some embodiments, the longest unique path length may be calculated or determined by creating the list of visited edges. In some embodiments, the longest unique path may be a feature corresponding to the graph associated with the incident, which may be added to the list of graph features in block 382.

[0153] In block 382, ​​the determined graph features may be added to the list of graph features. Adding a feature to the list of graph features allows method 350 to proceed to block 356. In block 356, if it is determined that the incident associated with the list of graph features is the last incident, method 350 may proceed to block 384 and terminate. In block 356, if it is determined that the incident associated with the list of graph features is not the last incident, method 350 may proceed to block 358, where events associated with other incidents are filtered, and method 350 may continue iterating until each incident has been analyzed.

[0154] Modifications, additions, or omissions may be made to Method 350 without departing from the scope of this disclosure. For example, the outlined steps and operations are provided only as examples, and some of the steps and operations may be optional, combined with fewer steps and operations, or extended to additional steps and operations without impairing the essence of the disclosed embodiments.

[0155] Returning to Figure 2, features generated using the feature generation module 216 and / or labels generated using the label generation module 212 may be included in the training data 206. In some embodiments, the training data 206 may be used to train the machine learning model 208. In some embodiments, labels and features generated using completed process flow data 202 may provide valuable input for training the machine learning model 208 to predict the behavior of the process flow in progress. The labels may represent various characteristics of the resolved incident, such as resolution time, severity level, or resource requirements. These labels may serve as target variables that the machine learning model 208 can learn to predict based on the input features. In some examples, accurate labels generated from completed process flow data 202 may improve the effectiveness of the loss function included in training the machine learning model 208, enabling more robust performance of the machine learning model 208. In some embodiments, by using precise labels, the training algorithm used to train the machine learning model 208 can more effectively improve the parameters included in the algorithm corresponding to the machine learning model 208, which can result in a reduction in the prediction error of the machine learning model 208 when predicting behavior associated with the ongoing process flow and an improvement in overall performance.

[0156] In some embodiments, features extracted from completed process flow data 202 may capture key attributes and patterns that indicate how an incident progresses and is resolved. In some examples, features may include metrics such as response time, number of escalations, type of action taken, or properties of the incident itself. By training on a diverse set of labeled examples with rich feature sets, the machine learning model 208 may learn to recognize complex relationships and patterns that can help predict the outcome of new ongoing incidents. As a result, in some examples, the machine learning model 208 may be better equipped to make accurate predictions about future behavior in an ongoing process flow, identify critical decision points, and / or anticipate potential bottlenecks or delays in resolving incidents associated with the ongoing process flow.

[0157] In these or other embodiments, the training data 206 may be the same as and / or similar to the training data 106, for example, further described and / or illustrated in this disclosure with respect to Figure 1.

[0158] The machine learning model 208 may be the same as and / or similar to the machine learning model 108, which may be further described and / or illustrated in this disclosure, for example with respect to Figure 1. In some embodiments, the machine learning model 208 may be configured to be trained using labels and features generated using the label generation module 212 and the feature generation module 216.

[0159] Figure 4 shows an exemplary environment 400, including a machine learning system 408 designed to generate future behavior predictions 410 associated with an ongoing process flow based on ongoing process flow data 402, according to one or more embodiments of the present disclosure. In the illustrated example, the environment 400 may include a feature generation system 404, a machine learning system 408, and a system 412. In some embodiments, the feature generation system 404 may be configured to receive ongoing process flow data 402 and generate an output 406 that can be sent to the machine learning system 408. The machine learning system 408 may be configured to generate one or more future behavior predictions 410 associated with an ongoing process flow based on the ongoing process flow data 402 and send the future behavior predictions 410 to the system 42.

[0160] The process flow data 402 in progress may include any data associated with the process flow in progress. As used in this disclosure, a process flow in progress is a description of events corresponding to an incident that has not yet been resolved. This is in contrast to a completed process flow, which is a description of events corresponding to an incident that has been resolved, as further described in this disclosure with respect to Figure 1, for example. In some embodiments, the data contained in the process flow data 402 in progress may be similar to the completed process flow data 102 and / or completed process flow data 202 described and / or illustrated in this disclosure, for example with respect to Figures 1 and 2, except that the process flow data 402 in progress corresponds to one or more incidents that have not yet been resolved.

[0161] In some embodiments, the process flow data 402 in progress may include data associated with a single process flow in progress. Additionally or alternatively, the process flow data 402 in progress may include data associated with multiple process flows in progress. In some embodiments, the process flow data 402 in progress may be transmitted to or communicated to a feature generation system 404 and / or extracted by the feature generation system 404.

[0162] The feature generation system 404 may include any suitable system, apparatus, or device configured to receive or otherwise acquire the ongoing process flow data 402. In some embodiments, the feature generation system 404 may be comprised of one or more other systems. Additionally or alternatively, the feature generation system 404 may instruct one or more other systems to perform one or more operations.

[0163] In some embodiments, the feature generation system 404 may be configured to receive or otherwise acquire ongoing process flow data 402. In some embodiments, the feature generation system 404 may receive ongoing process flow data 402 from one or more other systems. Additionally or alternatively, the feature generation system 404 may be configured to access and extract ongoing process flow data 402 from one or more databases and / or other data repositories.

[0164] In some embodiments, the feature generation system 404 may be configured to acquire ongoing process flow data 402 associated with an incident. In some embodiments, the ongoing process flow data 402 may be automatically acquired or extracted based on ongoing process flow data 402 indicating that the corresponding incident has elapsed for a predetermined period of time. For example, the ongoing process flow data 402 may include data corresponding to an incident that was ongoing for two weeks, which may be a predetermined time for extracting the ongoing process flow data 402. Continuing this example, since the predetermined time is two weeks, the ongoing process flow data 402 associated with the ongoing process flow may be automatically extracted or acquired by other means.

[0165] In some embodiments, a predetermined time for extracting the ongoing process flow data 402 may be determined or defined based on one or more machine learning models included in a machine learning system 408 that has been trained to predict the behavior of the ongoing process flow data 402.

[0166] For example, one or more machine learning models may be trained using one or more processes further described and / or illustrated in this disclosure, such as with respect to Figures 1-3C. Continuing this example, one or more machine learning models may be trained to predict the behavior of an ongoing process flow after one week has elapsed for that process flow. Continuing the example further, the feature generation system 404 may be configured to automatically extract ongoing process flow data 402 associated with an ongoing process flow when the corresponding ongoing process flow reaches a period of one week.

[0167] In some embodiments, the feature generation system 404 may be configured to perform one or more preprocessing operations on the ongoing process flow data 402. Additionally or alternatively, the feature generation system 404 may be configured to perform one or more operations on the ongoing process flow to generate one or more features 406. In these embodiments or other embodiments, the preprocessing operations and / or one or more operations for generating features 406 may be the same as and / or similar to the preprocessing operations and / or feature generation operations further described and / or illustrated in this disclosure, such as with respect to process flow pipeline 104, process flow pipeline 204, and / or corresponding operations, which may be further described and / or illustrated in this disclosure, such as with respect to Figures 1, 2, and 3A to 3C.

[0168] Feature 406 may be the same as and / or similar to features described and / or illustrated in this disclosure, for example with respect to Figures 1-3C, in terms of its generation, structure, and / or use. In some embodiments, feature 406 may be transmitted to a machine learning system 408 to generate one or more predictions of the future behavior of an ongoing process flow associated with ongoing process flow data 402.

[0169] The machine learning system 408 may include any suitable system, apparatus, or device configured to receive or otherwise acquire features 406 and generate one or more future behavior predictions 410. In some embodiments, the machine learning system 408 may include one or more machine learning models that can be configured to generate future behavior predictions 410. In some embodiments, one or more machine learning models that may be included in the machine learning system 408 may be trained using completed process flow data, e.g., completed process flow data 102 and / or completed process flow data 202, using one or more methods or operations further described and / or illustrated in this disclosure, e.g., with respect to Figures 1-3C and 5.

[0170] In some embodiments, if the machine learning system 408 includes multiple machine learning models, each machine learning model may be configured to generate future behavior predictions 410 based on features 406 generated at different times corresponding to the ongoing process flow. For example, the machine learning system 408 may include a first machine learning model configured to generate future behavior predictions 410 based on features 406 associated with a first period associated with the ongoing process flow data 402. Continuing this example, the machine learning system 408 may further include a second machine learning model configured to generate future behavior predictions 410 based on features 406 associated with a second period associated with the ongoing process flow data 402.

[0171] The future behavior forecast 410 includes data and / or information that may indicate the future behavior of the ongoing process flow. In some embodiments, indicating the future behavior of the ongoing process flow may include characteristics, trends, or patterns. In some embodiments, the future behavior forecast may include key performance indicators (KPIs), such as throughput, latency, or resource utilization, which are predicted to evolve based on the process's current trajectory. Additionally or alternatively, the future behavior forecast 410 may include identifying potential risks, delays, or bottlenecks that may affect the flow, enabling proactive adjustments or interventions. For example, the forecast may estimate the likelihood of certain events occurring, such as the process reaching a critical threshold or a component failure that could interrupt the flow. Additionally or alternatively, the future behavior forecast 410 may include a description of whether and when the ongoing process flow may be resolved. Additionally or alternatively, the future behavior forecast 410 may include a description or indication of events that have not yet occurred in the ongoing process flow.

[0172] In some embodiments, the future behavior prediction 410 may be communicated to or otherwise acquired by the system 412. The system 412 may include any suitable system, apparatus, or device configured to receive or acquire the future behavior prediction 410 and to perform one or more actions based on the future behavior prediction. In some embodiments, the system 412 may include a feature generation system 404 and / or a machine learning system 408. Additionally or alternatively, the system 412 may be separate from one or more of the feature generation system 404 and the machine learning system 408.

[0173] In some embodiments, the system 412 may be configured to perform one or more actions based on the future behavior prediction 410. For example, in the context of a customer service ticket as an ongoing process, the future behavior prediction 410 may include data and / or information indicating that a particular ticket is likely to escalate due to the nature of the inquiry or the involvement of a high-priority issue. Based on this prediction, the system 412 may automatically route the ticket to a senior support person or trigger a notification to ensure timely intervention.

[0174] As another example, in the context of an ongoing process flow describing the movement of a rental car, the future behavior prediction 410 may include data and / or information that can indicate that the vehicle may be returned late due to rental history or other conditions. In response, the system 412 may proactively adjust the rental schedule, warn the customer, or suggest alternative solutions to mitigate any operational disruption.

[0175] In some embodiments, system 412 may adjust one or more actions based on future behavior predictions 410 that adjust over time for the same ongoing process flow. For example, an incident may include a single move in the context of renting an eScooter. The move may be ongoing, for example, a passenger is en route from a starting position to an ending position, and therefore the data associated with the move may be considered ongoing process flow data. Continuing this example, a machine learning system 408 may include three machine learning models that can be configured to generate future behavior predictions 410 at different time intervals during the incident. The first machine learning model may be configured to generate future behavior predictions 410 for the ongoing process flow at 10 minutes, the second machine learning model at 15 minutes, and the third machine learning model at 20 minutes. Continuing the example further, ongoing process flow data 402 may be automatically acquired by the machine learning system 408 at 10 minutes, and the first machine learning model may generate future behavior predictions 410. The same applies to 15-minute and 20-minute intervals, and the second and third models can generate subsequent future behavior predictions 410 for the corresponding time intervals.

[0176] In some embodiments, the future behavior prediction 410 may be received by the system 412 in real time or near real time.

[0177] In some embodiments, the prediction of future behavior 410 may enable the system 412 to make informed decisions, improve resource allocation, improve customer satisfaction, etc., by predicting problems before they occur. Furthermore, the system 412 can trigger automated workflows such as initiating follow-up actions, adjusting timelines, or notifying relevant parties, thus enabling the process flow to continue smoothly and efficiently in line with the predicted future behavior 410.

[0178] Figure 5 shows an exemplary flowchart of Method 500, an exemplary method for training a machine learning model to predict behavior associated with one or more ongoing process flows, according to one or more embodiments of the present disclosure. Method 500 may be carried out by any suitable elements of a machine learning system, such as the process flow pipeline 104 in Figure 1, the process flow pipeline 204 in Figure 2, the feature generation system 404 in Figure 4, the machine learning system 408, and / or system 412. Although shown as separate steps, the various blocks of Method 500 may be divided into additional blocks, combined into fewer blocks, or deleted, depending on the desired implementation. Furthermore, the execution order of the different steps may differ depending on the required implementation.

[0179] In some embodiments, method 500 may include block 502, in which data corresponding to a completed process flow may be obtained. In some embodiments, the data corresponding to a completed process flow may be the same as and / or similar to the completed process flow data 102 and / or completed process flow data 202, which are further described and / or illustrated in this disclosure, for example with respect to Figures 1 and 2.

[0180] In these and / or other embodiments, any suitable device, apparatus, system, etc., may be used in block 502 to acquire data corresponding to the completed process flow. For example, process flow pipelines 104 and / or 204 may be further described and / or illustrated in this disclosure, for example with respect to Figures 1 and 2.

[0181] In block 504, one or more labels may be generated based on the acquired data corresponding to the completed process flow. In some embodiments, one or more labels may be used as ground truth or target variables for training one or more machine learning models, neural networks, etc. In some embodiments, the generated one or more labels may be included in training data 106 and / or training data 206, which are further described and / or illustrated in this disclosure, for example with respect to Figures 1 and 2. Furthermore, one or more labels may be generated using any suitable device, apparatus, system, etc. For example, process flow pipelines 104 and / or process flow pipelines 204 may be further described and / or illustrated in this disclosure, for example with respect to Figures 1 and 2.

[0182] In block 506, a subset of data may be extracted from data acquired in relation to a completed process flow. In some embodiments, the subset of data may include a portion of the data corresponding to a completed process flow. In some embodiments, the subset of data may be extracted based on a predetermined time range cutoff, as further described in this disclosure, for example with respect to the time cutoff module 214 in Figure 2.

[0183] In block 508, multiple features may be generated based on a subset of data. In some embodiments, the multiple feature vectors may be represented as corresponding multiple feature vectors that can be generated from a list including direct feature generation, activity-based feature generation, duration-based feature generation, time-delta-based feature generation, state graph-based feature generation, and / or calendar-based feature generation. In these or other embodiments, the multiple features may be generated using any suitable device, apparatus, system, etc. For example, process flow pipeline 104 and / or process flow pipeline 204 may be further described and / or illustrated in this disclosure, for example with respect to Figures 1 and 2. In addition, the multiple features may be determined, calculated, and / or generated using one or more methods further described in this disclosure, such as the method described with respect to the feature generation module 216, for example, with respect to Figures 2-3C.

[0184] In block 510, a machine learning model can be trained based on one or more generated labels and multiple features.

[0185] In block 512, second data corresponding to the ongoing process flow may be acquired. In some embodiments, the second data corresponding to the ongoing process flow may be the same as and / or similar to the ongoing process flow data 402 further described and / or illustrated in this disclosure, for example with respect to Figure 4.

[0186] In block 514, a second set of features may be generated based on the second set of data. In some embodiments, the second set of features may be represented as a second corresponding set of feature vectors that can be generated from a list including direct feature generation, activity-based feature generation, duration-based feature generation, time-delta-based feature generation, state-graph-based feature generation, and calendar-based feature generation.

[0187] In some embodiments, the second set of features may be generated using any suitable device, apparatus, system, etc. For example, a feature generation system 416 may be further described and / or illustrated in this disclosure, for example with respect to Figure 4. In some embodiments, the second set of features may be determined, calculated, and / or generated using one or more methods further described in this disclosure, for example, the method described with respect to the feature generation module 216 described with respect to Figures 2-4.

[0188] In block 516, a second set of features may be sent to a machine learning model. In some embodiments, the machine learning model may be trained to predict the future behavior of an ongoing process flow. In some embodiments, the machine learning model may be configured to generate predictions of future behavior based on a second set of features that may correspond to an ongoing process flow. In some embodiments, the machine learning model may be the same as and / or similar to machine learning models 108, 208, and / or 408 described and / or illustrated in this disclosure with respect to, for example, Figures 1, 2, and 4.

[0189] Modifications, additions, or omissions may be made to Method 300 without departing from the scope of this disclosure. For example, the outlined steps and operations are provided only as examples, and some of the steps and operations may be optional, combined with fewer steps and operations, or extended to additional steps and operations without impairing the essence of the disclosed embodiments.

[0190] Figure 6 shows a block diagram of an exemplary computing system 602 according to at least one embodiment of the present disclosure. The computing system 602 may be configured to perform or direct one or more appropriate operations described in the present disclosure. For example, the computing system 602 may be used in various elements of the above disclosure (e.g., the process flow pipeline 104, machine learning model 108 in Figure 1, the process flow pipeline 204, matching and selection module 210, label generation module 212, time cutoff module 214, feature generation module 216, machine learning model 208, feature generation system and / or machine learning system in Figure 4). In some embodiments, the computing system 602 may be used to control operations related to generating training data (e.g., labels and / or features for training one or more machine learning models). The computing system 602 may include a processor 650, memory 652, and data storage 654. The processor 650, memory 652, and data storage 654 may be communicatively coupled.

[0191] Typically, the processor 650 includes any suitable computer, computing entity, or processing unit including various computer hardware or software modules, and may be configured to execute instructions stored in any applicable computer-readable storage medium. For example, the processor 650 may include a microprocessor, microcontroller, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or any other digital or analog circuit configured to interpret and / or execute program instructions and / or process data. Although shown as a single processor in Figure 6, the processor 650 may include any number of processors configured to perform or direct any number of operations described herein, individually or collectively. Furthermore, one or more of the processors may reside on one or more different electronic devices, such as different servers.

[0192] In some embodiments, the processor 650 may be configured to interpret and / or execute program instructions and / or process data stored in memory 652, data storage device 654, or memory 652 and data storage device 654. In some embodiments, the processor 650 may fetch program instructions from data storage device 654 and load the program instructions into memory 652. After the program instructions are loaded into memory 652, the processor 650 may execute the program instructions.

[0193] The memory 652 and data storage device 654 may include computer-readable storage media that carry or have stored computer-executable instructions or data structures. For example, but not limited to, such computer-readable storage media may include tangible or non-temporary computer-readable media, including random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disk read-only memory (CD-ROM), or other optical disk storage devices, magnetic disk storage devices, or other magnetic storage devices, flash memory elements (e.g., solid memory devices), or any other non-temporary storage media that can be used to carry or store specific program code in the form of computer-executable instructions or data structures and can be accessed by a general-purpose or dedicated computer. In these and other embodiments, the term “non-temporary” as described herein should be considered to exclude only types of temporary media that are not included in the scope of patentable subject matter in Federal Circuit decision of In re Nuijten, 500 F.3d 1346 (Fed. Cir. 2007).

[0194] The above combinations may also be included in the scope of computer-readable storage media. Computer-executable instructions may include, for example, instructions and data configured to cause the processor 650 to perform a specific operation or a set of operations.

[0195] Modifications, additions, or omissions may be made to the computing system 602 without departing from the scope of this disclosure. For example, in some embodiments, the computing system 602 may include any number of other components, which may or may not be expressly described.

[0196] The terms used in this disclosure and in particular in the appended claims (e.g., the appended claim bodies) are generally intended to be “broad” terms (for example, the term “including” should be interpreted as “including, but not limited to,” and the term “having” should be interpreted as “having, but not limited to,” etc.).

[0197] Furthermore, where an enumeration of a specific number of introduced claims is intended, such intention is explicitly indicated in the claim; where there is no such enumeration, such intention does not exist. For example, for the sake of understanding, the claims attached below may include the use of the introductory phrases “at least one” and “one or more” to introduce an enumeration of claims. However, the use of such phrases should not be interpreted as meaning that the introduction of an enumeration of claims by the indefinite article “a” or “an” limits any particular claim containing such an introduced enumeration of claims to embodiments containing only one such enumeration, even when the same claim includes the introductory phrase “one or more” or “at least one” and the indefinite article “a” or “an” (for example, “a” and / or “an at least one” or “one or more”). In other words, the same applies to the use of the definite article used to introduce an enumeration of claims.

[0198] Furthermore, where an enumeration of claims introducing a specific number is explicitly stated, a person skilled in the art will understand that such an enumeration should be interpreted as meaning at least the number enumerated (for example, the statement “two enumerations” without other qualifications means at least two enumerations, or two or more enumerations). Furthermore, in examples where a statement similar to “at least one of A, B, and C etc.” or “one or more of A, B, and C etc.” is used, such a configuration is usually intended to include A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together, etc. Furthermore, the use of the term “and / or” is intended to be interpreted in this manner.

[0199] Furthermore, any disjunct word or phrase representing two or more alternative terms should be understood to imply the possibility of including one of the terms, either of the terms, or both of the terms, whether in the description, claims, or drawings. For example, the phrase "A or B" should be understood to include the possibility of "A" or "B" or "A and B," even if the terms "and / or" are used elsewhere.

[0200] All examples and conditional language described herein are intended for teaching purposes to help readers understand the disclosure and the concepts that the disclosure contributes to the further development of the technology, and should be construed as not being limited to such specifically listed examples and conditions. While embodiments of the disclosure are described in detail, various modifications, substitutions, and choices can be made to them without departing from the spirit and scope of the disclosure.

[0201] In addition to the embodiments described above, the following further notes are disclosed. (Note 1) A method, This involves obtaining data corresponding to a completed process flow, wherein the completed process flow is associated with a resolved incident. Based on the data corresponding to the completed process flow, generate one or more labels representing the characteristics of the incident, which will serve as target variables for training a machine learning model. Extracting a subset of the data corresponding to a predetermined time range cutoff defined within the total time required to resolve the incident, To train the machine learning model based on the subset of the data corresponding to the predetermined time range cutoff, generate a plurality of features representing identifiable attributes derived from the acquired data corresponding to the completed process flow, Training the machine learning model to predict the future behavior of the process flow in progress using the multiple features associated with the data corresponding to the completed process flow and the one or more labels. A method that includes this. (Note 2) Generating one or more labels is: Extracting label data associated with one or more characteristics in the data corresponding to the completed process flow, The label data is encoded into a format suitable for training the machine learning model. The method described in Appendix 1, including the method described in Appendix 1. (Note 3) Automatically acquire second data corresponding to the ongoing process flow based on the ongoing process flow reaching the predetermined time range cutoff, wherein the ongoing process flow is associated with an unresolved incident, and the second data occurs during the predetermined time range cutoff. To generate a second set of features associated with the second data corresponding to the predetermined time range cutoff, To generate predictions of future behavior associated with the ongoing process flow, the second set of features are transmitted to the machine learning model, Receiving the prediction of future behavior associated with the ongoing process flow, Based on the received prediction, perform one or more actions. The method described in Appendix 1, further including the above. (Note 4) The method according to Note 1, wherein the plurality of features are represented as a plurality of corresponding feature vectors generated using one or more of the following: direct feature generation, activity-based feature generation, duration-based feature generation, time-delta-based feature generation, state-graph-based feature generation, or calendar-based feature generation. (Note 5) Extracting multiple subsets of the data that occur during a plurality of corresponding predetermined time range cutoffs, each defined within the total time of the completed process flow, To generate the plurality of features for each of the plurality of subsets of the aforementioned data, The process involves training multiple machine learning models corresponding to each of the extracted subsets of the data, wherein each of the multiple machine learning models is trained to predict the future behavior of an ongoing process flow. The method described in Appendix 1, further including the above. (Appendix 6) The method according to Appendix 1, wherein the data corresponding to the completed process flow is transformed using one or more preprocessing operations, and the transformed data is configured to be used as training data for training the machine learning model. (Note 7) The method according to Note 6, wherein the one or more preprocessing operations include identifying and retaining relevant data contained in the data corresponding to the completed process flow, the relevant data including data points directly related to the training of the machine learning model. (Note 8) A system, One or more processors, One or more non-temporary computer-readable storage media configured to store instructions, The instruction includes, and in response to being executed, the system causes the system to perform an action, and the action is, This involves obtaining data corresponding to a completed process flow, wherein the completed process flow is associated with a resolved incident. Based on the data corresponding to the completed process flow, generate one or more labels representing the characteristics of the incident, which will serve as target variables for training a machine learning model. Extracting a subset of the data corresponding to a predetermined time range cutoff defined within the total time required to resolve the incident, To train the machine learning model based on the subset of the data corresponding to the predetermined time range cutoff, generate a plurality of features representing identifiable attributes derived from the acquired data corresponding to the completed process flow, Training the machine learning model to predict the future behavior of the process flow in progress using the multiple features associated with the data corresponding to the completed process flow and the one or more labels. A system that includes this. (Note 9) Generating one or more labels is Extracting label data associated with one or more characteristics in the data corresponding to the completed process flow, The label data is encoded into a format suitable for training the machine learning model. The system described in Appendix 8, including the system described in Appendix 8. (Note 10) The above operation is The method involves automatically acquiring second data corresponding to an ongoing process flow based on the ongoing process flow reaching a predetermined time range cutoff, wherein the ongoing process flow is associated with an unresolved incident, and the second data occurs during the predetermined time range cutoff. To generate a second set of features associated with the second data corresponding to the predetermined time range cutoff, To generate predictions of future behavior associated with the ongoing process flow, the second set of features are transmitted to the machine learning model, Receiving the prediction of future behavior associated with the ongoing process flow, Based on the received prediction, perform one or more actions. The system described in Appendix 9, which further includes the above. (Note 11) The system described in Note 8, wherein the aforementioned features are represented as corresponding feature vectors generated using one or more of the following methods: direct feature generation, activity-based feature generation, duration-based feature generation, time-delta-based feature generation, state-graph-based feature generation, or calendar-based feature generation. (Note 12) The above operation is Extracting multiple subsets of the data that occur during corresponding multiple predetermined time range cutoffs, each defined within the total time of the completed process flow, To generate the plurality of features for each of the plurality of subsets of the aforementioned data, The process involves training multiple machine learning models corresponding to each of the extracted subsets of the data, wherein each of the multiple machine learning models is trained to predict the future behavior of an ongoing process flow. The system described in Appendix 8, which further includes the above. (Note 13) The system as described in Note 8, wherein the data corresponding to the completed process flow is transformed using one or more preprocessing operations, and the transformed data is configured to be used as training data for training the machine learning model. (Note 14) The system as described in Note 13, wherein the one or more preprocessing operations include identifying and retaining relevant data contained in the data corresponding to the completed process flow, the relevant data including data points directly related to the training of the machine learning model. (Note 15) One or more non-temporary computer-readable storage media configured to store instructions, wherein the instructions cause the system to perform an action in response to their execution, and the action is This involves obtaining data corresponding to a completed process flow, wherein the completed process flow is associated with a resolved incident. Based on the data corresponding to the completed process flow, generate one or more labels representing the characteristics of the incident, which will serve as target variables for training a machine learning model. Extracting a subset of the data corresponding to a predetermined time range cutoff defined within the total time required to resolve the incident, To train the machine learning model based on the subset of the data corresponding to the predetermined time range cutoff, generate a plurality of features representing identifiable attributes derived from the acquired data corresponding to the completed process flow, Training the machine learning model to predict the future behavior of the process flow in progress using the multiple features associated with the data corresponding to the completed process flow and the one or more labels. One or more non-temporary computer-readable storage media, including [the specified text]. (Note 16) Generating one or more labels is Extracting label data associated with one or more characteristics in the data corresponding to the completed process flow, The label data is encoded into a format suitable for training the machine learning model. One or more non-temporary computer-readable storage media as described in Appendix 15, including the following: (Note 17) The above operation is The method involves automatically acquiring second data corresponding to an ongoing process flow based on the ongoing process flow reaching a predetermined time range cutoff, wherein the ongoing process flow is associated with an unresolved incident, and the second data occurs during the predetermined time range cutoff. To generate a second set of features associated with the second data corresponding to the predetermined time range cutoff, To generate predictions of future behavior associated with the ongoing process flow, the second set of features are transmitted to the machine learning model, Receiving the prediction of future behavior associated with the ongoing process flow, Based on the received prediction, perform one or more actions. One or more non-temporary computer-readable storage media as described in Appendix 15, further including the following: (Note 18) One or more non-temporary computer-readable storage media as described in Note 15, wherein the features are represented as corresponding feature vectors generated using one or more of the following methods: direct feature generation, activity-based feature generation, duration-based feature generation, time-delta-based feature generation, state-graph-based feature generation, or calendar-based feature generation. (Note 19) The above operation is Extracting multiple subsets of the data that occur during corresponding multiple predetermined time range cutoffs, each defined within the total time of the completed process flow, To generate the plurality of features for each of the plurality of subsets of the aforementioned data, The process involves training multiple machine learning models corresponding to each of the extracted subsets of the data, wherein each of the multiple machine learning models is trained to predict the future behavior of an ongoing process flow. One or more non-temporary computer-readable storage media as described in Appendix 15, further including the following: (Note 20) One or more non-temporary computer-readable storage media as described in Note 15, wherein the data corresponding to the completed process flow is transformed using one or more preprocessing operations, and the transformed data is configured to be used as training data for training the machine learning model.

Claims

1. It is a method, This involves obtaining data corresponding to a completed process flow, wherein the completed process flow is associated with a resolved incident. Based on the data corresponding to the completed process flow, generate one or more labels representing the characteristics of the incident, which will serve as target variables for training a machine learning model. Extracting a subset of the data corresponding to a predetermined time range cutoff defined within the total time required to resolve the incident, To train the machine learning model based on the subset of the data corresponding to the predetermined time range cutoff, generate a plurality of features representing identifiable attributes derived from the acquired data corresponding to the completed process flow, Training the machine learning model to predict the future behavior of the process flow in progress using the multiple features associated with the data corresponding to the completed process flow and the one or more labels. A method that includes this.

2. Generating one or more labels is Extracting label data associated with one or more characteristics in the data corresponding to the completed process flow, The label data is encoded into a format suitable for training the machine learning model. The method according to claim 1, including the method described in claim 1.

3. The method involves automatically acquiring second data corresponding to an ongoing process flow based on the ongoing process flow reaching a predetermined time range cutoff, wherein the ongoing process flow is associated with an unresolved incident, and the second data occurs during the predetermined time range cutoff. To generate a second set of features associated with the second data corresponding to the predetermined time range cutoff, To generate predictions of future behavior associated with the ongoing process flow, the second set of features are transmitted to the machine learning model, Receiving the prediction of future behavior associated with the ongoing process flow, Based on the received prediction, perform one or more actions. The method according to claim 1, further comprising:

4. The method according to claim 1, wherein the plurality of features are represented as a plurality of corresponding feature vectors generated using one or more of the following: direct feature generation, activity-based feature generation, duration-based feature generation, time-delta-based feature generation, state-graph-based feature generation, or calendar-based feature generation.

5. Extracting multiple subsets of the data that occur during corresponding multiple predetermined time range cutoffs, each defined within the total time of the completed process flow, To generate the plurality of features for each of the plurality of subsets of the aforementioned data, The process involves training multiple machine learning models corresponding to each of the extracted subsets of the data, wherein each of the multiple machine learning models is trained to predict the future behavior of an ongoing process flow. The method according to claim 1, further comprising:

6. The method according to claim 1, wherein the data corresponding to the completed process flow is transformed using one or more preprocessing operations, and the transformed data is configured to be used as training data for training the machine learning model.

7. The method according to claim 6, wherein the one or more preprocessing operations include identifying and retaining relevant data contained in the data corresponding to the completed process flow, the relevant data including data points directly related to the training of the machine learning model.

8. It is a system, One or more processors, One or more non-temporary computer-readable storage media configured to store instructions, The instruction includes, and in response to being executed, the system causes the system to perform an action, and the action is, This involves obtaining data corresponding to a completed process flow, wherein the completed process flow is associated with a resolved incident. Based on the data corresponding to the completed process flow, generate one or more labels representing the characteristics of the incident, which will serve as target variables for training a machine learning model. Extracting a subset of the data corresponding to a predetermined time range cutoff defined within the total time required to resolve the incident, To train the machine learning model based on the subset of the data corresponding to the predetermined time range cutoff, generate a plurality of features representing identifiable attributes derived from the acquired data corresponding to the completed process flow, Training the machine learning model to predict the future behavior of the process flow in progress using the multiple features associated with the data corresponding to the completed process flow and the one or more labels. A system that includes this.

9. Generating one or more labels is Extracting label data associated with one or more characteristics in the data corresponding to the completed process flow, The label data is encoded into a format suitable for training the machine learning model. The system according to claim 8, including the above.

10. The aforementioned operation is, The method involves automatically acquiring second data corresponding to an ongoing process flow based on the ongoing process flow reaching a predetermined time range cutoff, wherein the ongoing process flow is associated with an unresolved incident, and the second data occurs during the predetermined time range cutoff. To generate a second set of features associated with the second data corresponding to the predetermined time range cutoff, To generate predictions of future behavior associated with the ongoing process flow, the second set of features are transmitted to the machine learning model, Receiving the prediction of future behavior associated with the ongoing process flow, Based on the received prediction, perform one or more actions. The system according to claim 9, further comprising:

11. The system according to claim 8, wherein the plurality of features are represented as a plurality of corresponding feature vectors generated using one or more of the following methods: direct feature generation, activity-based feature generation, duration-based feature generation, time-delta-based feature generation, state-graph-based feature generation, or calendar-based feature generation.

12. The aforementioned operation is, Extracting multiple subsets of the data that occur during corresponding multiple predetermined time range cutoffs, each defined within the total time of the completed process flow, To generate the plurality of features for each of the plurality of subsets of the aforementioned data, The process involves training multiple machine learning models corresponding to each of the extracted subsets of the data, wherein each of the multiple machine learning models is trained to predict the future behavior of an ongoing process flow. The system according to claim 8, further comprising:

13. The system according to claim 8, wherein the data corresponding to the completed process flow is transformed using one or more preprocessing operations, and the transformed data is configured to be used as training data for training the machine learning model.

14. The system according to claim 13, wherein the one or more preprocessing operations include identifying and retaining relevant data contained in the data corresponding to the completed process flow, the relevant data including data points directly related to the training of the machine learning model.

15. One or more non-temporary computer-readable storage media configured to store instructions, wherein the instructions cause the system to perform an action in response to their execution, and the action is: This involves obtaining data corresponding to a completed process flow, wherein the completed process flow is associated with a resolved incident. Based on the data corresponding to the completed process flow, generate one or more labels representing the characteristics of the incident, which will serve as target variables for training a machine learning model. Extracting a subset of the data corresponding to a predetermined time range cutoff defined within the total time required to resolve the incident, To train the machine learning model based on the subset of the data corresponding to the predetermined time range cutoff, generate a plurality of features representing identifiable attributes derived from the acquired data corresponding to the completed process flow, Training the machine learning model to predict the future behavior of the process flow in progress using the multiple features associated with the data corresponding to the completed process flow and the one or more labels. One or more non-temporary computer-readable storage media, including [the specified text].

16. Generating one or more labels is Extracting label data associated with one or more characteristics in the data corresponding to the completed process flow, The label data is encoded into a format suitable for training the machine learning model. One or more non-temporary computer-readable storage media according to claim 15, including the following:

17. The aforementioned operation is, The method involves automatically acquiring second data corresponding to an ongoing process flow based on the ongoing process flow reaching a predetermined time range cutoff, wherein the ongoing process flow is associated with an unresolved incident, and the second data occurs during the predetermined time range cutoff. To generate a second set of features associated with the second data corresponding to the predetermined time range cutoff, To generate predictions of future behavior associated with the ongoing process flow, the second set of features are transmitted to the machine learning model, Receiving the prediction of future behavior associated with the ongoing process flow, Based on the received prediction, perform one or more actions. One or more non-temporary computer-readable storage media according to claim 15, further comprising:

18. The one or more non-temporary computer-readable storage medium according to claim 15, wherein the plurality of features are represented as a plurality of corresponding feature vectors generated using one or more of the following: direct feature generation, activity-based feature generation, duration-based feature generation, time-delta-based feature generation, state-graph-based feature generation, or calendar-based feature generation.

19. The aforementioned operation is, Extracting multiple subsets of the data that occur during corresponding multiple predetermined time range cutoffs, each defined within the total time of the completed process flow, To generate the plurality of features for each of the plurality of subsets of the aforementioned data, The process involves training multiple machine learning models corresponding to each of the extracted subsets of the data, wherein each of the multiple machine learning models is trained to predict the future behavior of an ongoing process flow. One or more non-temporary computer-readable storage media according to claim 15, further comprising:

20. The data corresponding to the completed process flow is transformed using one or more preprocessing operations, and the transformed data is configured to be used as training data for training the machine learning model, one or more non-temporary computer-readable storage media according to claim 15.

Images