Machine learning system and method for real-time anomaly detection and normative feedback

Abstract

A computer-implemented method for anomaly detection includes receiving a set of data parameters, retrieving a data set corresponding to the set of data parameters from a database, analyzing the data set to detect one or more anomalies in the data set, selecting a set of anomaly parameters corresponding to the detected one or more anomalies, filtering an output of a machine learning model according to the set of anomaly parameters, generating a set of instructions for identifying one or more anomalous items, executing the set of instructions for identifying the anomalous items to identify one or more anomalous items within the data set in real-time in response to an update to the data set, and transmitting information about the one or more anomalous items to a user computing device or another computing device.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Application No. 63 / 567,400, filed March 19, 2024, which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure relates in general to methods and systems for real-time anomaly detection using machine learning models, generating patterns for continuous monitoring of data, and assigning prescriptive actions in response to the detection of anomalies. Background Technology

[0004] The background description provided herein is for the purpose of presenting the overall context of this disclosure. Within the scope described in this background section, the work of the presently designated inventors and aspects of the specification that are not otherwise identified as prior art at the time of filing are neither expressly nor implicitly acknowledged as prior art relative to this disclosure.

[0005] Existing anomaly detection processes (such as those in retail anomaly-based reporting) lack the ability to create new data and detect anomalies in an on-demand, real-time manner. Furthermore, existing anomaly detection processes do not include: converting anomaly detection into a repetitive and permanent process; providing prescriptive actions to selected users based on responsibility and / or security levels to correct the anomaly; and transmitting prescriptive actions to external task management applications. Therefore, there is an opportunity to achieve on-demand, real-time anomaly detection. Summary of the Invention

[0006] In one implementation, a computer-implemented method for anomaly detection includes: receiving a set of data parameters via one or more processors; retrieving a dataset corresponding to the set of data parameters from a database via one or more processors; analyzing the dataset via one or more processors and using a machine learning model trained in real time to detect one or more anomalies in the dataset; selecting an anomaly parameter set corresponding to the detected one or more anomalies via one or more processors; filtering the output of the machine learning model based on the anomaly parameter set via one or more processors; generating a set of instructions for identifying one or more anomalies via one or more processors based on the set of data parameters, the set of anomaly parameters, and a detection pattern parameter set; executing the set of instructions for identifying anomalies via one or more processors to identify one or more anomalies in the dataset in real time in response to an update of the dataset; and transmitting information about the one or more anomalies to a user computing device or another computing device via one or more processors.

[0007] In one implementation, a computing system for anomaly detection includes one or more processors; one or more memories having computer-executable instructions stored thereon, which, when executed by the one or more processors, cause the computing system to: receive a set of data parameters; retrieve a dataset corresponding to the set of data parameters from a database; analyze the dataset using a machine learning model trained in real time to detect one or more anomalies in the dataset; select an anomaly parameter set corresponding to the detected one or more anomalies; filter the output of the machine learning model based on the anomaly parameter set; generate a set of instructions for identifying one or more anomalies based on the set of data parameters, the set of anomaly parameters, and the set of detection pattern parameters; execute the set of instructions for identifying anomalies to identify one or more anomalies in the dataset in real time in response to an update of the dataset; and transmit information about the one or more anomalies to a user computing device or another computing device. Attached Figure Description

[0008] The accompanying drawings described below illustrate various implementations of the systems and methods disclosed herein. It should be understood that each drawing illustrates one implementation of a particular system and method, and each drawing is intended to represent one possible implementation of the system and method. Furthermore, where possible, the reference numerals included in the following drawings will refer to the figures, wherein features shown in multiple drawings are indicated by consistent reference numerals.

[0009] The accompanying drawings depict preferred implementations for illustrative purposes only. Those skilled in the art will readily recognize from the following discussion that alternative implementations of the structures and methods illustrated herein can be employed without departing from the principles of the invention described herein.

[0010] Figure 1 A flowchart is depicted illustrating an exemplary method for real-time anomaly detection and prescriptive feedback using machine learning based on several aspects.

[0011] Figures 2A-2B An example process is described, which selects a set of data parameters based on several aspects and retrieves data from a database based on the selected set of data parameters.

[0012] Figures 3A-3B An example process for anomaly detection based on several aspects is described.

[0013] Figures 4A-4B Examples of filtering based on the results of an anomaly detection process and the causes of anomalies are described.

[0014] Figure 5An example user interface is depicted that generates a set of instructions for identifying anomalies based on several aspects in response to updates to the dataset.

[0015] Figure 6 The user interface describes the captured items transmitted according to some aspects and the normative actions sent to the appropriate user for processing.

[0016] Figure 7 A timing diagram is shown illustrating an example of a set of instructions executed based on certain aspects for real-time identification of anomalies.

[0017] Figures 8A-8B An example is described of transmitting normative initiatives to an external task management system based on several aspects.

[0018] Figure 9 An exemplary computing environment is described, which can implement the techniques disclosed herein in some respects. Detailed Implementation

[0019] Overview

[0020] This technology provides a system and method for on-demand real-time anomaly detection using machine learning. The system and method include, for example, receiving a set of data parameters; retrieving a dataset corresponding to the set of data parameters from a database; analyzing the dataset to detect one or more anomalies in the dataset; selecting a set of anomaly parameters corresponding to the detected one or more anomalies; filtering the output of a machine learning model based on the set of anomaly parameters; generating a set of instructions for identifying one or more anomalies; executing the set of instructions for identifying anomalies to identify one or more anomalies in the dataset in real time in response to updates to the dataset; and transmitting information about the one or more anomalies to a user computing device or another computing device.

[0021] As mentioned earlier, existing anomaly detection processes (such as those in retail anomaly-based reporting) lack the ability to create new data and detect anomalies in an on-demand, real-time manner. Existing anomaly detection processes do not transform anomaly detection into a repetitive and continuous process. These limitations are partly due to technical hurdles faced by system providers and retail customers. A particular customer may have dozens, hundreds, or thousands of stores, kiosks, warehouses, distribution centers, etc., each capturing data about merchandise, customers, and / or employees. Given the volume of data, the complexity of the captured data, differences in data collection time, and other factors, attempting to detect anomalies in the collected data is challenging. Therefore, these technical hurdles are partly a result of the sheer volume of data.

[0022] However, anomaly detection, particularly in areas like retail anomaly-based monitoring, faces specificity limitations. Anomaly detection systems cannot detect entirely new, previously unknown anomalies. While clients can design highly customized executable scripts to detect specific types of anomalies, they cannot identify anomalies in a trainable manner. Therefore, if an anomaly is not pre-scripted, it is likely to go undetected. This limitation is particularly pronounced for retailers facing employee theft: unethical employees continuously develop increasingly sophisticated methods to exploit the retailer's own computing systems for undetected merchandise theft. System designers also recognize the significant challenge of creating highly customized anomaly detection scripts and deploying them across remote locations, such as retail stores spanning an entire region. The more specific the anomaly to be detected, the more difficult it is to adapt the script to other locations or other anomaly detection scenarios. Furthermore, the more specific and localized the anomaly detection configuration, the more difficult it is to determine prescriptive measures and develop appropriate response plans, especially in on-demand models based on real-time data collection. It is virtually impossible for conventional systems to achieve such an on-demand, real-time anomaly detection system: one that can aggregate data from multiple remote locations and determine the response level of prescriptive actions, ranging from actions at specific data entry points (such as specific scanning stations in retail stores or warehouses) to actions requiring regulatory-level actions (such as continuous monitoring of employees across different locations or time windows).

[0023] To overcome these technical obstacles, this application describes a system and method for providing on-demand, real-time anomaly detection by using a machine learning model trained in real-time based on collected data. The result is a real-time trainable anomaly detection engine. The collected data may come from one or more locations, such as remote computing systems that transmit real-time data to servers, centralized computing systems, etc., which store an anomaly detection engine for executing the methods described herein. In various examples, the machine learning model may be an autoencoder neural network, although other exemplary machine learning models include, for example, linear or logistic regression, instance-based algorithms, regularization algorithms, decision trees, isolated forests, Bayesian networks, clustering analysis, association rule learning, artificial neural networks, deep learning, ensemble learning, reinforcement learning, dimensionality reduction, and support vector machines. In such examples, the architecture of the machine learning model does not require pre-training, yet it is still designed for anomaly detection on real-time data. In response to real-time data, anomaly detection parameters can be deployed, filtering the anomaly detection parameters before the real-time data is submitted to the anomaly detection engine. The real-time data and the output of the anomaly detection engine can be deployed as a continuous detection mode, which autonomously checks for anomalies in subsequently received real-time data. In other words, an anomaly detection engine can be used to generate anomaly patterns to be monitored. These anomaly patterns can be location-specific; for example, the anomaly detection engine detects potential anomalies at a specific location. These anomaly patterns can cover multiple remote locations. These anomaly patterns can be location-specific, product-specific, product type-specific, employee-specific, customer-specific, etc., or any combination thereof.

[0024] Therefore, the technology disclosed herein achieves a technological improvement over conventional technologies, at least by improving the functionality of computing devices (e.g., servers executing machine learning models). Specifically, the computing device uses a machine learning model trained in real time to analyze data, generates a set of instructions for identifying anomalies, and executes the instructions in a specific manner that enhances the efficiency of the computing device. Performing these actions enables the detection of previously unknown anomalies (i.e., unique anomalies that conventional systems may fail to detect) with an efficiency (i.e., in real time) that is unattainable with conventional technologies. In other words, this disclosure describes an improvement in the functionality of the computer itself, because, as a direct result of the machine learning model and the multiple sets of instructions generated, the computing device is able to identify anomalies more efficiently. This improvement over the prior art lies at least in the fact that existing systems cannot identify previously unknown data anomalies in real time, and / or cannot analyze data with the efficiency brought about by the machine learning model and the set of instructions of this disclosure.

[0025] Figure 1 This is a flowchart of a method for on-demand, real-time anomaly detection. In various examples, the method presented in this paper can... Figure 9The illustrated computing environment implementation may include computing resources for training and / or running machine learning models to detect anomalies. This environment may include user devices coupled via network communications, storage computing devices, task management systems, servers, databases, and / or cloud APIs. Users can access these resources via user devices (such as those described below). Figure 9 User devices (908) can access the application used for anomaly detection using a desktop or mobile browser.

[0026] The method begins at box 102, for example, user and user device (such as Figure 9 The system interacts with device 908 (which may be a smartphone, tablet, desktop computer, etc.) to select a set of data parameters, which indicates the data the system wants to analyze. The set of data parameters defines the query and may include dimensions and measures. A dimension is an entity the system intends to analyze, and a measure is an indicator associated with that dimension. An entity can be any identifiable entity in a remote location, including but not limited to data collection devices, systems, sites, or any entity associated with or operating said data collection devices, systems, or sites at a remote location. For example, such as... Figure 2B As shown, the dimension can be one or more cashiers, and the measure can be the number of receipts issued by the cashier, the amount of discounts applied by the cashier, the number of items discounted by the cashier, and the cashier's fixed amount of cancelled transactions. In another example, the dimension can be a store, and the measure can include the amount of suspended transactions and the sales amount of a specific product. Data parameters can be used to construct queries to submit to the database to retrieve data.

[0027] At box 104, you can retrieve a dataset from the database based on the data parameters by submitting a pre-constructed query to the database. The database can be a cloud database, such as Google BigQuery.

[0028] At box 106, a dataset can be input into a machine learning model to detect one or more anomalies in the dataset. The machine learning model can be an unsupervised neural network, such as an autoencoder neural network, and can be trained in real-time based on the data. The machine learning model can receive and analyze the data to determine rules for the dataset, and then reconstruct the dataset based on those rules. The machine learning model can determine whether anomalies (e.g., captured items) exist in a given dimension. For example, in... Figure 4A In the dataset, cashier 488 (63) is marked as having an anomaly ("Yes") in the "Is it anomaly?" column. Machine learning models can also determine percentage scores, which represent the similarity between data points in the original dataset provided to the machine learning model and corresponding data points in the reconstructed dataset returned by the model, such as anomaly scores. For example, in... Figure 4AIn the middle, the abnormal score of cashier 488 (63) was 100%.

[0029] At box 108, users can select a set of anomaly parameters to filter the output of the machine learning model, so that only the data of interest is displayed and / or saved and / or used for further analysis. Anomaly parameters may include anomaly indications (“Anomaly Yes / No”), anomaly scores (“Anomaly Score 0-100%”), and the first and second principal components of principal component analysis (“PCA1 and PCA2”). Users can filter the dataset output by the machine learning model based on one or more anomaly parameters, narrowing the dataset output by the machine learning model.

[0030] At box 110, a set of instructions (e.g., patterns) for identifying anomalies can be generated. The instruction set is a continuous and autonomous pattern used to scan the dataset for anomalies in response to updates to the dataset. The instruction set can be based on a set of data parameters, a set of anomaly parameters, and a set of detection pattern parameters. The set of data parameters defines the dataset that the machine learning model will analyze for anomalies. The anomaly parameters are used to filter the dataset output by the machine learning model, causing the pattern to scan the dataset for members that meet the anomaly detection criteria.

[0031] Users can select a set of detection mode parameters to further define the set of instructions used for anomaly detection. The set of detection mode parameters may include: a time range indicating which values ​​should be included in the second dataset; a schedule for further anomaly detection; one or more prescriptive actions associated with the one or more anomalies; a security level associated with the one or more anomalies; and / or a level of responsibility associated with the one or more anomalies. The time range indicates the values ​​to be included in the dataset to be analyzed, such as the past 7 days, the past 30 days, etc. The anomaly detection schedule includes the frequency of anomaly detection, such as daily, monthly, after a set number of occurrences, etc., and may include the start and end dates for anomaly detection. The schedule may also include the calendar type used to run the anomaly detection, and whether the anomaly detection runs automatically or at specific times. Prescriptive actions include: information about the anomaly, the cause of the numerical anomaly, and the actions to be taken to correct the anomaly in response to its detection. The security level may indicate which users are allowed access to information about the anomaly and / or to take actions in response to the anomaly. The level of responsibility may indicate which users are responsible for taking actions in response to the anomaly.

[0032] At box 112, there is a set of executable instructions to detect anomalies (e.g., capture items) within the system.

[0033] At box 114, anomalies can be transmitted to the user based on the security or responsibility level of one or more anomalies. Information about one or more anomalies can be provided to the user responsible for handling the captured items. This information may include an opportunity or task for the anomaly, providing an explanation of the anomaly and / or the basis for the anomaly detection pattern, the reason for the anomaly being flagged, and / or the prescriptive actions (i.e., corrective actions) to be taken in response to the anomaly. Analytical views and other data visualizations can be displayed to provide more information about the anomalies. In some embodiments, prescriptive actions may be transmitted to an external task management system. In some embodiments, prescriptive actions transmitted to an external task management system may not include any data identifying the anomaly, making it impossible for users of the external task management system to view details such as the cause of a particular anomaly.

[0034] Figures 2A-2B The system describes the set of data parameters for selecting the dataset that the system intends to analyze, and how to retrieve that dataset from the database.

[0035] Figure 2A A combination of block diagrams and flowcharts illustrates an example of retrieving a dataset from a database. A set of data parameters can be used to create query request 202, which is then validated at box 204. The set of data parameters may then be translated into a query definition at box 206, and the query definition may be used at box 208 to generate a query written in a programming language for storing and processing information in a relational database. For example, in Figure 2A In this context, the programming language is SQL, although other database languages ​​can also be used. Semantic layer 210 can map different terms used by different departments within a company to refer to the same thing to a single data entity, providing a unified view of the data. Other applications may be used to allow users less familiar with database programming languages ​​to analyze and view the data. At box 212, the query can be executed to retrieve data from database 214 (e.g., Google BigQuery). The result (i.e., the dataset) may be parsed at box 216 and returned to the user.

[0036] Figure 2B An example of a dataset retrieved from a database is depicted. The dataset is retrieved based on parameters. Data parameters may include dimension 220 (i.e., the entity to be observed), and a measure 222 associated with dimension 220, which may include metrics for dimension 220. For example, in Figure 2BIn this process, based on the selected set of data parameters, a dataset containing 224 (dimensions) about cashiers is retrieved from the database, along with measures including the number of receipts generated by each cashier (receipt #) 226, the dollar amount of invalid transactions processed by a specific cashier (fixed invalid $) 228, the discount amount processed by a specific cashier (discount $) 230, and the number of discount items processed by a specific cashier (discount #) 232.

[0037] Figures 3A-3B The process of anomaly detection is described.

[0038] like Figure 3A As shown, a request can be submitted by a user via a user interface (such as user interface 302). This request may invoke an anomaly detection application programming interface (API), which is stored, for example, in the memory of the server providing the anomaly detection service, such as... Figure 9 The API is located in memory 926 of server 904. This API may be implemented as an endpoint accessible via web service protocols such as Representational State Transfer (REST), Simple Object Access Protocol (SOAP), or JavaScript Object Notation (JSON). After a request is submitted, as... Figures 2A-2B As described in the diagram, data is retrieved from database 306 at box 304 based on a set of data parameters. This data can be preprocessed at box 308 to transform it for analysis and as input to a machine learning model. For example, the data may be cleaned, normalized, filtered, undergo feature extraction, feature selection, or otherwise transformed to prepare it for analysis. The preprocessed data may then be fed into a machine learning model for training and prediction.

[0039] The machine learning model can be trained at box 310. In some embodiments, the machine learning model can be as follows: Figure 3A The diagram shows an autoencoder neural network. An autoencoder has an encoding function and a decoding function. The encoding function transforms the input data into a latent space, thereby deriving rules from the dataset. The decoding function reconstructs the input data from the latent space based on the rules derived from the encoding step. In some implementations, the autoencoder can be trained in real-time on newly collected and / or updated data. At box 314, the trained model can be stored in cloud storage. At box 316, the model predicts anomalous data points in the dataset by generating anomaly scores by comparing the reconstructed data with the input data. Data points from the reconstructed dataset that deviate from their corresponding input data points (i.e., have high anomalous scores) are considered anomalous. The prediction results are processed at box 318, and the output is transmitted to the user interface 302 at box 320.

[0040] The use of autoencoders has advantages, such as real-time training on newly updated data. However, other machine learning techniques can also be used. For example, machine learning models can employ a variety of machine learning methods and algorithms, such as linear or logistic regression, instance-based algorithms, regularization algorithms, decision trees, Bayesian networks, cluster analysis, association rule learning, artificial neural networks, deep learning, ensemble learning, reinforcement learning, dimensionality reduction, and support vector machines. These methods and algorithms can be geared towards one or more categories of machine learning, including supervised learning, unsupervised learning, and reinforcement learning.

[0041] Figure 3B This describes a sample dataset processed for anomaly detection, the results of anomaly detection, and the filtering of such results. For example, such as... Figure 3B As shown, dataset 300B may include cashiers 330, receipt numbers 332 indicating the number of transaction receipts generated by the corresponding cashier, and fixed voided amounts 334 indicating the amount of voided transactions in USD for the corresponding cashier. Anomaly detection results from the machine learning model output may include whether an anomaly is associated with a specific cashier (“Anomaly” column 336) and an anomaly score 338, which in the example shown indicates the percentage confidence in the anomaly. Other example anomaly detection output data from the machine learning model includes the mean of a metric and other statistics, and the percentage difference between a metric's value and its mean. In some implementations, anomaly detection output data may be displayed graphically. These results (i.e., individual anomaly detection outputs, or anomaly detection outputs combined with input data fed to the anomaly detection system) may also be filtered based on a set of anomaly parameters to narrow the displayed dataset. Such filtering may occur, for example, after the output of the machine learning model, to filter the generated output. For example, in Figure 3B In the process, the output of the machine learning model has been filtered to show only cashiers marked as anomalous ("Yes" in column 336, "Is it anomalous?") and anomalous scores indicating the degree of anomalousness of the items ("Anomalous Score" column 338).

[0042] Figure 4A and Figure 4B The results of the anomaly detection process are shown, along with the reasons for the anomalies. Figure 4A Examples of detecting and filtering cashier anomalies are shown. For example, Table 402 shows that cashier 488 (63) has detected anomalies associated with that cashier. Box 404 shows the reasons why cashier 488 (63) was flagged as an anomaly. For example, cashier 488 (63)'s fixed voided amount value ($-$6,254.11) is significantly higher than the average ($-$25.51), and the receipt quantity value (20) is significantly lower than the average (64). Although Figure 4AOnly one entity marked as anomalous (i.e., cashier 488(63)) is depicted, but more than one anomalous entity may exist. In some implementations, the cause of the anomalous event may be transmitted to the user device and / or task management service. Figure 406 depicts a plot of data points in the dataset, where each point is labeled as anomalous or non-nominal. Point 408 refers to a fixed voided dollar value associated with the receipt quantity value of cashier 488(63). The greater distance of point 408 from the clusters of other data points indicates that point 408 is anomalous.

[0043] Figure 4B Examples of detecting and filtering store anomalies are shown. For instance, Table 410 shows that store 302 has detected anomalies associated with that store. In some implementations, metrics associated with a particular entity can be flagged based on comparisons with all other entities in the entity pool. For example, as shown in box 412, the receipt # and receipt $ values ​​are significantly lower than the average receipt # and receipt $ in the store pool. However, voided transactions $ and suspended transactions $ are only slightly lower than the pool average, resulting in a 24% difference in voided transactions $ and suspended transactions $ compared to the store pool average. Beer sales have the largest % difference from the pool average and are abnormally low compared to the pool average. Therefore, the receipt #, receipt $, voided transactions $, suspended transactions $, and beer sales for store 302 are anomalies compared to the store pool average and indicate that someone at store 302 may be giving away beer or that beer is being stolen. In some implementations, entities can be flagged as anomalies due to real-time, dynamic shifts in data. For example, an entity pool as a whole may experience real-time, dynamic offsets that affect metrics, such that an entity may only be flagged if its metric exceeds the range of the real-time offset experienced by the entity pool when compared with the entity pool.

[0044] Figure 5 An example user interface is depicted for generating a set of instructions for identifying anomalies in response to updates to the dataset. Detection mode parameters for generating the set of instructions for identifying anomalies include a time range 502, whether the item is a capture item 504, whether the capture item should be assigned according to safety 506 and / or whether the capture item should be assigned according to responsibility 508, prescriptive actions 510, and a schedule 512 for executing the instructions for identifying anomalies. In some embodiments, prescriptive actions may be predetermined and selected by the user. In some embodiments, prescriptive actions may be manually entered by the user via the user interface when creating the set of instructions for anomaly detection. The schedule 512 may include a start date 514 and / or an end date 516, the repetition method of anomaly detection 518 (e.g., frequency), whether the anomaly detection is performed automatically or at a specific time 520, and / or the calendar type used to run the anomaly detection 522. The user can select mode parameters via the user interface.

[0045] Figure 6 The diagram illustrates the transmitted capture item 602 and a user interface for sending a prescriptive action 604 to the appropriate user for processing. The prescriptive action may be sent to the user based on responsibility and / or security. In some implementations, a reason for the capture item's exception may be included.

[0046] Figure 7 A sequence diagram depicts sequence 700 associated with the execution of a set of instructions for real-time anomaly identification (i.e., pattern execution), which can be executed by a specific instance implementation of a pattern engine and analytics service stored in the server's memory, such as those stored in... Figure 9 The server 904 contains a pattern engine 932 and an analysis service 928 in its memory 926. Sequence 700 includes a timeline of events affecting the pattern engine 932 and the analysis service 928.

[0047] Sequence 700 can begin at step 702, where the pattern engine 932 exports the dataset to be analyzed to the analysis service 928 ("call RunQuery V2 to export query results"). At step 704, the analysis service returns a query instance identifier in response to the request from the pattern engine 932.

[0048] At step 706, if the pattern contains such filtering, the pattern engine 932 can prepare a payload for the request to filter the results of anomaly detection using anomaly parameters.

[0049] At step 708, the pattern engine 932 invokes the API (“Run Exception API”) to execute a set of instructions used to identify exceptions. This API may be implemented as an endpoint accessible via a web service protocol such as Representational State Transfer (REST), Simple Object Access Protocol (SOAP), or JavaScript Object Notation (JSON). The analytics service 928 uses the above... Figure 3A The machine learning model identifies anomalies and returns a query instance identifier at step 710.

[0050] In step 712, the pattern engine invokes the API (“RunQuery API”) to transmit a request to the analytics server 928 to filter the output of the machine learning model. The request may include a set of exception parameters to filter the data. At step 714, the analytics server 928 returns a query instance identifier.

[0051] At step 716, the pattern engine invokes the RunQuery API to request the analysis service 928 to read the analysis results. The analysis service 928 can analyze the identified anomalies to provide information about them. For example, the analysis service 928 can determine the reason why an item was marked as an anomaly, and / or prescriptive actions to correct the anomalies, based on instructions in the pattern. At step 718, the analysis service 928 can transmit the analysis data about the anomalies to the pattern engine 930.

[0052] Figures 8A-8B Describe the standardized procedures for transmitting data to external task management devices.

[0053] Figure 8A The diagram illustrates a sequence diagram of sequence 800A associated with a transmission normative action (e.g., a pending matter), which can be executed by instructions stored in memory 926 of server 904, task management system 910, and one or more cloud APIs 914, such as... Figure 9 As shown. Sequence 800A contains a timeline of events affecting scheduler 802, pending events service 804, operation queue 806, task management 808, and message service 810. Scheduler 802 can schedule jobs and can be a service API, such as the GCP cloud scheduler. Pending events service 804 can be a service that identifies prescriptive actions for correcting anomalies, and can be... Figure 9 The analysis service 928 shown is part of the analysis service. Operation queue 806 may be a database (such as...) for storing and retrieving operations. Figure 9 The database (906) shown is a server, such as an SQL server. Task management (808) can be an external task management system that displays tasks to various users, such as... Figure 9 The task management system 910 is shown. The message service 810 is a message service API (such as GCP Pub / Sub) that facilitates communication from various services and supports asynchronous communication.

[0054] like Figure 8A As shown, at step 820, scheduler 802 may transmit a signal to pending task service 804 to initiate batch processing of operations. Scheduler 802 may initiate such processing periodically.

[0055] At step 822, the pending matters service 804 can query the operation queue 806 for operation batches. Operations can be sorted by priority. At step 824, the pending matters service can receive the requested operation batches from the operation queue 806.

[0056] At step 826, the pending items service 804 may execute the operation batch to identify pending items, and / or changes to pending items, and / or updates to pending items; that is, to identify normative actions, and / or changes to normative actions, and / or updates to normative actions to be transmitted to external task management 808. Such normative actions may be identified based on a set of instructions for anomaly detection.

[0057] At step 828, message service 810 may generate a POST request for creating a pending item. At step 830, pending item service may transfer the operation associated with the pending item to operation queue 806 for addition to operation queue 806.

[0058] At step 832, the pending task service 804 may transmit a request 832 to the task management 808 for obtaining an authorization token, such that the pending task may be added to the external task management system 808 and / or modified in the external task management system 808. At step 834, the pending task service 804 may receive the authorization token.

[0059] At step 836, the pending task service 804 may transmit the pending task to the task management 808 in a release request. The pending task and the authorization token may be included in the request.

[0060] At step 838, message service 810 may generate a patch request for modifying and / or updating an existing pending item. At step 840, the pending item service may transfer the operation associated with the pending item to operation queue 806 to modify and / or update the existing pending item in operation queue 806. Pending item service 804 may transfer the modification and / or update of the existing pending item at step 836. At step 834, the request for transferring the modification and / or update of the existing pending item may include an authorization token received from task management 808.

[0061] At step 842, task management 808 may transmit a confirmation message indicating that the pending item has been successfully added to task management 808, modified, or updated within task management 808. This confirmation message may be sent to message service 810. At step 844, message service may generate a request to publish the confirmation message and transmit it to pending item service 804.

[0062] At step 846, after receiving a confirmation message from message service 810, the pending matters service can add any corresponding subtask operation to operation queue 802.

[0063] At step 848, the pending items service can remove the operation batch from the queue. At step 850, the operation service can repeat this process for subsequent batches of operations.

[0064] Figure 8B Describes an external task management system (such as Figure 8A Example user interface for Task Management (808). Figure 8B As shown, a prescriptive action 860 (i.e., a pending action) can be received by the task management service and displayed to the user in the user interface 800B. The prescriptive action 860 can be selected according to the specifications in the anomaly detection instruction set. In some embodiments, the prescriptive action can be transmitted to the external task management user interface without any information about the anomaly (e.g., the cause of the anomaly and / or the prescriptive action). For example, the user of the task management interface can only see the action required to correct the anomaly, without seeing the reason why the item is an anomaly. In some embodiments, information about the anomaly may be transmitted to the task management interface, allowing the user of the task management interface to view information about the anomaly. In some embodiments, the amount of information about the anomaly transmitted to the task management interface depends on the security level and / or responsibility level. In some embodiments, the prescriptive action may include a priority 862, a status 864, and a user identifier 866 assigned to the prescriptive action.

[0065] Figure 9 It is a block diagram of an example system that can be used to implement various systems and methods for identifying the source of an exception. Figure 9 The system may include one or more storage computing devices 902, a server 904, one or more databases 906, one or more user devices 908, and an external task management system 910. The computing system may also include one or more cloud application programming interfaces (APIs) 914. The storage computing devices 902, server 904, database 906, and user devices 908 may be communicatively coupled via a network 912.

[0066] Storage computing device 902 can be any computing device located in the store. Storage computing device 902 can be a smartphone, tablet, desktop computer, cash register, or other device used for store operations. Each of the user devices 908 may include a processor and memory (not shown) containing instructions that, when executed, cause storage computing device 902 to collect data related to store operations. The storage computing device can transfer the collected data to other components of system 900, such as server 904 or database 906.

[0067] Figure 9Server 904 may include one or more processors 920, one or more network interfaces 924, and one or more memories 926. The one or more memories 926 may store an anomaly detector / analysis service module 928 (e.g., one or more sets of instructions for detecting anomalies in data collected from storage computing devices 902) and a pattern engine 932 (e.g., one or more sets of instructions for generating anomaly detection instruction sets). In some aspects, memory 926 may include additional modules and / or services for receiving and processing data from one or more other components of system 900, such as one or more cloud APIs 914, one or more storage computing devices 902, one or more user devices 908, a database 906, and / or an external task management system 910.

[0068] The processor 920 shown in the example can be implemented in hardware and may include semiconductor-based (e.g., silicon-based) devices. The processor 920 may be, for example, one or more programmable microprocessors, controllers, digital signal processors (DSPs), graphics processing units (GPUs), and / or any suitable type of programmable processor capable of executing instructions to, for example, implement the operation of the exemplary methods described herein. Additionally or alternatively, the processor 920 may be a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc., which can implement the operation of the exemplary methods described herein without executing instructions.

[0069] Figure 9 The exemplary server 904 includes one or more communication interfaces, such as one or more network interfaces 924. The communication interfaces 924 enable... Figure 9 The server 904 is capable of communicating with other devices, systems, etc. (e.g., storage computing device 902, database 906, user device 908), any other database, and / or any other machine.

[0070] Figure 9 Example server 904 includes multiple network interfaces 924 to enable communication with other machines via, for example, one or more networks (such as network 912). Exemplary network interfaces 924 include any suitable type of communication interface (e.g., wired and / or wireless interface) configured to operate according to any suitable communication protocol. Exemplary network interfaces 924 include TCP / IP interfaces, WiFi™ transceivers (e.g., compliant with IEEE 802.11x series standards), Ethernet transceivers, cellular transceivers, satellite transceivers, asynchronous transfer mode (ATM) transceivers, digital subscriber line (DSL) modems, coaxial cable modems, dial-up modems, or any other suitable interface based on any other suitable communication protocol or standard.

[0071] The memory 926 may include volatile and / or non-volatile storage media. For example, the memory 926 may include one or more random access memories, one or more read-only memories, one or more caches, one or more hard disk drives, one or more solid-state drives, one or more non-volatile memory host controller interface specification memories, one or more optical drives, one or more universal serial bus flash drives, one or more external hard disks, one or more network-connected storage devices, one or more cloud storage instances, one or more tape drives, etc.

[0072] As described above, the memory 926 may store an anomaly detection / analysis service module 928, for example, as one or more computer-executable instruction sets for implementing methods to identify one or more anomalies. The anomaly detection / analysis service module 928 can be implemented using any suitable computer programming language (e.g., Python, JavaScript, C, C++, Rust, C#, Swift, Java, Go, LISP, Ruby, Fortran, etc.). The anomaly detection / analysis service module 928 may include one or more sub-modules, including a machine learning module 930.

[0073] The machine learning module 930 may include instructions for detecting anomalies in a dataset created from data collected from the storage computing device 902. In some implementations, the machine learning module 930 may include instructions for preprocessing the dataset to prepare input for a machine learning model. The machine learning module 930 may also include instructions for training and operating the machine learning model. For example, as described above, this technology may include training an autoencoder in real time and using the autoencoder to detect anomalies in a dataset created from data collected from the storage computing device 902.

[0074] The memory 926 may store a pattern engine 932, such as a set of one or more computer-executable instructions for generating and executing instructions for anomaly detection on an updated dataset (i.e., a pattern). The pattern engine 932 can be implemented using any suitable computer programming language, such as Python, JavaScript, C, C++, Rust, C#, Swift, Java, Go, LISP, Ruby, Fortran, etc. The pattern engine 932 can communicate with the anomaly detection / analysis service module 928 to instruct it to detect anomalies on the dataset.

[0075] In some examples, server 904 also includes one or more databases 906 or other data storage facilities (one or more of hard disk drives, optical storage drives, solid-state storage devices, CDs, CD-ROMs, DVDs, Blu-ray discs, redundant arrays of independent disks, data repositories, etc.), or is otherwise communicatively coupled to the aforementioned databases or data storage facilities. In some examples, database 906 may be a cloud database accessible via cloud API 914.

[0076] Server 904 can communicate with user equipment 908. User equipment can be a smartphone, tablet, desktop computer, or similar device. User equipment can be used to interact with server 904. Each user equipment 908 may include a processor and memory (not shown), the memory containing instructions (e.g., instructions corresponding to an application program) that, when executed, cause information received from server 904 (e.g., detected anomalies) to be displayed on user equipment 908.

[0077] In some examples, server 904 may communicate with external task management system 910 via network 912. External task management system 910 may include instructions for causing user device 908 to display information associated with anomalies. External task management system 910 may be implemented on another server (not shown). In some examples, external task management system 910 may be a cloud application. External task management system 910 may include one or more APIs (not shown) for enabling one or more other components within environment 900 to access the functionality of external task management system 910, such as receiving pending matters (i.e., tasks) from other components within environment 900.

[0078] In some embodiments, user equipment 902 and / or server 904 may offload some or all of their respective functions to one or more cloud APIs 914. In various aspects, one or more cloud APIs 914 may include one or more public clouds, one or more private clouds, and / or one or more hybrid clouds. One or more cloud APIs 914 may include one or more resources provided under one or more service models, such as Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), and Function as a Service (FaaS). For example, one or more cloud APIs 914 may include one or more cloud computing resources, such as compute instances, electronic databases, operating systems, email resources, etc. One or more cloud APIs 914 may include distributed computing resources capable of enabling task communication, for example, between server 904, database 906, and task management system 910. In some aspects, one or more cloud APIs 914 may include APIs such as GCP cloud schedulers, GCP Pub / Sub, etc.

[0079] Additional Notes

[0080] The various embodiments described above can be combined to provide further embodiments. All U.S. patents, U.S. patent application publications, U.S. patent applications, foreign patents, foreign patent applications, and non-patent publications mentioned in this specification and / or listed in the Application Data Sheet are incorporated herein by reference in their entirety for all purposes. If necessary, implementations of embodiments can be modified to incorporate concepts from various patents, applications, and publications to provide further embodiments.

[0081] Based on the detailed description above, these and other changes can be made to the embodiments. Generally, the terminology used in the appended claims should not be construed as limiting the claims to the specific embodiments disclosed in the specification and claims, but should be interpreted to include all possible embodiments and the full scope of equivalents conferred by such claims. Therefore, the claims are not limited to this disclosure.

[0082] The following considerations also apply to the foregoing discussion. Throughout this specification, multiple instances can implement operations or structures described as a single instance. Although the individual operations of one or more methods are illustrated and described as separate operations, one or more of the operations can be performed simultaneously, and they do not need to be performed in the illustrated order. These and other variations, modifications, additions, and improvements fall within the scope of this document.

[0083] It should also be understood that unless a term is explicitly defined in this patent using phrases such as “as used herein,” “this is hereby defined as meaning…” or similar statements, there is no intention to limit the meaning of a term, expressly or impliedly, beyond its ordinary or common meaning, and the term should not be construed as being limited in scope based on any statement made in any part of this patent (other than the language of the claims). Where any term recited in the claims at the end of this patent is referenced in this patent in a manner consistent with a single meaning, this is only for clarity so as not to confuse the reader and is not intended to limit such claim term to that single meaning by implication or otherwise. Finally, unless a claim element is defined by reciting the words “device” and “function” without describing any structure, the scope of any claim element is not intended to be interpreted based on the application of Section 112(f) of Title 35 of the United States Code.

[0084] Unless otherwise specifically stated, discussions using terms such as “processing,” “computing,” “calculating,” “determining,” “presenting,” and “displaying” in this document may refer to the actions or processes of a machine (e.g., a computer) that manipulate or transform data represented as physical (e.g., electronic, magnetic, or optical) quantities within one or more memories (e.g., volatile memory, non-volatile memory, or a combination thereof), registers, or other machine parts that receive, store, transmit, or display information.

[0085] As used herein, any reference to "one implementation" or "an implementation" means that a particular element, feature, structure, or characteristic described in connection with that implementation is included in at least one implementation. The phrase "in one implementation" appearing in different places in the specification does not necessarily refer to the same implementation.

[0086] As used herein, the terms “comprises,” “comprising,” “includes,” “including,” or any other variation thereof are intended to cover non-exclusive inclusion. For example, a process, method, article of manufacture, or apparatus that includes a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such process, method, article of manufacture, or apparatus. Furthermore, unless the contrary is expressly stated, “or” refers to an inclusive or, not an exclusive, or. For example, condition A or B is satisfied if A is true (or exists) and B is false (or does not exist), A is false (or does not exist) and B is true (or exists), and both A and B are true (or exist).

[0087] Additionally, the terms "a" or "an" are used to describe elements and components of the implementations herein. This is done merely for convenience and to give the general meaning of the invention. The description should be understood to include one or at least one, and the singular includes the plural, unless it is obvious otherwise.

[0088] Upon reading this disclosure, those skilled in the art will understand additional alternative structural and functional designs for implementing the concepts disclosed herein through the principles disclosed herein. Therefore, while specific implementations and applications have been illustrated and described, it should be understood that the disclosed implementations are not limited to the precise constructions and components disclosed herein. Various modifications, alterations, and variations, which will be apparent to those skilled in the art, may be made to the arrangement, operation, and details of the methods and apparatus disclosed herein without departing from the spirit and scope defined by the appended claims.

Claims

1. A method for anomaly detection, the method comprising: Receive a set of data parameters via one or more processors; The dataset corresponding to the set of data parameters is retrieved from the database via the one or more processors. The dataset is analyzed via the one or more processors and using a machine learning model trained in real time to detect one or more anomalies in the dataset; The one or more processors select a set of anomaly parameters corresponding to the detected one or more anomalies; The output of the machine learning model is filtered based on the set of abnormal parameters via the one or more processors. The one or more processors generate a set of instructions for identifying one or more anomalies based on the data parameter set, the anomaly parameter set, and the detection mode parameter set. The set of instructions for identifying anomalies is executed via the one or more processors to identify one or more anomalies in the dataset in real time in response to an update of the dataset; as well as Information about the one or more exceptions is transmitted to a user computing device or another computing device via the one or more processors.

2. The method as described in claim 1, wherein, The machine learning model is an autoencoder neural network.

3. The method of claim 2, further comprising: The autoencoder neural network is trained in real time by providing it with data corresponding to one or more anomalies in the dataset.

4. The method of claim 3, further comprising: The autoencoder neural network is trained in real time by providing one or more predefined rules to it.

5. The method of claim 4, wherein, The set of instructions used to identify exceptions is obtained from an external server that is accessible via a network by the one or more processors.

6. The method of claim 1, wherein, The machine learning model is an unsupervised neural network, and the method further includes: training the unsupervised neural network in real time by providing the unsupervised neural network with data corresponding to one or more anomalies in the dataset.

7. The method of claim 1, wherein, The set of detection mode parameters includes one or more of the following: (i) a time frame indicating which value should be included in the second dataset, (ii) a scheduling plan for further anomaly detection, (iii) one or more prescriptive measures associated with the one or more anomalies, (iv) a security level associated with the one or more anomalies, and / or (v) a level of responsibility associated with the one or more anomalies.

8. The method of claim 7, wherein, The information regarding the one or more anomalies includes one or more of the following: (i) an explanation of the anomalies affecting the one or more anomalies, and / or (ii) prescriptive measures for correcting the one or more anomalies.

9. The method of claim 7, wherein, Transmitting the information about the one or more anomalies to the user interface includes transmitting the information based on one or more of (i) the security level and / or (ii) the responsibility level.

10. The method of claim 9, wherein, Transmitting the information about the one or more anomalies to the user equipment or the other computing device includes: Identify at least one data category associated with the one or more anomalies; and The security level and / or the responsibility level are identified based on the at least one data category.

11. The method of claim 9, wherein, Transmitting the information about the one or more anomalies to the user equipment or the other computing device includes: The information identifies scheduling plan data, which includes normative measures for correcting the anomalies and an identifier of an external task management system for receiving the normative measures.

12. The method of claim 11, further comprising: The prescriptive measures are transmitted from the one or more processors to the external task management system, wherein the prescriptive measures do not contain data for identifying the one or more anomalies.

13. The method of claim 1, wherein, The abnormality parameters include one or more of the following: (i) an indication of an anomaly, (ii) an anomaly score, and / or (iii) the first and second principal components of the principal component analysis.

14. The method of claim 1, further comprising: The dataset is analyzed via one or more processors to detect one or more anomalies in the dataset by applying one or more predefined rules.

15. The method of claim 1, wherein, Filtering the output of the machine learning model based on the set of anomaly parameters includes applying predetermined anomaly parameter rules in real time.

16. The method of claim 1, wherein, The filtering of the output of the machine learning model is performed based on explanatory data characterizing the one or more anomalies.

17. The method of claim 16, wherein, The explanatory data was generated by a trained machine learning model.

18. A system for anomaly detection, the system comprising: One or more processors, and One or more memories having computer-executable instructions stored thereon, which, when executed by the one or more processors, cause the computing system to: Receive data parameter set; Retrieve the dataset corresponding to the set of data parameters from the database; The dataset is analyzed using a machine learning model trained in real time to detect one or more anomalies in the dataset; Select a set of anomaly parameters corresponding to the one or more anomalies detected; The output of the machine learning model is filtered based on the set of abnormal parameters; Based on the data parameter set, the anomaly parameter set, and the detection mode parameter set, a set of instructions for identifying one or more anomalies is generated. Execute the set of instructions for identifying anomalous items to identify one or more anomalous items in the dataset in real time in response to an update of the dataset; as well as Transmit information about the one or more anomalies to a user's computing device or another computing device.

19. The system of claim 18, wherein, The machine learning model is an autoencoder neural network.

20. The system of claim 19, wherein the one or more memories have computer-executable instructions stored thereon, which, when executed by the one or more processors, further cause the computing system to: The autoencoder neural network is trained in real time by providing it with data corresponding to one or more anomalies in the dataset.

21. The system of claim 20, wherein the one or more memories have computer-executable instructions stored thereon, which, when executed by the one or more processors, further cause the computing system to: The autoencoder neural network is trained in real time by providing one or more predefined rules to it.

22. The system of claim 21, wherein, The set of instructions used to identify exceptions is obtained from an external server that is accessible via a network by the one or more processors.

23. The system of claim 18, wherein, The machine learning model is an unsupervised neural network, and the method further includes: training the unsupervised neural network in real time by providing the unsupervised neural network with data corresponding to one or more anomalies in the dataset.

24. The system of claim 18, wherein, The set of detection mode parameters includes one or more of the following: (i) a time frame indicating which value should be included in the second dataset, (ii) a scheduling plan for further anomaly detection, (iii) one or more prescriptive measures associated with the one or more anomalies, (iv) a security level associated with the one or more anomalies, and / or (v) a level of responsibility associated with the one or more anomalies.

25. The system of claim 24, wherein, The information regarding the one or more anomalies includes one or more of the following: (i) an explanation of the anomalies affecting the one or more anomalies, and / or (ii) prescriptive measures for correcting the one or more anomalies.

26. The system of claim 24, wherein, Transmitting the information about the one or more anomalies to the user equipment or the other computing device includes transmitting the information based on one or more of (i) the security level and / or (ii) the responsibility level.

27. The system of claim 26, wherein, Transmitting the information about the one or more anomalies to the user equipment or the other computing device includes: Identify at least one data category associated with the one or more anomalies; and The security level and / or the responsibility level are identified based on the at least one data category.

28. The system of claim 26, wherein, Transmitting the information about the one or more anomalies to the user interface includes: The information identifies scheduling plan data, which includes normative measures for correcting the anomalies and an identifier of an external task management system for receiving the normative measures.

29. The system of claim 28, wherein the one or more memories have computer-executable instructions stored thereon, which, when executed by the one or more processors, further cause the computing system to: The normative action is transmitted to the external task management system, wherein the normative action does not contain data for identifying the one or more anomalies.

30. The system of claim 18, wherein, The abnormality parameters include one or more of the following: (i) an indication of an anomaly, (ii) an anomaly score, and / or (iii) the first and second principal components of the principal component analysis.

31. The system of claim 18, wherein the one or more memories have computer-executable instructions stored thereon, which, when executed by the one or more processors, further cause the computing system to: Analyze the dataset to detect one or more anomalies in the dataset by applying one or more predefined rules.

32. The system of claim 18, wherein, Filtering the output of the machine learning model based on the set of anomaly parameters includes applying predetermined anomaly parameter rules in real time.

33. The system of claim 18, wherein, The filtering of the output of the machine learning model is performed based on explanatory data characterizing the one or more anomalies.

34. The system of claim 33, wherein, The explanatory data was generated by a trained machine learning model.

Images